Information providing method, storage medium and information providing device

Abstract

This invention provides an information provision method, storage medium, and information provision device that improve user convenience. The information provision method causes a computer of a terminal device equipped with a camera and installed on a mobile object to perform the following processing: acquiring an image; determining, based on the surrounding conditions of the mobile object obtained from the image, whether the level of alertness for the operator of the mobile object is a first level of alertness or a second level of alertness that is lower than the first level of alertness; and, based on the result of the determination, causing a display unit on the housing of the terminal device, on which the camera is mounted, to display information corresponding to the level of alertness.

Description

Technical Field

[0001] This invention relates to information provision methods, storage media, and information provision devices. Background Technology

[0002] Previously, a vehicle equipped with a camera and an alarm device was known, which outputs an alarm to the alarm device based on the processing results of the image captured by the camera (Japanese Patent Application Publication No. 2019-96056). Summary of the Invention

[0003] However, some of the aforementioned technologies are not very convenient for users of mobile devices.

[0004] The present invention was made in consideration of such circumstances, and one of its objectives is to provide an information provision method, storage medium, and information provision device that can improve the convenience of the user.

[0005] Solution for solving the problem

[0006] The information providing method, storage medium, and information providing device involved in this invention adopt the following structure.

[0007] (1): The information providing method causes a computer of a terminal device equipped with a camera and installed on a mobile body to perform the following processing: acquiring an image; determining, based on the surrounding conditions of the mobile body obtained from the image, whether the level of alertness for the operator of the mobile body is a first level of alertness or a second level of alertness that is lower than the first level of alertness; and, based on the result of the determination, causing a display unit provided on the housing on which the camera is provided of the terminal device to display information corresponding to the level of alertness.

[0008] (2): The information providing method causes a computer of a terminal device equipped with a camera and installed on a mobile body to perform the following processing: acquire an image; determine, based on the surrounding conditions of the mobile body obtained from the image, whether the level of alertness for the operator of the mobile body is above a predetermined level; and if the level of alertness is above a predetermined level, cause the display unit of the terminal device provided on the housing on which the camera is provided to display information indicating that the level of alertness is above a predetermined level, and cause the light-emitting unit to provide an overtaking instruction.

[0009] (3): Based on the above (1) or (2) scheme, the computer further performs the following processing: determining the direction that the operator should be alerted to based on the surrounding conditions; and causing the display unit to display an image indicating the direction of the alert.

[0010] (4): Based on any of the above schemes (1) to (3), the computer further performs the following processing: when the alert level is a first alert level, the display unit displays an image of a first color; and when the alert level is a second alert level that is less than the first alert level, the display unit displays an image of a second color that is different from the first color.

[0011] (5): Based on any of the above schemes (1) to (4), the computer further performs the following processing: based on the surrounding conditions, it determines whether the alert level is a first alert level, a second alert level which is less than the first alert level, or a third alert level which is less than the second alert level; and based on the result of the determination, it causes the display unit to display information corresponding to the alert level.

[0012] (6): Based on any of the schemes in (1) to (5) above, the computer further performs the following processing: if the traffic participant of the object being observed is within a first distance from the terminal device, or is predicted to be within the first distance for a specified time from the current moment, the alert level is determined to be a first alert level; if the traffic participant is within a second distance from the terminal device that is farther than the first distance, or is predicted to be within the second distance for a specified time from the current moment, the alert level is determined to be a second alert level that is smaller than the first alert level.

[0013] (7): Based on any of the above schemes (1) to (6), the computer further performs the following processing: when a traffic participant is present or is predicted to be present in a warning area, the display unit displays the information in a manner that is at least a predetermined level of warning, the warning area being a region of a predetermined angle based on a reference direction; when the moving vehicle is traveling straight, the warning area is set in front of the moving vehicle; when the moving vehicle turns left, the warning area is set in front of the moving vehicle and to the left; when the moving vehicle turns right, the warning area is set in front of the moving vehicle and to the right.

[0014] (8): Based on the above (7) scheme, the computer further performs the following processing: when a traffic participant exists or is predicted to exist in areas a, b, and c, the alert level is determined to be a first alert level, and the display unit displays information about the first alert level corresponding to the area where the traffic participant exists or is predicted to exist; and when a traffic participant exists or is predicted to exist in areas d, e, and f, the alert level is determined to be a second alert level lower than the first alert level, and the display unit displays information about the second alert level corresponding to the area where the traffic participant exists or is predicted to exist, where area a is a range of a first angle based on a reference direction and is a range of a first distance from the terminal device, and area b is a range of the leftward direction of the reference direction. Region c is the range between the first angle and a second angle larger than the first angle relative to the reference direction, and is also the range of the first distance; region d is the range between the first angle and the second angle relative to the reference direction, and is also the range between the first distance and a second distance farther from the terminal device than the first distance; region e is the range between the first angle and the second angle, and is also the range between the first distance and the second distance, and is also the range between the first distance and the second distance, and is also the range between the first distance and the second distance, and is also the range between the first angle and the second angle, and is also the range between the first distance and the second distance, and is also the range between the first angle and the second distance ... angle and the second distance, and is also the range between the first distance and the second distance, and is also the range between the first angle and the second distance, and is also the range between the first distance and the second distance, and is also the range between the first angle and the second distance, and is also the range between the first angle and the second distance, and is also the range between the first distance and the second distance, and is also the range between the first angle and the second distance, and is also the range between the first angle and the second distance, and is also the range between the first angle and the second distance, and is also the range between the first angle and the second distance, and is also the range between the first angle and the second distance, and is also the range between the first angle and the second distance, and is also the range between the first angle and the second distance, and is also the range between the first angle and the second distance, and is also the range between the first angle and the second distance,

[0015] (9): Based on the above scheme (8), the computer further performs the following processing: when a traffic participant is present or is predicted to be present in area a, area b or area c, the display unit displays information indicating that the moving body is under alert in the forward direction, left direction or right direction respectively at the first level of alert; and when a traffic participant is present or is predicted to be present in area d, area e or area f, the display unit displays information indicating that the moving body is under alert in the forward direction, left direction or right direction respectively at the second level of alert.

[0016] (10): Based on any of the above schemes (1) to (9), the computer further performs the following processing: based on the surrounding conditions, it determines whether the alert level is a first alert level, a second alert level which is less than the first alert level, or a third alert level which is less than the second alert level; and if there is a column of objects that are present near the moving body and connected in the same direction as the moving body's direction of travel, it determines that it is the third alert level.

[0017] (11): Based on the above (10) scheme, the computer further performs the following processing: when the distance between the connected objects is below a first threshold and the lateral distance between the connected objects and the moving body is below a second threshold, it is determined to be the third alert level.

[0018] (12): Based on any of the above schemes (1) to (11), the computer further performs the following processing: based on the surrounding conditions, it determines whether the alert level is a first alert level, a second alert level which is less than the first alert level, or a third alert level which is less than the second alert level; and when the moving body traveling on the first road enters the second road connected to the first road, if there are objects in front of or behind the connection between the first road and the second road in the extension direction of the first road, it is determined to be the third alert level.

[0019] (13): Based on the above (12) scheme, the computer further performs the following processing: if the distance between the object about to enter the connection part and the object behind the object is below the third threshold, it is determined to be the third alert level.

[0020] (14): Based on any of the above schemes (1) to (9), the computer further performs the following processing: according to the level of alert, the light-emitting part controlled by the computer gives an overtaking instruction.

[0021] (15): Based on the above (14) scheme, the light-emitting part is disposed on the mobile body and is able to communicate with the computer. The mobile body is a two-wheeled vehicle, so that the computer also performs the following processing: communicating with the light-emitting part to make the light-emitting part give an overtaking instruction.

[0022] (16): Based on the above scheme (14) or (15), the computer further performs the following processing: when the alert level is a first alert level, compared with the case where the alert level is a second alert level which is lower than the first alert level, the cycle of the overtaking instruction is shortened.

[0023] (17): Based on any of the above schemes (14) to (16), the computer further performs the following processing: based on the surrounding conditions, it determines whether the alert level is a first alert level, a second alert level which is less than the first alert level, or a third alert level which is less than the second alert level; if there is a line of objects near the moving body that are connected in the same direction as the moving body's direction of travel, it is determined to be the third alert level; or if the moving body traveling on the first road enters the second road connected to the first road, and there are objects in front of or behind the connection between the first road and the second road in the extension direction of the first road, it is determined to be the third alert level; if it is the first alert level or the second alert level, the display unit displays the information and performs the overtaking instruction; and if it is the third alert level, the overtaking instruction is not performed but the display unit displays the information.

[0024] (18): An application program stored in the storage medium of one aspect of the present invention causes the computer of the terminal device to perform the following processing: acquiring an image; determining, based on the surrounding conditions of the moving object obtained from the image, whether the level of alertness for the operator of the moving object is a first level of alertness or a second level of alertness that is lower than the first level of alertness; and, based on the result of the determination, causing the display unit to display information corresponding to the level of alertness.

[0025] (19): An information providing device according to one aspect of the present invention includes: a camera disposed on a housing; a display unit disposed on the housing; a determination unit that determines, based on the condition of the surroundings of a moving object obtained from an image captured by the camera, whether the level of alertness for an operator of the moving object is a first level of alertness or a second level of alertness that is lower than the first level of alertness; and a display control unit that, based on the determination result, causes the display unit disposed on the housing on which the camera is disposed to display information corresponding to the level of alertness.

[0026] Invention Effects

[0027] According to (1)-(19), the information providing method, the application program or terminal device stored in the storage medium enables the display unit of the terminal device, which is provided on the housing with the camera, to display information corresponding to the level of alert, thereby improving the convenience of the user.

[0028] According to (2), the information providing method causes the display unit to display information indicating that the warning level is above a specified level, and gives an overtaking instruction, thereby enabling the operator of the moving body and the surrounding area of ​​the moving body to be reported.

[0029] According to (3), the information providing method enables the display unit to show the direction that the operator should be wary of, thereby enabling the operator to easily identify the direction that should be wary of and to take action to prevent approaching the object.

[0030] According to (4), the information providing method enables the display unit to display information corresponding to the level of alert, thereby enabling the user to easily identify the level of alert.

[0031] According to (7), the information providing method sets an appropriate warning zone based on the direction of travel of the moving body, thereby enabling the display unit to display information corresponding to a more appropriate level of warning.

[0032] According to (8) or (9), the information providing method enables the display unit to display information corresponding to the area that should be warned, thereby providing the operator with further useful information.

[0033] According to (10) or (11), the information provision method, for example, in the case of traffic congestion forming a line of vehicles, determines the likelihood of a traffic participant approaching from between vehicles, thereby informing the user of the likelihood of the traffic participant approaching. As a result, the user can take actions to avoid approaching a traffic participant who is likely to approach.

[0034] According to (12) or (13), the information provision method, for example, in the case of traffic congestion near an intersection forming a line of vehicles, determines the likelihood of a traffic participant approaching laterally from the congested vehicles, thereby informing the user of the likelihood of the traffic participant approaching. As a result, the user can take actions to avoid approaching a traffic participant who is likely to approach.

[0035] According to (14)-(17), the information provision method provides overtaking instructions based on the level of alert, thereby enabling surrounding traffic participants to identify the presence of the moving body. Attached Figure Description

[0036] Figure 1 It is a diagram used to illustrate the overview of the support system.

[0037] Figure 2 This is a diagram illustrating an example of a scenario related to support.

[0038] Figure 3 This is another example of a scenario related to support.

[0039] Figure 4 This is another example of a scenario related to support.

[0040] Figure 5 This is another example of a scenario related to support.

[0041] Figure 6 This is a diagram illustrating an example of the functional structure of a support system.

[0042] Figure 7 This diagram shows an example of a terminal device installed on this vehicle.

[0043] Figure 8 This is a diagram illustrating an example of the structure of a terminal device, including its functional components.

[0044] Figure 9 This is a flowchart illustrating an example of a process executed by an application (terminal device).

[0045] Figure 10 This is an example of an image displayed on the display section of a terminal device.

[0046] Figure 11 This is a diagram used to illustrate the processing related to the estimation of the amount of movement.

[0047] Figure 12 This diagram illustrates the processing related to posture determination.

[0048] Figure 13 This is a flowchart illustrating an example of a processing flow performed by the processing department.

[0049] Figure 14 This diagram illustrates the process of detecting traffic participants.

[0050] Figure 15 This is a diagram used to illustrate the process of predicting the actions of detected traffic participants.

[0051] Figure 16 This is a diagram illustrating an example of the definition of risk when traveling straight.

[0052] Figure 17 This diagram illustrates an example of how to handle the risk of a vehicle traveling straight.

[0053] Figure 18 This is a diagram illustrating an example of the definition of risk when turning left.

[0054] Figure 19 This diagram illustrates an example of how to handle the risk of a vehicle making a left turn.

[0055] Figure 20 This is a diagram illustrating an example of the definition of the risk when turning right.

[0056] Figure 21 This diagram illustrates an example of how to handle the risk of this vehicle turning right.

[0057] Figure 22 This is a diagram illustrating an example of the treatment of presumed potential risks.

[0058] Figure 23 This is a flowchart illustrating an example of a processing flow performed by the processing department.

[0059] Figure 24 This is a flowchart illustrating an example of a process related to alerts / attention.

[0060] Figure 25 It is a diagram used to illustrate the scenarios and notifications that generate forward-looking warning risks.

[0061] Figure 26 It is a diagram used to illustrate the scenarios and notifications that generate left-side or right-side alarm risks.

[0062] Figure 27 It is a diagram used to illustrate scenarios and notifications that generate risk awareness.

[0063] Figure 28 It is a diagram used to illustrate scenarios and notifications that may generate potential risks. Detailed Implementation

[0064] Hereinafter, the storage medium, information providing method and terminal device involved in the embodiments of the present invention will be described with reference to the accompanying drawings.

[0065] Hereinafter, the information providing method, storage medium and information providing device involved in the embodiments of the present invention will be described with reference to the accompanying drawings.

[0066] [Summary 1]

[0067] Figure 1 This diagram illustrates the general outline of support system 1. In this embodiment, as an example, a two-wheeled vehicle (hereinafter, sometimes referred to as "this vehicle") is described, but it can also be applied to various other mobile bodies such as light vehicles and three-wheeled vehicles. Hereinafter, the forward / backward direction of this vehicle is sometimes referred to as the X direction, the direction orthogonal to the forward / backward direction is referred to as the Y direction, and the direction orthogonal to both the X and Y directions is referred to as the Z direction.

[0068] Support system 1 includes a passing indicator light PL, a terminal device (e.g., a smartphone) SP, and an application app (AP) installed on the terminal device SP. The passing indicator light PL is installed, for example, around the headlights, in a manner that illuminates in the positive X direction (forward direction) (passing indication). The passing indicator light PL-R within the passing indicator light PL is installed in the positive Y direction of the vehicle, and the passing indicator light PL-L within the passing indicator light PL is installed in the negative Y direction of the vehicle.

[0069] The terminal device SP has a camera and a light mounted on its first side, and a display unit on its second side, opposite to the first side. The terminal device SP is installed in the vehicle. It is positioned such that the camera can capture images of the vehicle in the X-direction, and the light illuminates in the X-direction (overtaking indicator). The terminal device SP is positioned so that the driver (operator) can visually recognize the display unit.

[0070] The support system 1 uses one or both of the hazard lights and the overtaking indicator light PL of the terminal device SP to instruct traffic participants to overtake. The support system 1 further causes the display unit of the terminal device SP to display an alarm. Hereinafter, the two aspects of instructing overtaking and displaying an alarm will sometimes be referred to as "support".

[0071] For example, two-wheeled vehicles tend to have long braking distances, and therefore tend to have a low ability to avoid approaching other road users. In order to suppress approaching other road users, support system 1 reports the approach to the driver in advance, conveys the possibility of approach to other road users through automatic overtaking instructions via overtaking indicator lights, and entrusts cooperation in suppressing the approach (deceleration, stopping, avoidance, etc.).

[0072] Support System 1, such as Figure 2 The system provides support to prevent head-on collisions. For example, support system 1 provides support to avoid the vehicle from approaching pedestrians, two-wheeled vehicles, or four-wheeled vehicles.

[0073] Support System 1, such as Figure 3 The system shown provides support to suppress approaching traffic participants at intersections. Support system 1 is as follows: Figure 4 The indicated method is to provide support for suppressing the approach of a traffic participant when the vehicle is squeezed laterally past a four-wheeled vehicle in a four-wheeled vehicle congestion, or to provide support for suppressing the approach of a traffic participant who has stopped in front of it.

[0074] Support System 1, such as Figure 5 The system provides support to suppress approaching vehicles based on potential risks. For example, support system 1 suppresses the risk of the vehicle turning right in a four-wheeled vehicle congestion, or the risk of a traffic participant suddenly running from behind a parked vehicle. For example, in this embodiment, as... Figure 5As shown in the right figure, if there is a line of objects (four-wheeled vehicles) located near the vehicle and connected in the same direction as the vehicle's travel direction, it is determined that there is a potential risk. In this embodiment, as... Figure 5 As shown in the left diagram, if a vehicle traveling on the first road enters the second road connecting to the first road (by turning right from the first road), and there is an object (a four-wheeled vehicle) in front of or behind the junction (intersection) of the first road and the second road in the direction of extension of the first road, it is determined that there is a potential risk. The details are explained below.

[0075] [Functional Structure of the Support System]

[0076] Figure 6 This is a diagram illustrating an example of the functional structure of support system 1. Support system 1, for example, is as follows: Figure 1 As explained, it includes a terminal device SP and a passing indicator PL. For example, the passing indicator PL and the terminal device SP are connected to the vehicle M's battery BT via a power line and operate by receiving power from the battery BT. The terminal device SP can also operate by receiving power from the battery BT via the passing indicator PL, or it can operate by using power from the battery mounted on the terminal device SP.

[0077] The terminal device SP and the overtaking indicator PL are connected by a communication line and can communicate with each other. For example, the terminal device SP sends an execution signal to activate the overtaking indicator PL, and the overtaking indicator PL executes the overtaking instruction according to the execution signal.

[0078] Terminal device SP such as Figure 7 The device is mounted on the vehicle by a retainer. For example, the camera Cam mounted as the terminal device SP takes pictures in the positive X direction, the lamp Lig of the terminal device SP emits light in the positive X direction, and the display unit Dis faces the negative X direction.

[0079] Figure 8 This diagram illustrates an example of the functional structure of a terminal device SP. The terminal device SP, for example, includes an acquisition unit 10, a processing unit 20, a notification unit 30, and a storage unit 50. The acquisition unit 10, processing unit 20, and notification unit 30 are implemented, for example, by a hardware processor such as a CPU (Central Processing Unit) executing an application program (AP) stored in the storage unit 50. The application program AP is provided, for example, by a server device that provides the application program AP. The application program AP may also be provided, for example, by an operator that manufactures or sells this vehicle. The application program AP includes, for example, information that functions as a DNN (Deep Neural Network) 52; details about this DNN will be described later.

[0080] Some or all of the components of the aforementioned functional units can be implemented using hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit), including the circuitry unit, or through the coordinated use of software and hardware. The application program (AP) can also be pre-stored in a storage device such as a flash memory (a storage device with a non-transitory storage medium) in the storage unit 50, or it can be stored in a removable storage medium such as an SD card, and installed in the storage unit of the terminal device SP by mounting the storage medium (a non-transitory storage medium) onto the terminal device SP.

[0081] The acquisition unit 10 acquires images captured by the camera Cam. The processing unit 20 analyzes the images and performs various processing steps based on the analysis results. The notification unit 30 issues warnings such as alerts and cautions to the driver based on the processing results of the processing unit 20. Details regarding these processing steps will be described later.

[0082] [flow chart]

[0083] Figure 9 This is a flowchart illustrating an example of a process executed by an application (terminal device SP). First, the terminal device SP estimates the amount of movement (step S100) and estimates the posture (step S110). The amount of movement refers to the amount of movement of a traffic participant or the movement of the vehicle itself. The posture refers to the posture of the vehicle itself.

[0084] Next, the terminal device SP detects traffic participants (step S120) and predicts the actions of the detected traffic participants (step S130). Next, the terminal device SP detects the risk to its own vehicle when going straight (step S140), the risk when turning left (step S150), and the risk when turning right (step S160). Next, the terminal device SP detects potential risks (step S170). Next, the terminal device SP performs a decision-making process (step S180).

[0085] In the aforementioned determination process, the terminal device SP displays information based on the determination result. Figure 10 This refers to an alarm level report or attention level report as shown. Alarm level reports or attention level reports can be reports related to the front, left or left-front, right or right-front, etc. This concludes the processing of the first routine in this flowchart. The details of these processes are explained below.

[0086] [Estimation of movement]

[0087] Figure 11 This diagram illustrates the processing related to the estimation of movement. The acquisition unit 10 of the terminal device SP acquires images of a time series (k…kn) captured by the camera Cam ("images captured at different times") (step S101). Next, the processing unit 20 of the terminal device SP derives the difference between the image captured at the current time and the image captured i times ago (step S102). Then, the processing unit 20 binarizes the derived difference (step S104) and calculates the sum of all pixel values ​​(step S106). The sum of all pixel values ​​refers to the sum of pixels where a difference is determined to exist. For example, the processing unit 20 derives the difference in brightness values ​​for each pixel and binarizes the difference based on whether the derived difference is above or below a threshold. For example, pixels with a difference above the threshold are represented as black, and pixels with a difference below the threshold are represented as white. Differences in RGB components can also be used. In this case, binarization can also be performed for each pixel by comparing the value obtained by statistically processing the differences of each component with a threshold.

[0088] For example, when the vehicle is stationary (parked), the difference between the images is small (due to high similarity), so the whiteness increases when the difference is binarized. Conversely, when the vehicle is moving, the difference is large (due to low similarity), so the whiteness decreases when the difference is binarized.

[0089] Next, the processing unit 20 determines whether the sum of all pixel values ​​is below a threshold (step S108). If the sum is below the threshold (high whiteness), the processing unit 20 determines that the vehicle has stopped (F_RUN = 0) (step S110). If the sum is not below the threshold (low whiteness), the processing unit 20 determines that the vehicle is in motion or that the movement of surrounding traffic participants is high (F_RUN = 1) (step S110).

[0090] As described above, the terminal device SP can easily determine the status of the vehicle and the status of traffic participants based on images.

[0091] [Presumption of Posture]

[0092] Figure 12This diagram illustrates the processing related to posture determination. The processing unit 20 extracts a rectangular region AR1 from the image at time k in the time series image. This rectangular region AR1 is centered on the center of the image and is centered on a reference position (e.g., center of gravity) of the image. The processing unit 20 extracts a region from the image at time ki in the time series image that is the same as or approximately the region captured in the image at time k, and enlarges the image of the extracted region in a manner close to that of the image at time k. This processing is an example of a process that "makes the size of the extracted image obtained by extracting a specific region from the image captured at the first time close to the size of the image captured at the second time."

[0093] Regarding the image magnification described above, the magnification amount can be determined as follows. For example, when the vehicle's speed is obtained by a positioning device that measures the position of the terminal device SP, the processing unit 20 can magnify the image by the amount of movement at sampling time i, based on the vehicle's speed. For example, the processing unit 20 can magnify the image by referring to information that establishes a correspondence between speed and magnification. For example, the processing unit 20 can also magnify the image by a magnification corresponding to the speed when the speed is obtained as described above, and magnify the image by a predetermined value when the speed is not obtained.

[0094] Alternatively, instead of the magnification process described above, image processing can be used to adjust the image range at time k to match the image range at time k-1. For example, the processing unit 20 can also obtain an area including a specified landscape from the images at time k and k-1, and reduce the size of the image obtained from the image at time k that includes the aforementioned area to a size close to that of the image obtained from the image at time k-1 that includes the aforementioned area.

[0095] Next, the processing unit 20 acquires multiple images obtained by rotating the magnified image to the left and right by a predetermined angle from a reference position. For example, it acquires images obtained by rotating the image counterclockwise and clockwise around the reference position. The images are assigned parameters ID according to the degree of rotation. For example, it acquires images obtained by rotating to the left with parameters ID +1…+n and images obtained by rotating to the right with parameters ID -1…--n.

[0096] The processing unit 20 sets a rectangular region AR2 centered on the center of the image to which the parameter ID is assigned. Rectangular region AR2 is, for example, the region corresponding to rectangular region AR1. Rectangular region AR2 is a region whose reference position is set centered on the center of the image, regardless of image rotation.

[0097] Next, the processing unit 20 compares the image of the rectangular region AR1 at time k (an example of the "second image") with the image of the rectangular region AR1 assigned parameter ID (an example of the "first image") to derive the difference. Then, the processing unit 20 binarizes the derived differences and sums the total pixel values ​​for each image. Furthermore, the processing unit 20 selects the image with the smallest total pixel sum (the sum of pixels determined to have a difference) as its parameter ID.

[0098] At times k and ki, if the vehicle is traveling straight, choose... Figure 12 The parameter ID is "0". At times ki and k, if the vehicle is about to turn left, for example, by selecting... Figure 12 The parameter ID "+n" is used when the vehicle is about to turn right at times ki and k, for example, when selecting... Figure 12 The parameter ID is "-n".

[0099] Figure 13 This is a flowchart illustrating an example of the processing flow performed by the processing unit 20. First, the processing unit 20 determines whether the parameter ID is greater than the threshold + a (step S110). If the parameter ID is greater than the threshold + a, the image is rotating to the left, therefore, the processing unit 20 determines that the vehicle intends to turn left (step S112). The determination is "F_LTURN = 1" or "F_RTURN = 0".

[0100] If parameter ID is not greater than threshold + a, the processing unit 20 determines whether parameter ID is less than threshold - a (step S114). If parameter ID is less than threshold - a, the image is rotating to the right, therefore, the processing unit 20 determines that the vehicle intends to turn right (step S116). The determination is "F_LTURN = 0" or "F_RTURN = 1".

[0101] If parameter ID is not less than the threshold -a, the processing unit 20 determines that the vehicle intends to proceed straight (step S118). The determination is "F_LTURN=0" and "F_RTURN=0". Thus, step 1 of this flowchart ends.

[0102] As described above, the terminal device SP can accurately determine whether the vehicle intends to turn right, left, or go straight based on images captured at different times.

[0103] Alternatively, the processing unit 20 may replace the above-described processing (or further) determine the vehicle's intention to turn left, right, or go straight based on the detection results of an acceleration sensor mounted on the terminal device SP that can detect the tilt of the terminal device SP. For example, the processing unit 20 may also determine that the vehicle intends to turn in the direction in which the terminal device SP is tilted.

[0104] In the above example, we illustrated the case of preparing multiple images obtained by rotating each by a predetermined angle. However, we can also compare an image obtained by rotating to the left and an image obtained by rotating to the right with the image taken at time k, and determine that the vehicle is about to travel in the direction corresponding to the image with the smallest difference. In the above example, we illustrated the case of comparing the central portions of the images with each other, but we can also compare other areas instead of the central portions. Furthermore, the processing of adjusting the size of the image at time ki or time k in the time series images when the interval between the captured images is less than a predetermined value (e.g., the processing of enlarging the image at time k-1) can be omitted. If we can determine the vehicle's intention to turn right, turn left, or go straight with good accuracy, we can also omit the above processing.

[0105] [Inspecting traffic participants]

[0106] Figure 14 This diagram illustrates the process of detecting traffic participants. The processing unit 20 uses a DNN52 to determine the category of traffic participants in the image. The DNN52 is a learned model that, when given an input image, outputs information representing the category of traffic participants in the image and information used to determine the location of those traffic participants. The categories of traffic participants include, for example, four-wheeled vehicles, two-wheeled vehicles, bicycles, and pedestrians. As information for determining location, for example, a correspondence is established between the traffic participant and a detection box, which is a rectangular region. This detection box is, for example, a region defined in a way that includes the traffic participant.

[0107] [Predicting and handling the actions of detected traffic participants]

[0108] Figure 15 This diagram illustrates the process of predicting the actions of detected traffic participants. The processing unit 20 infers the direction of movement of traffic participants based on changes in their positions determined from images taken at different times, and predicts their actions based on the inference results.

[0109] When the processing unit 20 enlarges the detection box that corresponds to the traffic participant obtained from the image at time k compared to the detection box that corresponds to the traffic participant obtained from the image at time k-1, it determines that there is a tendency for the vehicle to approach the traffic participant. In this case, for example, the traffic participant is approaching the vehicle, or the vehicle is approaching the traffic participant.

[0110] When the processing unit 20 determines that the vehicle is moving away from the traffic participant if the detection box corresponding to the traffic participant obtained from the image at time k is reduced in size compared to the detection box corresponding to the traffic participant obtained from the image at time k-1, this indicates a tendency for the vehicle to move away from the traffic participant. In this case, for example, the traffic participant is moving away from the vehicle, or the vehicle is moving away from the traffic participant. The processing unit 20 estimates the amount of movement of the traffic participant based on the scaling or shrinking of the detection box.

[0111] If the position of the detection frame at time k moves to the right relative to the position of the detection frame at time k-1, the processing unit 20 determines that the traffic participant has moved to the right; if the position of the detection frame at time k moves to the left, the processing unit 20 determines that the traffic participant has moved to the left. The processing unit 20 estimates the amount of movement of the traffic participant based on the amount of movement of the detection frame.

[0112] As described above, the processing unit 20 estimates the direction and amount of movement of traffic participants and predicts their actions. Based on the detection results of the positions of traffic participants at time k-1 and time k, the processing unit 20 predicts the future positions of traffic participants.

[0113] [Handling of detected risks]

[0114] (Risks related to driving straight)

[0115] Figure 16 This diagram illustrates an example of the definition of risk when traveling straight. The risk of traveling straight (straight-ahead risk) is set within a defined range in front of the vehicle. The straight-ahead risk is set within a first angular range, based on a line segment (d1) in the vehicle's direction of travel or the road's extension direction (reference direction). The second angular range, smaller than the first angle, based on the line segment in the direction of travel, is the forward risk range. Within the range between the first and second angular ranges, the range to the left is the left-hand risk range, and the range to the right is the right-hand risk range. The range within the aforementioned risk range at a first distance from the vehicle is the warning level range, and the range at a second distance from the vehicle is the caution level range.

[0116] Hereinafter, the region encompassing the first angle range and the first distance, with the reference direction (direction of travel) as the reference, is sometimes referred to as "region a"; the region encompassing the range between the first angle and the second angle (which may also be an angle different from the second angle mentioned above, as described below) in the left direction, and the range encompassing the first distance, is referred to as "region b"; and the region encompassing the range between the first angle and the second angle in the right direction, and the range encompassing the first distance, is referred to as "region c" (see below). Figure 17 , Figure 19 , Figure 21 ).

[0117] Sometimes, the region between the first and second angles, and between the first and second distances, with the reference direction (direction of travel) as the reference, is called "region d"; the region between the first and second angles, and between the first and second distances, in the left direction is called "region e"; and the region between the first and second angles, and between the first and second distances, in the right direction is called "region f" (see below). Figure 17 , Figure 19 , Figure 21 .

[0118] Figure 17 This diagram illustrates an example of the processing for determining the risk of the vehicle proceeding straight. Based on the results of the aforementioned processing, the processing unit 20 determines whether the vehicle should proceed straight (step S300). If the vehicle is not proceeding straight, the process described later continues. Figure 19 The processing of step S400.

[0119] When the vehicle is traveling straight, the processing unit 20 determines which area among areas a-e the risk to the vehicle exists in. For example, in the above-described processing, the processing unit 20 determines which area among areas a-e the location (object location) or future location (object location) of the target traffic participant (e.g., the traffic participant located closest to the vehicle) is located in. In the figure, "★" represents the object location.

[0120] For example, if the object is located in region a, assign the flag "F_FR=1", if the object is located in region b, assign the flag "F_LR=1", and if the object is located in region c, assign the flag "F_RR=1".

[0121] For example, if the object is located in region d, assign the flag "F_FC = 1"; if the object is located in region d, assign the flag "F_LC = 1"; if the object is located in region e, assign the flag "F_RC = 1". For all other cases, assign the flag 0.

[0122] (Risks associated with turning left)

[0123] Figure 18 This diagram illustrates an example of the definition of risk when turning left. The risk of turning left (left turn risk) centers on the front left of the vehicle. While the risk of going straight is defined with the vehicle's direction of travel as the reference direction, the risk of turning left is defined using a line segment (dL) obtained by rotating the line segment in the vehicle's direction of travel to the left by a specified angle as the reference direction. The specified angle is, for example, an angle within the range of 30 to 60 degrees, determined according to the shape of the road.

[0124] Figure 19 This diagram illustrates an example of processing to determine the risk of the vehicle making a left turn. Based on the results of the above processing, the processing unit 20 determines whether the vehicle is making a left turn or has the intention to do so (step S400). If the vehicle is neither making a left turn nor has the intention to do so, it proceeds to... Figure 21 The processing step S500. If the vehicle is making a left turn or has the intention to make a left turn, the processing unit 20 determines which area of ​​area a-area e the risk of the vehicle exists in.

[0125] like Figure 17 As explained, in the relationship between object location and region, assign the flags "F_FR=1", "F_LR=1", "F_RR=1", "F_FC=1", "F_LC=1", "F_RC=1" or other flags = 0.

[0126] (Risks associated with turning right)

[0127] Figure 20 This diagram illustrates an example of the definition of risk when turning right. The risk of turning right (right turn risk) centers on the right front of the vehicle. While the risk of going straight is defined with the vehicle's direction of travel as the reference direction, the risk of turning right is defined using a line segment (dR) obtained by rotating the line segment in the vehicle's direction of travel to the right by a specified angle as the reference direction. The specified angle is, for example, an angle within the range of 30 to 60 degrees, depending on the shape of the road.

[0128] Figure 21 This diagram illustrates an example of processing to determine the risk of the vehicle making a right turn. Based on the results of the above processing, the processing unit 20 determines whether the vehicle is making a right turn or intends to do so (step S500). If the vehicle is neither making a right turn nor intends to turn right, the process proceeds to... Figure 23 The processing step S600. If the vehicle is making a right turn or has the intention to make a right turn, the processing unit 20 determines which area of ​​area a-area e the risk of the vehicle exists in.

[0129] like Figure 17 As explained, in the relationship between object location and region, assign the flags "F_FR=1", "F_LR=1", "F_RR=1", "F_FC=1", "F_LC=1", "F_RC=1" or other flags = 0.

[0130] As described above, the processing unit 20 sets a risk (warning zone) in front of the vehicle when the vehicle is traveling straight, sets a risk in front of the vehicle and to the left when the vehicle is turning left, and sets a risk in front of the vehicle and to the right when the vehicle is turning right, thereby setting an appropriate risk corresponding to the direction of travel of the vehicle.

[0131] [Estimation of potential risks]

[0132] Figure 22 This diagram illustrates an example of the process for presuming potential risks. The processing unit 20 detects the average inter-vehicle distance Lvv_ave and the lateral distance Lev based on the positions of other vehicles in the image. The average inter-vehicle distance Lvv_ave is, for example, the detected average inter-vehicle distance from the current vehicle to a specified number of vehicles (e.g., all vehicles) located on the side of other vehicles. Other vehicles refer to vehicles located at positions closest to the current vehicle. The inter-vehicle distance is, for example, the distance between the reference positions of vehicles forming a column in the direction of travel. When there is only one vehicle, the inter-vehicle distance is set to a value equal to a pre-set distance (e.g., 30m).

[0133] Lateral distance Lev is the lateral distance between this vehicle and other vehicles. Other vehicles, for example, are those located at the closest possible distance to this vehicle. The average inter-vehicle distance Lvv_ave and lateral distance Lev mentioned above are used in the following treatments related to the assumption of potential risks.

[0134] Figure 23 This is a flowchart illustrating an example of the processing flow performed by the processing unit 20. The processing unit 20 determines whether the average inter-vehicle distance Lvv_ave is equal to the threshold Lvv_LR (step S600). The average inter-vehicle distance Lvv_ave is, for example, the longitudinal distance between the first object about to enter the intersection (the connection between the first road and the second road) and the second object behind that first object, or the average inter-vehicle distance based on the inter-vehicle distances from the first object to the nth object located behind the second object. In step S600, it may also be determined whether other vehicles are stopped and whether the average inter-vehicle distance Lvv_ave is equal to the threshold Lvv_LR. To reduce the computational load in this process, the determination of whether other vehicles are stopped is omitted; if the average inter-vehicle distance Lvv_ave is equal to the threshold Lvv_LR, it is determined instead that other vehicles are stopped. If the average inter-vehicle distance Lvv_ave is not equal to the threshold Lvv_LR, proceed to... Figure 24 The processing of step S700.

[0135] When the average inter-vehicle distance Lvv_ave is equal to the threshold Lvv_LR, the processing unit 20 determines whether the vehicle is turning right or intends to turn right (step S602). If the vehicle is turning right or intends to turn right, the processing unit 20 determines that there is a potential risk in the left direction (step S604). If there is a potential risk in the left direction, the flag "F_LLR = 1" is set.

[0136] If the vehicle is not turning right or does not intend to turn right, the processing unit 20 determines whether the vehicle is turning left or intends to turn left (step S606). If the vehicle is turning left or intends to turn left, the processing unit 20 determines that there is a potential risk in the right direction (step S608). If there is a potential risk in the right direction, the flag "F_RLR=1" is set.

[0137] When this vehicle is making a left turn, there is a possibility that a traffic participant may approach the vehicle from the left. However, since the driver can identify potential approaching traffic participants by observing the gaps between vehicles before making the left turn, this focus is on traffic participants approaching from the right, which are difficult to identify. While the driver is looking to the left when making a left turn, the possibility that the driver's driving may become unstable due to shifting their face to that side cannot be ruled out. Therefore, to suppress this possibility, the driver focuses on the right.

[0138] If the vehicle is not making a left turn or does not intend to make a left turn, the processing unit 20 determines whether the lateral distance |Lev| is less than the threshold Lev_LR (step S610). If the lateral distance |Lev| is less than the threshold Lev_LR, the processing unit 20 determines that there is a potential risk in front of the vehicle (step S612). If there is a potential risk in front, the flag "F_FLR = 1" is set.

[0139] If the lateral distance |Lev| is not less than the threshold Lev_LR, the processing unit 20 determines that there is no potential risk ahead (step S614). In this case, the flags “F_LLR=0”, “F_RLR=0”, and “F_FLR=0” are set.

[0140] [Handling related to alarms / attention]

[0141] Figure 24 This is a flowchart illustrating an example of a process related to alarm / attention handling (determination processing). The processing unit 20 determines whether the vehicle is in motion (step S700). If the vehicle is not in motion, step 1 of this flowchart ends.

[0142] (Handling of alert-related risks)

[0143] When the vehicle is in motion, the processing unit 20 determines whether there is a forward warning risk (step S702). If there is no forward warning risk, the processing unit 20 determines whether there is a left turn warning risk (step S704). If there is no left turn warning risk, the processing unit 20 determines whether there is a right turn warning risk (step S706).

[0144] In the event of a forward warning risk, a left-hand warning risk, or a right-hand warning risk, the notification unit 30 displays a warning corresponding to the risk level (step S708), and further provides an overtaking instruction in the first manner (step S710). As described above... Figure 17 , Figure 19 , Figure 21 If the flags "F_FR=1 (signal indicating a forward warning risk)," "F_LR=1 (signal indicating a left-direction warning risk)," or "F_RR=1 (signal indicating a right-direction warning risk)" are set as described above, then steps S708 and S710 are executed. For details regarding these settings, please refer to the description below. Figure 25 , Figure 26 To illustrate.

[0145] (Handling related to risk awareness)

[0146] If there is no right-direction warning risk, the processing unit 20 determines whether there is a forward warning risk (step S712). If there is no forward warning risk, the processing unit 20 determines whether there is a left-direction warning risk (step S714). If there is no left-direction warning risk, the processing unit 20 determines whether there is a right-direction warning risk (step S716).

[0147] In cases where there is a risk of danger ahead, a risk of danger to the left, or a risk of danger to the right, the notification unit 30 displays the corresponding risk warning (step S718), and further provides a second-mode overtaking instruction (step S720). As described above... Figure 17 , Figure 19 , Figure 21 When the flags "F_FC = 1 (signal indicating forward danger)," "F_LC = 1 (signal indicating left-hand danger)," or "F_RC = 1 (signal indicating right-hand danger)" are set as described above, the processes in steps S718 and S720 are executed. For details regarding these, please refer to the description below. Figure 27 To illustrate.

[0148] (Handling of potential risks)

[0149] If there is no risk of danger to the right, the processing unit 20 determines whether there is a potential risk ahead (step S722). If there is no potential risk ahead, the processing unit 20 determines whether there is a potential risk to the left (step S724). If there is no potential risk to the left, the processing unit 20 determines whether there is a potential risk to the right (step S726).

[0150] If there is a potential risk ahead, a potential risk to the left, or a potential risk to the right, the notification unit 30 displays the information corresponding to the potential risk (step S728). As described above... Figure 17 , Figure 19 , Figure 21 If the flags "F_FLR = 1 (signal indicating potential risk ahead)," "F_LLR = 1 (signal indicating potential risk to the left)," or "F_RLR = 1 (signal indicating potential risk to the right)" are set as described above, then the process in step S728 is executed. For details regarding these flags, please refer to the description below. Figure 28 This concludes the explanation of the first routine in this flowchart.

[0151] [Notifications corresponding to the alert risk (Level 1)]

[0152] Figure 25 This diagram illustrates a scenario and notification that generates a forward warning risk. When a forward warning risk exists, the notification unit 30 causes the display unit to show warning information indicating a forward warning risk, further executing the overtaking instruction in the first manner. This warning display is a pre-defined display showing a forward-direction arrow (e.g., a red arrow). The first method involves illuminating the overtaking indicator light PL at a high cycle.

[0153] Figure 26 This diagram illustrates the scenario and notification of a left-hand or right-hand traffic warning risk. When a left-hand traffic warning risk exists, the notification unit 30 causes the display unit to show warning information indicating a left-hand traffic warning risk, further executing the overtaking instruction of the first method. This warning display is a prescribed display, such as a left-hand arrow (e.g., a red arrow), as shown in the diagram.

[0154] If there is a warning risk in the right direction, the notification unit 30 causes the display unit to display a warning message indicating a warning risk in the right direction, and further executes the overtaking instruction of the first method. This warning display is a prescribed display, such as showing a right-direction arrow (e.g., a red arrow), as illustrated.

[0155] [Notification corresponding to the risk awareness (Level 2 alert)]

[0156] Figure 27This diagram illustrates a scenario where a risk of danger arises and provides notification. When a risk of danger exists in the forward direction, the notification unit 30 causes the display unit to show a warning message indicating the presence of such a risk, further executing the overtaking instruction in a second manner. This warning display is a pre-defined display, such as showing a forward-direction arrow (e.g., a yellow arrow). The second method involves illuminating the overtaking indicator light PL at a low cycle.

[0157] If there is a risk of danger to the left, the notification unit 30 causes the display unit to display a warning message indicating a risk of danger to the left, and further executes the overtaking instruction in the second manner. This warning display is a pre-defined display, such as a left-hand arrow (e.g., a yellow arrow), as shown in the illustration.

[0158] If a risk of danger exists to the right, the notification unit 30 causes the display unit to show a warning message indicating a risk of danger to the right, and further executes the overtaking instruction in the second manner. This warning display is a pre-defined display showing a right-direction arrow (e.g., a yellow arrow), as illustrated. As described above, the color of the image displayed on the display unit when a risk of danger exists (an arrow or other warning image) is different from the color of the image displayed on the display unit when a warning risk exists.

[0159] [Notification corresponding to potential risks (Level 3 Alert)]

[0160] Figure 28 This diagram illustrates a scenario where a potential risk arises and provides notification. When a potential risk exists in the forward direction, the notification unit 30 causes the display unit to show potential risk information indicating the presence of a potential risk in the forward direction. This potential risk display is a predefined display, such as showing an arrow pointing forward (e.g., a yellow arrow), as illustrated.

[0161] If there is a potential risk to the left, the notification unit 30 causes the display unit to show a warning message indicating a potential risk to the left. This potential risk display is a pre-defined display, such as a forward-direction arrow (e.g., a yellow arrow), as shown in the illustration. Even when a potential risk exists, sometimes there are no approaching traffic participants, and therefore the overtaking instruction is not executed.

[0162] The above example illustrates a scenario where the camera and display are located on the terminal device SP. However, it is also possible for one or both of the camera and display to be located on other devices or mobile bodies instead of the terminal device SP. In this case, the terminal device SP communicates with other devices or mobile bodies to acquire images or display information.

[0163] In the above embodiments, it was explained that the system determines whether it is a first alert level or a second alert level, and provides different information for the first and second alert levels. However, alternatively (or based on this), the application may not determine whether it is a first alert level or a second alert level, and, in the case of a situation requiring alert (where the alert level is above a predetermined level), display information related to the alert on the display unit. Furthermore, in this case, the application may also execute an overtaking instruction. A situation requiring alert refers to a situation of first alert level, a situation of second alert level, or a situation where there is a potential risk.

[0164] According to the above-described implementation method, the application causes the computer of the terminal device equipped with a camera and installed on the mobile body to perform the following processing: acquiring an image; determining, based on the surrounding conditions of the mobile body obtained from the image, whether the level of alertness for the operator of the mobile body is a first level of alertness or a second level of alertness that is lower than the first level of alertness; and, based on the determination result, causing the display unit provided on the housing on which the camera is installed on the terminal device to display information corresponding to the level of alertness, thereby improving the convenience for the user.

[0165] According to the implementation described above, the application causes the computer of a terminal device equipped with a camera and installed on a mobile body to perform the following processing: acquiring images taken at different times; based on the images taken at different times, inferring whether the mobile body intends to turn right or left; and based on the inference result, causing the display unit of the terminal device provided on the housing on which the camera is provided to display an image of a report related to the presence of a traffic participant, thereby enabling a more accurate and better estimation of the mobile body's intention to move.

[0166] The above description illustrates specific embodiments of the present invention, but the present invention is not limited to such embodiments in any way, and various modifications and substitutions can be made without departing from the spirit of the present invention.

Claims

1. An information providing method, wherein the information providing method causes a computer of a terminal device mounted on a mobile body and equipped with a camera to execute processing of: acquiring an image; determining whether or not a degree of caution for an operator of the mobile body is a first degree of caution or a second degree of caution smaller than the first degree of caution, based on a situation of a periphery of the mobile body obtained from the image; and causing a display section provided on a housing in which the camera is provided to display information corresponding to the degree of caution according to a result of the determination, displaying the information by the display section in a manner that the degree of caution is a prescribed degree or more, in a case where a traffic participant exists or is predicted to exist in a caution region that is a region of a prescribed angle range with reference to a reference direction, setting the caution region in front of the mobile body in a case where the mobile body is going straight, setting the caution region in front of the mobile body and in a left direction in a case where the mobile body is turning left, setting the caution region in front of the mobile body and in a right direction in a case where the mobile body is turning right.

2. An information providing method, wherein the information providing method causes a computer of a terminal device mounted on a mobile body and equipped with a camera to execute processing of: acquiring an image; determining whether or not a degree of caution for an operator of the mobile body is a prescribed degree or more, based on a situation of a periphery of the mobile body obtained from the image; and causing a display section provided on a housing in which the camera is provided to display information indicating that the degree of caution is a prescribed degree or more, and causing a light emitting section to perform a passing indication, in a case where the degree of caution is a prescribed degree or more, displaying the information by the display section in a manner that the degree of caution is a prescribed degree or more, in a case where a traffic participant exists or is predicted to exist in a caution region that is a region of a prescribed angle range with reference to a reference direction, setting the caution region in front of the mobile body in a case where the mobile body is going straight, setting the caution region in front of the mobile body and in a left direction in a case where the mobile body is turning left, setting the caution region in front of the mobile body and in a right direction in a case where the mobile body is turning right.

3. The information providing method according to claim 1 or 2, wherein the information providing method causes the computer to further execute processing of: determining a direction in which the operator should be cautious, based on the situation of the periphery; and causing the display section to display an image indicating the direction in which the operator should be cautious.

4. The information providing method according to claim 1 or 2, wherein the information providing method causes the computer to further execute processing of: causing the display section to display an image of a first color, in a case where the degree of caution is a first degree of caution; and causing the display section to display an image of a second color different from the first color, in a case where the degree of caution is a second degree of caution smaller than the first degree of caution.

5. The information providing method according to claim 1 or 2, wherein ​ ​ ​ The information providing method causes the computer to further execute processing of: determining whether the level of vigilance is a first level of vigilance, a second level of vigilance that is lower than the first level of vigilance, or a third level of vigilance that is lower than the second level of vigilance, based on a situation of the periphery; and causing the display section to display information corresponding to the level of vigilance, according to a result of the determination.

6. The information providing method according to claim 1 or 2, wherein the information providing method causes the computer to further execute processing of: determining that the level of vigilance is a first level of vigilance, in a case where a traffic participant of an object of the gaze exists within a range of a first distance from the terminal device or is predicted to exist within the range of the first distance within a prescribed time from a current time; and determining that the level of vigilance is a second level of vigilance that is lower than the first level of vigilance, in a case where the traffic participant exists within a range of a second distance farther than the first distance from the terminal device or is predicted to exist within the range of the second distance within the prescribed time from the current time.

7. The information providing method according to claim 1 or 2, wherein the information providing method causes the computer to further execute processing of: determining that the level of vigilance is a first level of vigilance, in a case where a traffic participant exists or is predicted to exist in a region a, a region b, and a region c, and causing the display section to display information on the first level of vigilance corresponding to the region in which the traffic participant exists or is predicted to exist; and determining that the level of vigilance is a second level of vigilance that is lower than the first level of vigilance, in a case where the traffic participant exists or is predicted to exist in a region d, a region e, and a region f, and causing the display section to display information on the second level of vigilance corresponding to the region in which the traffic participant exists or is predicted to exist, the region a is a range of a first angle with reference to a reference direction and is a range of a first distance from the terminal device, the region b is a range between the first angle and a second angle larger than the first angle with reference to the reference direction on a left direction of the reference direction and is a range of the first distance, the region c is a range between the first angle and the second angle with reference to the reference direction on a right direction of the reference direction and is a range of the first distance, the region d is a range between the first angle and the second angle with reference to the reference direction and is a range between the first distance and a second distance farther than the first distance from the terminal device, the region e is a range between the first angle and the second angle on a left direction of the reference direction and is a range between the first distance and the second distance, the region f is a range between the first angle and the second angle on a right direction of the reference direction and is a range between the first distance and the second distance.

8. An information providing method, wherein The information providing method causes a computer of a terminal device mounted on a mobile body and equipped with a camera to execute the following processes: acquiring an image; determining whether or not a degree of caution for an operator of the mobile body is a prescribed degree or more based on a situation of a periphery of the mobile body obtained from the image; and causing a display portion provided on a housing in which the camera is provided to display information corresponding to the degree of caution as a result of the determination, in a case where a traffic participant exists or is predicted to exist in a region a, a region b, and a region c, the degree of caution is determined to be a first degree of caution, and the display portion displays information about the first degree of caution corresponding to the region in which the traffic participant exists or is predicted to exist, in a case where a traffic participant exists or is predicted to exist in a region d, a region e, and a region f, the degree of caution is determined to be a second degree of caution smaller than the first degree of caution, and the display portion displays information about the second degree of caution corresponding to the region in which the traffic participant exists or is predicted to exist, the region a is a range of a first angle with reference to a reference direction and is a range of a first distance from the terminal device, the region b is a range between the first angle and a second angle larger than the first angle in a left direction of the reference direction and is a range of the first distance, the region c is a range between the first angle and the second angle in a right direction of the reference direction and is a range of the first distance, the region d is a range between the first angle and the second angle and is a range between the first distance and a second distance farther than the first distance from the terminal device, the region e is a range between the first angle and the second angle in the left direction of the reference direction and is a range between the first distance and the second distance, the region f is a range between the first angle and the second angle in the right direction of the reference direction and is a range between the first distance and the second distance.

9. An information providing method, wherein the information providing method causes a computer of a terminal device mounted on a mobile body and equipped with a camera to execute the following processes: acquiring an image; determining whether or not a degree of caution for an operator of the mobile body is a prescribed degree or more based on a situation of a periphery of the mobile body obtained from the image; and in a case where the degree of caution is the prescribed degree or more, causing a display portion provided on a housing in which the camera is provided to display information indicating that the degree of caution is the prescribed degree or more, and causing a light emitting portion to perform a passing indication. In a case where a traffic participant exists or is predicted to exist in the area a, the area b, and the area c, the degree of alertness is determined to be a first degree of alertness, and the display portion is caused to display information about the first degree of alertness corresponding to the area in which the traffic participant exists or is predicted to exist, In a case where a traffic participant exists or is predicted to exist in the area d, the area e, and the area f, the degree of alertness is determined to be a second degree of alertness that is smaller than the first degree of alertness, and the display portion is caused to display information about the second degree of alertness corresponding to the area in which the traffic participant exists or is predicted to exist, The area a is a range of a first angle with reference to a reference direction, and is a range of a first distance from the terminal device, The area b is a range between the first angle and a second angle larger than the first angle in a left direction of the reference direction, and is a range of the first distance, The area c is a range between the first angle and the second angle in a right direction of the reference direction, and is a range of the first distance, The area d is a range between the first angle and the second angle with reference to the reference direction, and is a range between the first distance and a second distance farther than the first distance from the terminal device, The area e is a range between the first angle and the second angle in a left direction of the reference direction, and is a range between the first distance and the second distance, The area f is a range between the first angle and the second angle in a right direction of the reference direction, and is a range between the first distance and the second distance.

10. The information providing method according to claim 8 or 9, wherein The information providing method causes the computer to further execute processing of: in a case where a traffic participant exists or is predicted to exist in the area a, the area b, or the area c, causing the display portion to display information indicating that a front direction, a left direction, or a right direction of the moving body is alerted at the first degree of alertness, respectively; and in a case where a traffic participant exists or is predicted to exist in the area d, the area e, or the area f, causing the display portion to display information indicating that the front direction, the left direction, or the right direction of the moving body is alerted at the second degree of alertness, respectively.

11. The information providing method according to any one of claims 1, 2, 8, and 9, wherein The information providing method causes the computer to further execute processing of: determining, based on a situation of the surroundings, whether the degree of alertness is a first degree of alertness, or a second degree of alertness that is smaller than the first degree of alertness, or a third degree of alertness that is smaller than the second degree of alertness; and in a case where there is a line of objects existing in the vicinity of the moving body and connected in the same direction as a direction of travel of the moving body, determining that the third degree of alertness.

12. The information providing method according to claim 11, wherein The information providing method causes the computer to further execute a process of: determining that the third caution level is the third caution level when the distance between the connected objects is equal to or less than a first threshold value and the lateral distance between the connected objects and the moving body is equal to or less than a second threshold value.

13. The information providing method according to any one of claims 1, 2, 8, and 9, wherein The information providing method causes the computer to further execute a process of: determining, based on the situation of the periphery, whether the caution level is a first caution level, a second caution level that is smaller than the first caution level, or a third caution level that is smaller than the second caution level; and determining that the third caution level is the third caution level when the moving body traveling on a first road enters a second road connected to the first road and there are objects in front of and behind a connection site between the first road and the second road in the extension direction of the first road.

14. The information providing method according to claim 13, wherein The information providing method causes the computer to further execute a process of: determining that the third caution level is the third caution level when the distance between the object that is about to enter the connection site and the object behind the object is equal to or less than a third threshold value.

15. The information providing method according to any one of claims 1, 2, 8, and 9, wherein The information providing method causes the computer to further execute a process of: causing a light emitting portion controlled based on the instruction of the computer to perform a passing indication according to the caution level.

16. The information providing method according to claim 15, wherein the light emitting portion is provided to a moving body that is a two-wheeled vehicle and is capable of communicating with the computer, The information providing method causes the computer to further execute a process of: communicating with the light emitting portion to cause the light emitting portion to perform the passing indication.

17. The information providing method according to claim 15, wherein The information providing method causes the computer to further execute a process of: shortening the cycle of the passing indication when the caution level is a first caution level compared to when the caution level is a second caution level that is smaller than the first caution level.

18. The information providing method according to claim 15, wherein The information providing method causes the computer to further execute a process of: determining, based on the situation of the periphery, whether the caution level is a first caution level, a second caution level that is smaller than the first caution level, or a third caution level that is smaller than the second caution level; the third warning level is determined when the mobile body traveling on a first road enters a second road connected to the first road, and there is an object in front of or behind a connection between the first road and the second road in the extension direction of the first road on the first road; the display displays the information and the overtaking instruction is performed when the warning level is the first warning level or the second warning level; and the display displays the information without the overtaking instruction when the warning level is the third warning level.

19. A storage medium storing an application program, wherein the application program causes a computer of a terminal device to execute the following processing: acquiring an image; determining whether a warning level for an operator of a mobile body is a first warning level or a second warning level smaller than the first warning level, based on a situation of a periphery of the mobile body obtained from the image; and causing a display to display information corresponding to the warning level according to the result of the determination, the display displays the information in a manner of a degree or more of the warning level when a traffic participant exists or is predicted to exist in a warning area that is an area of a prescribed angle with a reference direction as a reference, the warning area is set in front of the mobile body when the mobile body is straight ahead, the warning area is set in front of the mobile body and in a left direction when the mobile body is turning left, the warning area is set in front of the mobile body and in a right direction when the mobile body is turning right.

20. A storage medium storing an application program, wherein the application program causes a computer of a terminal device to execute the following processing: acquiring an image; determining whether a warning level for an operator of a mobile body is a first warning level or a second warning level smaller than the first warning level, based on a situation of a periphery of the mobile body obtained from the image; and causing a display to display information corresponding to the warning level according to the result of the determination, the warning level is determined to be the first warning level and the display displays information on the first warning level corresponding to an area in which a traffic participant exists or is predicted to exist when the traffic participant exists or is predicted to exist in an area a, an area b, and an area c, the warning level is determined to be the second warning level smaller than the first warning level and the display displays information on the second warning level corresponding to an area in which a traffic participant exists or is predicted to exist when the traffic participant exists or is predicted to exist in an area d, an area e, and an area f, the area a is an area of a first angle with a reference direction as a reference and is an area of a first distance from the terminal device, the area b is an area of a second angle with the reference direction as a reference and is an area of a second distance from the terminal device, the area c is an area of a third angle with the reference direction as a reference and is an area of a third distance from the terminal device, the area d is an area of a fourth angle with the reference direction as a reference and is an area of a fourth distance from the terminal device, the area e is an area of a fifth angle with the reference direction as a reference and is an area of a fifth distance from the terminal device, and the area f is an area of a sixth angle with the reference direction as a reference and is an area of a sixth distance from the terminal device. The region b is a range between the first angle and a second angle larger than the first angle with reference to the reference direction, on a left direction of the reference direction, and is a range of the first distance, The region c is a range between the first angle and the second angle with reference to the reference direction, on a right direction of the reference direction, and is a range of the first distance, The region d is a range between the first angle and the second angle with reference to the reference direction, and is a range between the first distance and a second distance farther than the first distance from the terminal device, The region e is a range between the first angle and the second angle, on a left direction of the reference direction, and is a range between the first distance and the second distance, The region f is a range between the first angle and the second angle, on a right direction of the reference direction, and is a range between the first distance and the second distance.

21. An information providing apparatus, wherein the information providing apparatus includes: a camera provided to a housing; a display portion provided to the housing; a determination portion that determines whether a degree of caution for an operator of a moving body is a first degree of caution or a second degree of caution smaller than the first degree of caution, based on a situation of a periphery of the moving body obtained from an image captured by the camera; and a display control portion that causes the display portion provided to the housing on which the camera is provided to display information corresponding to the degree of caution according to a result of the determination, in a case where a traffic participant exists or is predicted to exist in a caution region that is a region of a range of a prescribed angle with reference to a reference direction, the display portion displays the information in a manner of a degree or more of the degree of caution prescribed, in a case where the moving body is straight ahead, the caution region is set in front of the moving body, in a case where the moving body is turning left, the caution region is set in front of the moving body and in a left direction, in a case where the moving body is turning right, the caution region is set in front of the moving body and in a right direction.

22. An information providing apparatus, wherein the information providing apparatus includes: a camera provided to a housing; a display portion provided to the housing; a determination portion that determines whether a degree of caution for an operator of a moving body is a first degree of caution or a second degree of caution smaller than the first degree of caution, based on a situation of a periphery of the moving body obtained from an image captured by the camera; and a display control portion that causes the display portion provided to the housing on which the camera is provided to display information corresponding to the degree of caution according to a result of the determination, in a case where a traffic participant exists or is predicted to exist in a region a, a region b, and a region c, the degree of caution is determined to be the first degree of caution, and the display portion displays information on the first degree of caution corresponding to the region in which the traffic participant exists or is predicted to exist, ​ ​ In a case where a traffic participant exists or is predicted to exist in the region d, the region e, and the region f, the degree of alertness is determined to be a second degree of alertness that is smaller than the first degree of alertness, and the display portion displays information about the second degree of alertness corresponding to the region in which the traffic participant exists or is predicted to exist, The region a is a range of a first angle with reference to a reference direction, and is a range of a first distance from a terminal device of the moving body, The region b is a range between the first angle and a second angle larger than the first angle with reference to the reference direction in a left direction of the reference direction, and is a range of the first distance, The region c is a range between the first angle and the second angle with reference to the reference direction in a right direction of the reference direction, and is a range of the first distance, The region d is a range between the first angle and the second angle with reference to the reference direction, and is a range between the first distance and a second distance farther than the first distance from the terminal device, The region e is a range between the first angle and the second angle in a left direction of the reference direction, and is a range between the first distance and the second distance, The region f is a range between the first angle and the second angle in a right direction of the reference direction, and is a range between the first distance and the second distance.

Images