Information processing device, information processing method, and program

Abstract

This information processing device comprises: a recognition unit that recognizes areas in which a mobile object can move on the basis of the road structure in the vicinity of the mobile object, and a focus position which is the current position or future position of a traffic participant that exists in the vicinity of the mobile object; and a control unit that uses a notification unit to notify the occupant of the mobile object of the existence of the traffic participant when the focus position is included in a preset notification area in the vicinity of the mobile object and is within the area in which movement is possible, and that limits the notification when the focus position is included in the notification area and is outside the area in which movement is possible.

Description

[Technical Field]

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program. [Background Art]

[0002] Conventionally, a vehicle warning device that issues an alarm according to the possibility of collision of an object has been disclosed (see, for example, Patent Document 1). [Prior Art Documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2021-64084 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] However, with the above-mentioned device, there have been cases where appropriate notification cannot be provided to the occupant.

[0005] The present invention has been made in consideration of such circumstances, and an object thereof is to provide an information processing apparatus, an information processing method, and a program that can perform appropriate notification to an occupant. For example, unnecessary notification can be suppressed. [Means for Solving the Problem]

[0006] The information processing apparatus, information processing method, and program according to the present invention employ the following configuration. (1) An information processing device according to one aspect of the present invention includes: a recognition unit that recognizes an area in which a mobile body can move and a point of interest which is the current or future position of traffic participants present in the vicinity of the mobile body, based on the road structure in the vicinity of the mobile body; and a control unit that, if the point of interest is included in a notification area set in advance in the vicinity of the mobile body and is within the mobile area, uses a notification unit to notify the occupants of the mobile body of the presence of the traffic participants, and restricts the notification if the point of interest is included in the notification area and is outside the mobile area.

[0007] (2) In the embodiment of (1) above, the restriction means not providing the notification or providing a notification with a reduced intensity.

[0008] (3) In the embodiment of (1) above, the memory unit stores information of points where there is a high risk of the traffic participant entering the movable area of ​​the moving body from outside the movable area, along with information on the degree of risk. The control unit, when the point of interest is included in a notification area set in advance around the moving body and is outside the movable area, provides notification with an intensity corresponding to the degree of risk of the high-risk location if the point of interest matches the high-risk location stored in the memory unit.

[0009] (4) In the embodiment of (3) above, the control unit provides a stronger notification when the position of interest is a high-risk position than when the position of interest is a low-risk position.

[0010] (5) In the embodiment of (1) above, the control unit is provided with a gaze acquisition unit that acquires the direction of the gaze of the driver of the moving body, and the control unit provides a stronger notification than when the gaze position is included in a notification area set in advance around the moving body, is within the movable area, and the gaze is not directed towards the traffic participant.

[0011] (6): In the embodiment of (1) above, the control unit is provided with a gaze acquisition unit that acquires the direction of the gaze of the driver of the moving body, and the control unit provides a stronger notification than when the gaze position is included in a notification area set in advance around the moving body, is outside the movable area, and the gaze is not directed towards the traffic participant.

[0012] (7) In the embodiment of (1) above, the memory unit stores information of points where there is a high risk of the traffic participant entering the movable area from outside the movable area of ​​the moving body, along with information on the degree of risk. The control unit, when the point of interest is included in a notification area set in advance around the moving body and is outside the movable area, and when the point of interest matches a high-risk location stored in the memory unit, provides notification with an intensity corresponding to the degree of risk of the high-risk location. When the point of interest does not match a high-risk location stored in the memory unit, provides notification with an intensity lower than the intensity corresponding to the degree of risk, or does not provide notification at all.

[0013] (8) In any embodiment of (7) above, the control unit is provided with a gaze acquisition unit that acquires the direction of the gaze of the driver of the moving body, and the control unit provides a first intensity notification if the point of focus is included in a notification area set in advance around the moving body, is outside the movable area, the point of focus coincides with the high-risk location, and the gaze is directed toward the traffic participant, and provides a second intensity notification which is stronger than the first intensity notification if the point of focus is included in a notification area set in advance around the moving body, is outside the movable area, the point of focus coincides with the high-risk location, and the gaze is not directed toward the traffic participant.

[0014] (9): An information processing method according to another aspect of the present invention involves a computer performing the following processes: recognizing an area in which the mobile body can move, and a point of interest which is the current or future position of traffic participants present in the vicinity of the mobile body, based on the road structure surrounding the mobile body; and, if the point of interest is included in a pre-set notification area around the mobile body and is within the mobile area, using a notification unit to notify the occupants of the mobile body of the presence of the traffic participants; and if the point of interest is included in the notification area and is outside the mobile area, restricting the notification.

[0015] (10): A program according to another aspect of the present invention causes a computer to perform the following processes: recognize an area in which the mobile body can move based on the road structure around the mobile body, and a point of interest which is the current or future position of traffic participants present around the mobile body; and, if the point of interest is included in a pre-set notification area around the mobile body and is within the mobile area, use a notification unit to notify the occupants of the mobile body of the presence of the traffic participants; and if the point of interest is included in the notification area and is outside the mobile area, restrict the notification. [Effects of the Invention]

[0016] According to the embodiments described in (1) to (10) above, appropriate information can be provided to the crew. [Brief explanation of the drawing]

[0017] [Figure 1] This is a diagram showing the configuration of a vehicle M equipped with the vehicle system 1 (camera 10 and driver assistance device 100) of the embodiment. [Figure 2] This figure shows an example of the functional configuration of vehicle system 1. [Figure 3] This figure shows an example of a scenario where a bicycle is present around vehicle M. [Figure 4] This figure shows an example of a notification displayed on the display unit. [Figure 5] This figure shows an example of a scenario in which pedestrians are present around vehicle M. [Figure 6] It is a flowchart showing an example of the flow of processing executed by the driving support device 100. [Figure 7] It is a diagram showing an example of the content of correspondence information 150. [Figure 8] It is a flowchart showing an example of the flow of processing executed by the driving support device 100A. [Figure 9] It is a flowchart showing an example of the flow of processing executed by the driving support device 100. [Figure 10] It is a flowchart showing an example of the flow of processing executed by the driving support device 100A.

Mode for Carrying Out the Invention

[0018] Hereinafter, embodiments of an information processing device, an information processing method, and a program according to the present invention will be described with reference to the drawings.

[0019] Hereinafter, the forward direction of the moving object may be referred to as the positive X direction, the rearward direction of the moving object as the negative X direction, the right direction in the direction orthogonal to the front-rear direction (the right direction when facing the positive X direction) as the positive Y direction, the left direction in the direction orthogonal to the front-rear direction (the left direction when facing the positive X direction) as the negative Y direction, the vertically upward direction which is orthogonal to the X direction and the Y direction as the positive Z direction, and the vertically downward direction which is orthogonal to the X direction and the Y direction as the negative Z direction.

[0020] <First Embodiment> [Overall Configuration] FIG. 1 is a configuration diagram of a vehicle M on which the vehicle system 1 (the camera 10 and the driving support device 100) according to the embodiment is mounted. The vehicle M is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and the driving source thereof is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using electric power generated by a generator connected to the internal combustion engine, or electric power discharged from a secondary battery or a fuel cell. The vehicle M is an example of a "moving object".

[0021] Vehicle M is equipped with one or more cameras 10. Each of the multiple cameras 10 captures images of the area around vehicle M (for example, 360 degrees around). By each of the multiple cameras 10 capturing images within its respective imaging area, the area around vehicle M is imaged. The cameras 10 may be arbitrarily positioned as long as they are in a location that can capture images of the area around vehicle M. The number of cameras 10 may also be arbitrarily set.

[0022] The driver assistance device 100 recognizes or estimates the occupant's visual field based on the image captured by the camera 10, and controls the notification based on the estimation result.

[0023] Figure 2 shows an example of the functional configuration of vehicle system 1. Vehicle M is equipped with, for example, one or more cameras 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, a driver monitor camera 60, a driver control device 70, a driver assistance device 100, a driving force output device 200, a brake device 210, and a steering device 220. These devices and equipment are connected to each other by multiplex communication lines such as CAN (Controller Area Network) communication lines, serial communication lines, wireless communication networks, etc. Note that the configuration shown in Figure 1 is merely an example, and some of the configuration may be omitted, or other configurations may be added.

[0024] Camera 10 is a digital camera that uses a solid-state image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). Camera 10 is mounted at any location on the vehicle M. Camera 10 periodically and repeatedly images the area around the vehicle M. Camera 10 may also be a stereo camera.

[0025] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by objects (reflected waves) to determine at least the position (distance and bearing) of the object. The radar device 12 can be mounted at any location on the vehicle M. The radar device 12 may also detect the position and velocity of the object using the FM-CW (Frequency Modulated Continuous Wave) method.

[0026] LIDAR14 irradiates light (or electromagnetic waves with a wavelength close to light) around vehicle M and measures the scattered light. Based on the time from emission to reception, LIDAR14 detects the distance to the target. The irradiated light is, for example, pulsed laser light. LIDAR14 can be attached to any location on vehicle M.

[0027] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the camera 10, radar device 12, and LIDAR 14 to recognize the position and speed of an object. The object recognition device 16 outputs the recognition results to the driver assistance device 100. The object recognition device 16 may output the detection results from the camera 10, radar device 12, and LIDAR 14 directly to the driver assistance device 100. The object recognition device 16 may be omitted from the vehicle system 1.

[0028] The HMI30 displays various information to the occupants of vehicle M and accepts input operations from the occupants. The HMI30 includes various display devices, speakers, buzzers, vibration generators (vibrators), touch panels, switches, keys, etc.

[0029] The vehicle sensor 40 includes a vehicle speed sensor for detecting the speed of the vehicle M, an acceleration sensor for detecting acceleration, a yaw rate sensor for detecting angular velocity around the vertical axis, and an orientation sensor for detecting the orientation of the vehicle M.

[0030] The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver, a guidance control unit, and a memory unit that stores map information. The GNSS receiver determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be determined or supplemented by an INS (Inertial Navigation System) that utilizes the output of the vehicle sensor 40. The guidance control unit determines, for example, a route from the position of the vehicle M determined by the GNSS receiver (or any input position) to a destination input by the occupant, by referring to map information, and causes the HMI 30 to output guidance information so that the vehicle M travels along the route. The map information is, for example, information in which the shape of the road is represented by links indicating roads and nodes connected by links. The map information may also include road curvature and POI (Point of Interest) information. The navigation device 50 may transmit the current position of the vehicle M and the destination to the navigation server via a communication device and obtain the route from the navigation server.

[0031] The driver monitor camera 60 is a digital camera that uses a solid-state image sensor such as a CCD or CMOS. The driver monitor camera 60 is mounted at any location in the vehicle M in a position and orientation that allows it to capture the head of the occupant (hereinafter referred to as the driver) seated in the driver's seat of the vehicle M from the front (in a direction that captures the face). The driver monitor camera 60 outputs an image of the interior of the vehicle M, including the driver, taken from its mounted position, to the driver assistance device 100.

[0032] The driver control elements 70 include, for example, an accelerator pedal, a brake pedal, a steering wheel, a shift lever, and other controls. The driver control elements 70 are equipped with sensors that detect the amount of operation or whether or not an operation is being performed, and the detection results are output to some or all of the driving force output device 200, the brake device 210, and the steering device 220.

[0033] The driving force output device 200 outputs driving force (torque) to the drive wheels for the vehicle to move. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, and an ECU (Electronic Control Unit) that controls them. The ECU controls the above configuration according to information input from the driver assistance device 100 or from the driver control device 70.

[0034] The brake system 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and an ECU. The ECU controls the electric motor according to information input from the driver assistance device 100 or from the driver control unit 70, so that brake torque corresponding to the braking operation is output to each wheel. The brake system 210 may also include a backup mechanism that transmits hydraulic pressure generated by the operation of the brake pedal included in the driver control unit 70 to the cylinder via a master cylinder. The brake system 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic brake system that controls an actuator according to information input from the driver assistance device 100 to transmit hydraulic pressure from the master cylinder to the cylinder.

[0035] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies force to a rack and pinion mechanism to change the direction of the steering wheels. The steering ECU drives the electric motor to change the direction of the steering wheels according to information input from the driver assistance device 100 or from the driver control device 70.

[0036] [Driving assistance system] The driver assistance device 100 includes, for example, a recognition unit 110, a driver recognition unit 120, a processing unit 130, and a driver assistance unit 140. These functional units are realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or GPU (Graphics Processing Unit), or by the cooperation of software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transient storage medium) such as the HDD or flash memory of the driver assistance device 100, or it may be stored in a removable storage medium such as a DVD or CD-ROM and installed in the HDD or flash memory of the driver assistance device 100 when the storage medium (non-transient storage medium) is mounted on a drive device. The combined functional configuration of the processing unit 130 and the driving support unit 140 is an example of a "control unit".

[0037] The recognition unit 110 recognizes objects present around the vehicle M based on the recognition results from the object recognition device 16. The recognition unit 110 recognizes the type, position, direction of movement, and speed of movement of the objects. Objects include some or all of the following obstacles: moving objects such as vehicles, bicycles, motorcycles, and pedestrians; road boundaries such as road markings, steps, guardrails, shoulders, and median strips; structures installed on the road such as road signs and billboards; fallen objects present on the road; and objects present around the vehicle M. The recognition unit 110 recognizes the current and future positions of traffic participants such as vehicles, bicycles, motorcycles, and pedestrians present around the vehicle M as described above. The recognition unit 110 recognizes the future positions of traffic participants, for example, based on the direction of movement and speed of movement of the traffic participants. The recognition unit 110 may also acquire information such as the presence, position, and type of objects by inputting the image captured by the camera 10 into a trained model that has been trained to output information such as the presence, position, and type of objects when an image captured by the camera 10 is input.

[0038] The recognition unit 110 analyzes the road structure around the vehicle M based on detection information from the object recognition device 16 (camera 10, radar device 12, LIDAR 14), and recognizes the area where the vehicle M can move based on the analyzed road structure. The area where the vehicle can move is, for example, the area where the vehicle M can drive or stop. The area where the vehicle can move is, for example, the roadway. Sidewalks are excluded from the area where the vehicle can move. The road structure includes the shape of the road, objects placed on the road, signs installed on the road, markings on the road, etc. The recognition unit 110 may identify the area where the vehicle can move based on information indicating the boundary between the roadway and the sidewalk, such as lanes and markings on the road, or it may identify the area where the vehicle can move using a trained model or a predetermined algorithm. A trained model is a model that, when an image is input, outputs information indicating the roadway, sidewalk, and the boundary between the roadway and the sidewalk. An algorithm is, for example, semantic segmentation. The recognition unit 110 may use semantic segmentation technology to classify each pixel in the image frame into a class (such as roadway, sidewalk, boundary, or obstacle) and assign a label to it, and recognize a movable area based on the assigned label. The driver assistance device 100 may, instead of (or in addition to) the detection information of the object recognition device 16, use high-precision map information that includes road structure information to recognize the road structure of the target location. For example, high-precision map information may be stored in advance in the memory unit of the driver assistance device 100, and the driver assistance device 100 may refer to the high-precision map information to recognize the location of vehicle M or the road structure around the location of vehicle M. For example, the driver assistance device 100 may identify a movable area based on information such as the location and type of lane corresponding to the location of vehicle M, and the location of the roadway or sidewalk, which are included in the map information.

[0039] The driver recognition unit 120 acquires direction information indicating the direction the user is looking, based on the image of the user's face captured by the driver monitor camera 60. The driver recognition unit 120 may recognize the direction information based on the orientation of the face, or it may recognize the direction information based on a predetermined part of the face or line of sight. The driver recognition unit 120 acquires the direction information using a known method. In the first embodiment, the driver recognition unit 120 may be omitted.

[0040] The processing unit 130 determines whether or not there are any traffic participants in the vicinity of vehicle M who should be notified to the occupants. Details of this process will be described later.

[0041] The driver assistance unit 140 performs driver assistance for the vehicle M based on the processing results of the processing unit 130. The driver assistance unit 140 also provides notifications to the occupants based on the processing results of the processing unit 130.

[0042] [Handling of traffic participants when they are within the notification area and within a movable area] The driver assistance unit 140 notifies the occupant (e.g., the driver) of the presence of a traffic participant if the traffic participant is within the notification area and within a movable area. Figure 3 shows an example of a scenario where a bicycle is present around vehicle M. The processing unit 130 sets the notification area AR1 around vehicle M. The notification area AR1 is an area set based, for example, on the speed of vehicle M and the direction in which vehicle M is traveling. In the example in Figure 3, the area behind vehicle M is excluded from the notification area AR1. In the example in Figure 3, since the notification area AR1 is set within the roadway, the entire notification area is a movable area. If bicycle B is present within the notification area AR1, the driver assistance unit 140 notifies the occupant of the presence of bicycle B (see Figure 4).

[0043] Figure 4 shows an example of a notification displayed on the display unit. The driver assistance unit 140 displays an image on the display unit (HMI 30) that includes content indicating attention to bicycles and content indicating the position of bicycles relative to vehicle M, as shown in Figure 4, and also outputs an alarm sound to the HMI 30 to notify the presence of traffic participants. The notification can be a combination of one or more notification methods, such as image notification, sound notification, or vibration notification (for example, a notification that vibrates the seat belt). For example, if a pedestrian is on the sidewalk, the driver assistance unit 140 may display on the display unit content that includes text information such as "Please pay attention to pedestrians" and a schematic diagram of the road structure (for example, a diagram showing the road shape, sidewalk shape, and the positional relationship between vehicle M, the road and the sidewalk) with the pedestrian to be paid attention to in the corresponding position.

[0044] As described above, the driver assistance unit 140 can notify the occupants of the presence of traffic participants by issuing a notification when traffic participants are present within the notification area and within the movable area.

[0045] In the example above, the driver assistance unit 140 was described as providing notification when the current location of a traffic participant is within the notification area and the movable area. In addition to (or instead of) this, the driver assistance unit 140 may also provide notification when the future location of a traffic participant is within the notification area and the movable area. In this case, the processing unit 130 obtains the future location of the traffic participant and determines whether the future location is within the notification area and the movable area. The future location is the location after a predetermined time. In the following description as well, the future location may be the subject of determination in addition to (or instead of) the current location.

[0046] [Handling of cases where traffic participants are located within the notification area but outside the area where movement is possible] The driver assistance unit 140 restricts notification if there are traffic participants within the notification area but outside the movable area. Restriction means, for example, not providing notification or weakening the intensity of the notification. Weakening the intensity of the notification means, for example, making the intensity of the notification weaker than when there are traffic participants within the notification area but outside the movable area. Weakening the intensity may also mean weakening the intensity of the notification sound below the standard. Furthermore, weakening the intensity may also mean not displaying the notification screen. For example, if the intensity is not weakened, the notification screen may be displayed on the display unit, and if the intensity is weakened, the notification screen may be displayed on the display unit.

[0047] Figure 5 shows an example of a scenario in which a pedestrian is present around vehicle M. The processing unit 130 sets the notification area AR1 around vehicle M. In the example in Figure 5, notification area AR1 includes notification area AR1-1, notification area AR1-2, and notification area AR1-3. Notification area AR1-1 is a movable area, while notification areas AR1-2 and AR1-3 are outside the movable area. Notification areas AR1-2 and AR1-3 are areas such as a sidewalk. The driver support unit 140 restricts notification if a pedestrian is present in notification area AR1-2.

[0048] As described above, the driver assistance unit 140 can provide appropriate notifications to the occupants by restricting notifications when traffic participants are present within the notification area but outside the movable area. For example, unnecessary notifications are suppressed, which is preferable for the occupants.

[0049] [Flowchart (Part 1)] Figure 6 is a flowchart showing an example of the processing flow performed by the driver assistance device 100. First, the recognition unit 110 recognizes the road structure around the vehicle M obtained from the image (step S100). Next, the recognition unit 110 identifies the area where movement is possible based on the recognized road structure (step S110).

[0050] Next, the processing unit 130 sets the notification area (step S120). Then, the processing unit 130 identifies the first area (AR1-1 in Figure 5) where the notification area and the movable area overlap (step S130).

[0051] Next, the processing unit 130 determines whether or not there are traffic participants in the first area (step S140). If there are no traffic participants in the first area, the process proceeds to step S160. If there are traffic participants in the first area, the driving support unit 140 notifies the occupants of the presence of traffic participants (step S150).

[0052] As described above, the driver support unit 140 can provide appropriate notifications to the occupants by notifying them to keep an eye on the traffic participants when traffic participants are present in the first area.

[0053] Next, the processing unit 130 uses the processing steps S100-S120 to identify the second area (AR1-2, AR1-3 in Figure 5) that is outside the movable area within the notification area (step S160).

[0054] Next, the processing unit 130 determines whether or not there are traffic participants in the second area (step S170). If there are no traffic participants in the second area, the processing unit 130 does not issue a notification (step S180). If there are traffic participants in the second area, the driver support unit 140 restricts the notification to inform the occupants of the presence of traffic participants (step S190). The notification at this time is, for example, weaker in intensity than the notification issued when there are traffic participants in the first area. This completes the processing of one routine in this flowchart.

[0055] In the above process, if there are traffic participants in the second area, the driver support unit 140 does not need to provide notification.

[0056] As described above, the driver support unit 140 can appropriately control the notification when there are traffic participants in the second area. For example, the second area is not a movable area, and the importance of the occupant's attention may be lower than in the first area. Therefore, the driver support unit 140 can appropriately control the notification by suppressing the notification when there are traffic participants in the second area, as described above.

[0057] According to the first embodiment described above, the driver assistance device 100 can provide appropriate notification to the occupants when traffic participants are present in either or both of the first area and the second area.

[0058] <Second Embodiment> The second embodiment will be described below. In the second embodiment, the driver assistance device 100A provides notification by referring to risk information at the location of a traffic participant when a traffic participant is present in the second area. For example, if the location of interest, which is the current or future location of the traffic participant, is included in the notification area and is outside the movable area, the driver assistance device 100A refers to the corresponding information and provides notification with an intensity based on the risk information associated with the corresponding location of the corresponding information if the location of interest matches the location of the corresponding information. The differences from the first embodiment will be described below.

[0059] Figure 7 shows an example of the configuration of the driver assistance device 100A including the correspondence information 150. In the second embodiment, the driver assistance device 100A includes a storage unit ST, in which the correspondence information 150 is stored. The storage unit ST is implemented by, for example, ROM (Read Only Memory), flash memory, SD card, RAM (Random Access Memory), HDD (Hard Disk Drive), registers, etc.

[0060] Correspondence information 150 is information that stores information about points where there is a high risk of a traffic participant entering the movable area of ​​vehicle M from outside the movable area, along with information on the level of risk. Correspondence information 150 is, for example, information that associates a point (location) with risk information. Risk information is, for example, information indicating the level of risk. Correspondence information 150 associates risk information with locations outside the movable area.

[0061] The corresponding information 150 may be associated with road structure features instead of a location. Road structure features include, for example, the width of the sidewalk, the combination of sidewalk width and roadway width, and information indicating the structure of the boundary between the roadway and sidewalk (e.g., the presence or absence of curbs or steps). In this case, the processing unit 130 acquires risk information corresponding to the road structure features recognized by the recognition unit 110.

[0062] A high-risk location is, for example, an area where there is a high risk (high probability) of traffic participants entering the roadway. A high-risk location may also be an area where, based on past statistical data, traffic participants have been identified as having a tendency to enter the roadway. A high-risk area may also be one where there are no steps, fences, guardrails, etc., at the boundary between the roadway and the sidewalk, and the boundary is structured in a way that makes it easy for traffic participants to enter the roadway.

[0063] The driver assistance device 100A provides notification by referring to the corresponding information 150. Figure 8 is a flowchart showing an example of the processing flow executed by the driver assistance device 100A. The processing in steps S100-S170 of this flowchart is the same as the processing in steps S100-S170 of Figure 6. The differences from the flowchart in Figure 6 will be explained in detail.

[0064] The processing unit 130 identifies a second area (AR1-2, AR1-3 in Figure 5) that is outside the movable area within the notification area (step S160). Next, the processing unit 130 determines whether or not there are traffic participants in the second area (step S170). If there are no traffic participants in the second area, the processing unit 130 does not issue a notification (step S180).

[0065] If there are traffic participants in the second area, the processing unit 130 refers to the correspondence information 150 (step S172) and determines whether the second area where the traffic participants are located is a risk area to which risk information is associated in the correspondence information 150 (step S174).

[0066] If the area is not a risk area to which risk information is associated, the process proceeds to step S180. In other words, no notification is given. If the area is not a risk area to which risk information is associated, the driver support unit 140 may give a notification to the occupants that there are traffic participants in the second area (a notification with an intensity that is suppressed compared to the intensity based on the risk information). This notification is, for example, weaker in intensity than the notification in step S176 described later.

[0067] If the risk information corresponds to a risk area, the driver support unit 140 provides notification (intensity notification based on risk information) according to the risk associated with the second area in the corresponding information 150 (step S176). This completes the processing of one routine in this flowchart.

[0068] According to the second embodiment described above, the driver assistance device 100A can provide notification in accordance with the risk information of the second area when there are traffic participants in the second area, thereby providing appropriate notification to the occupants.

[0069] <Third Embodiment> The third embodiment will be described below. In the third embodiment, the driver assistance device 100 provides notification taking into account the direction of the occupant's gaze. The differences from the first embodiment will be described below.

[0070] Figure 9 is a flowchart showing an example of the processing flow performed by the driver assistance device 100. The processes from steps S100 to S140 and steps S160 to S190 in Figure 9 are the same as the processes with the same step numbers in the flowchart of Figure 6, so their explanation is omitted.

[0071] If it is determined in step S140 that there are traffic participants in the first area, the processing unit 130 determines whether the occupant's (e.g., the driver's) gaze is directed towards the direction where the traffic participants are located (step S152). If the occupant's gaze is directed towards the direction where the traffic participants are located, the driving support unit 140 provides a first-intensity notification (step S154). If the occupant's gaze is not directed towards the direction where the traffic participants are located, the driving support unit 140 provides a second-intensity notification (step S156). The second-intensity notification is stronger than the first-intensity notification.

[0072] According to the third embodiment described above, the driver assistance device 100 can provide appropriate notification to the occupant based on whether or not the occupant's gaze is directed towards the traffic participants.

[0073] <Fourth Embodiment> The fourth embodiment will now be described. In the fourth embodiment, the driver assistance device 100A provides notification even when there are traffic participants in the second area, taking into account the direction of the occupant's gaze. The following will mainly describe the differences from the second embodiment.

[0074] Figure 10 is a flowchart showing an example of the processing flow performed by the driver assistance device 100A. The processing from step S100 to step S174 is the same as the processing of the same step number in the flowchart of Figure 8, so the explanation is omitted.

[0075] If the area is not a risk area to which risk information has been associated in step S174, the process proceeds to step S180. If the area is not a risk area to which risk information has been associated, the driver support unit 140 may notify the occupants that there are traffic participants in the second area (notification with an intensity that is suppressed compared to the intensity based on the risk information). If the area is a risk area to which risk information has been associated, the processing unit 130 determines whether the occupants' gaze is directed in the direction where the traffic participants are located (step S175A).

[0076] If the occupant's gaze is directed towards the direction of the traffic participants, the driver support unit 140 issues a first notification (step S175B). If the occupant's gaze is not directed towards the direction of the traffic participants, the driver support unit 140 issues a second notification (step S175C). The first notification is less intense than the second notification. This completes the processing of one routine in this flowchart.

[0077] According to the fourth embodiment described above, the driver assistance device 100A can provide appropriate notification to the occupant by providing notification in accordance with the direction of the occupant's line of sight when the second area where traffic participants are present is a risk area.

[0078] According to the embodiments described above, the driver assistance device 100 can provide appropriate notifications to the occupants based on the notification area, the movable area, and the positions of traffic participants.

[0079] The embodiments described above can be expressed as follows. A storage medium that stores computer-readable instructions, A processor connected to the storage medium, The processor executes the computer-readable instructions to: A process for recognizing the area in which the moving object can move, and the current or future location of traffic participants present around the moving object, based on the road structure surrounding the moving object. If the location of interest is included in a pre-set notification area around the moving body and is within the movable area, the notification unit is used to notify the occupant of the moving body of the presence of the traffic participant. If the location of interest is included in the notification area but outside the movable area, the process of restricting the notification is performed. An information processing device configured in such a way.

[0080] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of Symbols]

[0081] 10 Cameras 100 Driving support devices 110 Recognition part 130 Processing Unit 140 Driver Support Department

Claims

1. A recognition unit recognizes an area in which the moving object can move based on the road structure around the moving object, and recognizes a point of interest, which is the current or future position of traffic participants present around the moving object, based on the detection results of an object detection means that detects objects. A gaze acquisition unit that acquires the direction of the gaze of the driver of the moving object, A storage unit that stores risk information indicating the degree of risk of the traffic participant entering the movable area from outside the movable area, and correspondence information indicating the correspondence between risk information and location information, If the location of interest is included in a pre-set notification area around the moving body and is within the movable area, the notification unit is used to notify the occupant of the moving body of the presence of the traffic participant. A control unit restricts the notification if the point of interest is included in the notification area but outside the movable area. Equipped with, The control unit, When the point of interest is included in the notification area and outside the movable area, By referring to the aforementioned correspondence information, obtain the risk information corresponding to the location of interest, If the risk at the point of focus is greater than or equal to a predetermined degree, and the line of sight is directed towards the traffic participant, a first intensity notification is given. If the risk at the point of focus is greater than a predetermined degree and the line of sight is not directed towards the traffic participant, a second intensity notification, which is stronger than the first intensity notification, will be issued. If the risk at the location of interest is less than a predetermined degree, the notification will be made with reduced intensity, or the notification will not be made. Information processing device.

2. The aforementioned restriction means not providing the notification, or providing a notification with reduced intensity. The information processing apparatus according to claim 1.

3. The memory unit stores information about points where there is a high risk of the traffic participant entering the movable area of ​​the moving object from outside the movable area, along with information on the degree of risk. The control unit, If the point of interest is included in the notification area pre-set around the moving body and is outside the movable area, If the location of interest matches the high-risk location stored in the memory unit, notification is given with an intensity corresponding to the level of risk at the high-risk location. The information processing apparatus according to claim 1.

4. The control unit provides a stronger notification when the location of interest is a high-risk location than when the location of interest is a low-risk location. The information processing apparatus according to claim 3.

5. The control unit, If the point of focus is included in a pre-set notification area around the moving object, within the movable area, and the line of sight is not directed towards the traffic participant, a stronger notification is provided than when the point of focus is included in a pre-set notification area around the moving object, within the movable area, and the line of sight is directed towards the traffic participant. The information processing apparatus according to claim 1.

6. The memory unit stores information about points where there is a high risk of the traffic participant entering the movable area of ​​the moving object from outside the movable area, along with information on the degree of risk. The control unit, If the point of interest is included in the notification area pre-set around the moving body and is outside the movable area, If the location of interest matches the high-risk location stored in the memory unit, notification is given with an intensity corresponding to the level of risk at the high-risk location. If the location of interest does not match the high-risk location stored in the memory unit, notification will be given at an intensity lower than that corresponding to the level of risk, or no notification will be given at all. The information processing apparatus according to claim 1.

7. Computers A process that recognizes the area in which the moving object can move based on the road structure around the moving object, and recognizes a point of interest, which is the current or future position of traffic participants present around the moving object, based on the detection results of an object detection means that detects objects. The process of obtaining the direction of the driver's gaze of the moving object, If the location of interest is included in a pre-set notification area around the moving body and is within the movable area, the notification unit is used to notify the occupant of the moving body of the presence of the traffic participant. If the location of interest is included in the notification area but outside the movable area, the process of restricting the notification is performed. When the point of interest is included in the notification area and outside the movable area, The system obtains risk information corresponding to the location of interest by referring to correspondence information that shows the relationship between risk information, which indicates the degree of risk of the traffic participant entering the movable area from outside the movable area, stored in the memory unit, and location information. If the risk at the point of focus is greater than or equal to a predetermined degree, and the line of sight is directed towards the traffic participant, a first intensity notification is given. If the risk at the point of focus is greater than a predetermined degree and the line of sight is not directed towards the traffic participant, a second intensity notification, which is stronger than the first intensity notification, will be issued. If the risk at the location of interest is less than a predetermined degree, the notification will be made with reduced intensity, or the notification will not be made. Information processing methods.

8. On the computer, A process that recognizes the area in which the moving object can move based on the road structure around the moving object, and recognizes a point of interest, which is the current or future position of traffic participants present around the moving object, based on the detection results of an object detection means that detects objects. The process of obtaining the direction of the driver's gaze of the moving object, If the location of interest is included in a pre-set notification area around the moving body and is within the movable area, the notification unit is used to notify the occupant of the moving body of the presence of the traffic participant. If the location of interest is included in the notification area but outside the movable area, the process of restricting the notification is performed. When the point of interest is included in the notification area and outside the movable area, The system obtains risk information corresponding to the location of interest by referring to correspondence information that shows the relationship between risk information, which indicates the degree of risk of the traffic participant entering the movable area from outside the movable area, stored in the memory unit, and location information. If the risk at the point of focus is greater than or equal to a predetermined degree, and the line of sight is directed towards the traffic participant, a first intensity notification is issued. If the risk at the point of focus is greater than a predetermined degree, and the line of sight is not directed towards the traffic participant, a second-intensity notification, which is stronger than the first-intensity notification, will be issued. If the risk at the location of interest is below a predetermined degree, the notification will be made with reduced intensity, or the notification will not be made. program.

Images