Center device, driver assistance system, and driver assistance method

Abstract

To share and make better use of information about birds and animals among multiple vehicles. [Solution] The center device according to the embodiment includes a controller. The controller collects wildlife detection data from the in-vehicle device, which is data at the time of detection of wildlife detected based on images from an in-vehicle camera. The controller also generates and updates shared information to be shared among multiple vehicles based on the wildlife detection data. The controller also causes the in-vehicle device to provide route guidance for the vehicle to avoid or track wildlife based on the shared information.

Description

Technical Field

[0001] The disclosed embodiments relate to a center device, a driving support system, and a driving support method.

Background Art

[0002] Conventionally, wild animals such as bears, deer, and monkeys may appear in human residential areas and passage areas. When wild animals appear, many of the vehicles traveling in the vicinity will stop running and wait to leave to avoid damage caused by wild animals, or will take actions such as avoiding the place where they appear.

[0003] On the other hand, when wild animals appear, vehicles of exterminators, etc. need to take actions to track the wild animals upon receiving information about their appearance. As a driving support technology related to the appearance of such wild animals, a technology has been proposed that discriminates wild animals from the camera images mounted on vehicles, identifies their species, and notifies the user (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the above-described conventional technology has not fully considered sharing information about detected wild animals among a plurality of vehicles and effectively utilizing it.

[0006] For example, the conventional technology described above has a mechanism that, if a bird or animal is detected, notifies a following vehicle within a predetermined distance that a bird or animal is present. However, vehicles that are not currently within that distance but will pass through the same area later will not be notified of the same bird or animal. Furthermore, using the conventional technology described above, it is not possible to know whether the bird or animal that appeared is still present or has been exterminated by a pest control company.

[0007] One embodiment, made in view of the above, aims to provide a center device, a driving support system, and a driving support method that can share and more effectively utilize information about birds and animals among multiple vehicles. [Means for solving the problem]

[0008] A center device according to one embodiment includes a controller. The controller collects wildlife detection data from the in-vehicle device, which is data at the time of detection of wildlife based on images from an in-vehicle camera. The controller also generates and updates shared information to be shared among multiple vehicles based on the wildlife detection data. The controller also causes the in-vehicle device to provide route guidance for the vehicle to avoid or track the wildlife based on the shared information. [Effects of the Invention]

[0009] According to one embodiment, the controller of the center device collects wildlife detection data, which is data obtained when wildlife is detected from the in-vehicle device, and manages it as shared information that can be shared among multiple vehicles. The controller also causes the in-vehicle device to provide route guidance so that the vehicle can avoid or track wildlife based on the shared information. This enables real-time avoidance or tracking of wildlife, making it possible to provide effective driving assistance for general users and pest control companies, for example. In other words, according to one embodiment, information regarding wildlife can be shared among multiple vehicles and utilized more effectively. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is an overview diagram (part 1) of the driving assistance method according to the embodiment. [Figure 2] Figure 2 is an overview diagram (part 2) of the driving assistance method according to the embodiment. [Figure 3] Figure 3 is an overview diagram (part 3) of the driving assistance method according to the embodiment. [Figure 4] Figure 4 is an overview diagram (part 4) of the driving assistance method according to the embodiment. [Figure 5] Figure 5 is an overview diagram (part 5) of the driving assistance method according to the embodiment. [Figure 6] Figure 6 shows an example of the configuration of a driver assistance system according to the embodiment. [Figure 7] Figure 7 shows an example of the configuration of a drive recorder according to the embodiment. [Figure 8] Figure 8 shows an example of the configuration of a navigation device according to an embodiment. [Figure 9] Figure 9 shows an example of the configuration of a center device according to an embodiment. [Figure 10] Figure 10 shows the basic processing sequence performed by the driver assistance system according to the embodiment. [Figure 11] Figure 11 shows an example of a screen for configuring the use of wildlife sighting information. [Figure 12] Figure 12 shows an example of an authentication screen when a user selects tracking. [Figure 13] Figure 13 is a diagram (part 1) showing the processing sequence (part 1) executed by the driver assistance system according to the embodiment when setting up avoidance. [Figure 14] Figure 14 is a diagram (part 2) showing the processing sequence when the avoidance use setting is configured, which is performed by the driver assistance system according to the embodiment. [Figure 15] Figure 15 is a diagram (part 1) showing the processing sequence (part 1) executed by the driver assistance system according to the embodiment when setting up tracking usage. [Figure 16]FIG. 16 is a diagram (part 1) showing an operation example at the time of setting tracking usage executed by the driving support system according to the embodiment. [Figure 17] FIG. 17 is a diagram (part 2) showing an operation example at the time of setting tracking usage executed by the driving support system according to the embodiment. [Figure 18] FIG. 18 is a diagram (part 2) showing a processing sequence at the time of setting tracking usage executed by the driving support system according to the embodiment. MODE FOR CARRYING OUT THE INVENTION

[0011] Hereinafter, embodiments of the center device, driving support system, and driving support method disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments shown below.

[0012] In addition, hereinafter, the driving support system according to the embodiment will be described by taking the driving support system 1 (see FIG. 1) as an example. Further, hereinafter, it is assumed that the center device according to the embodiment is the center device 100 (see FIG. 1) included in the driving support system 1. The driving support method according to the embodiment is assumed to be a driving support method executed by the controller 103 (see FIG. 9) of this center device 100. Further, hereinafter, the in-vehicle device according to the embodiment will be described by taking the in-vehicle device 5 (see FIG. 1) as an example.

[0013] In addition, hereinafter, when it is necessary to distinguish between a plurality of identical elements, this element may be numbered in the form of “-n” (n is a natural number) after the reference sign indicating this element. When there is no particular need to distinguish, this numbering will not be performed.

[0014] In addition, expressions such as “specific”, “predetermined”, and “constant” in the following description may be read as “predetermined in advance”. Further, hereinafter, the car navigation device will be abbreviated as the “navigation device”.

[0015] First, an overview of the driving assistance method according to the embodiment will be explained using Figures 1 to 5. Figures 1 to 5 are schematic diagrams (1) to (5) illustrating the overview of the driving assistance method according to the embodiment. As shown in Figure 1, the driving assistance system 1 includes in-vehicle devices 5-1, 5-2, ..., 5-m (where m is a natural number of 3 or more) and a center device 100.

[0016] The in-vehicle device 5 includes a drive recorder 10, a navigation device 20, and an in-vehicle sensor 30 (see Figure 6). The drive recorder 10 is a video recording device mounted in the vehicle. The drive recorder 10 has a camera 12a. The camera 12a is provided to capture at least images of the area around the vehicle.

[0017] The drive recorder 10 records a certain period of driving record data, including video footage captured by the camera 12a, into a ring buffer memory while the vehicle is running, in an overwriteable format. The certain period is, for example, 24 hours. The driving record data includes not only video footage but also date and time information, location information, etc.

[0018] Furthermore, the drive recorder 10 performs image recognition processing on the video footage in parallel with recording the driving record data, for example, using an AI (Artificial Intelligence) model for image recognition. The AI ​​model is, for example, a DNN (Deep Neural Network) model trained using a machine learning algorithm. This AI model is pre-trained to be able to detect the type, position, and color of each object in the video footage. In this embodiment, this AI model is pre-trained to be able to detect at least certain birds and animals and their species. These specific birds and animals are, for example, bears, deer, monkeys, etc.

[0019] Furthermore, the drive recorder 10 is designed to detect specific events that meet pre-set event conditions. For example, the drive recorder 10 can detect the presence of birds or animals as a specific event.

[0020] Furthermore, when the drive recorder 10 detects a specific event, it sets the driving record data for a certain period before and after the detection time to be protected from overwriting. Hereafter, this driving record data that has been set to be protected from overwriting will be referred to as "event data." In addition, among the event data, the event data corresponding to the event in which wildlife was detected will be referred to as "wildlife detection data."

[0021] The drive recorder 10 can also record event data to a recording medium other than the ring buffer memory. This recording to a different recording medium and the aforementioned overwrite prevention setting may be performed by instructions from the central device 100.

[0022] Furthermore, the drive recorder 10 transmits event data to the central device 100 when it detects a specific event. The event data includes date and time information, location information, video footage, etc. Of course, the drive recorder 10 also transmits wildlife detection data to the central device 100.

[0023] The central device 100 is configured to collect event data transmitted from each drive recorder 10 and to analyze the vehicle's status at the time of event detection based on the collected event data. Furthermore, the central device 100 is configured to perform information processing based on the analysis results.

[0024] In the driving assistance method according to this embodiment, the center device 100 generates wildlife sighting sharing information, which is information about the appearance of wildlife, based on wildlife detection data from event data, for sharing among at least multiple vehicles. The center device 100 also manages the wildlife sighting sharing information based on wildlife detection data transmitted from multiple vehicles each time. For example, the center device 100 updates the wildlife sighting sharing information as appropriate in response to the received wildlife detection data.

[0025] Furthermore, the center device 100 transmits the relevant shared information on the appearance of wild animals to the in-vehicle device 5 in response to an acquisition request from the in-vehicle device 5. In addition, if the shared information on the appearance of wild animals that the in-vehicle device 5 has already received is updated, the center device 100 transmits the updated information to the in-vehicle device 5.

[0026] Furthermore, in the driving assistance method according to this embodiment, the in-vehicle device 5 provides route guidance to the navigation device 20 to avoid or track birds and animals based on shared information about the presence of birds and animals received from the center device 100.

[0027] Let me explain in more detail. As shown in Figure 1, when each in-vehicle device 5 detects a bird or animal while the vehicle is starting up (step S1), it transmits bird / animal detection data corresponding to this event to the central device 100 (step S2). The bird / animal detection data includes the date and time, location, video, type of bird or animal, etc.

[0028] Then, the controller 103 of the central device 100 collects wildlife detection data from each in-vehicle device 5 and generates and manages wildlife sighting shared information (step S3). The controller 103 also transmits wildlife sighting shared information to each in-vehicle device 5 in response to acquisition requests from each in-vehicle device 5 (step S4).

[0029] Then, each on-board device 5 provides route guidance to avoid or track birds and animals based on the shared bird and animal sighting information received from the center device 100 (step S5). Here, an example of the operation in step S5 will be explained in detail.

[0030] As shown in Figure 2, first, the vehicle user sets an arbitrary route R1 from the current location to the destination via the HMI (Human Machine Interface) unit 23 of the navigation device 20.

[0031] The navigation device 20 then acquires shared information on the presence of wild animals around the destination and around the set route R1 from the center device 100. If the navigation device 20 finds information on the presence of wild animals that have not been exterminated, it displays a pop-up screen M1 on the HMI unit 23 to inform the user, as shown in Figure 3.

[0032] In response, when the user requests more detailed information, the navigation device 20 displays a detailed screen M2 of the relevant wildlife sighting information on the HMI unit 23, as shown in Figure 3. The detailed screen M2 displays the date and time, location, type, and current status of the wildlife sighting, as well as a video display screen M21 that can play back the video footage from the time of the sighting. This allows for the sharing of information regarding wildlife sightings, including at least the date and time, location, video footage, type, and status.

[0033] Additionally, the details screen M2 displays a "Avoid" button B1 and a "Track" button B2. If the user selects the "Avoid" button B1, the navigation device 20 guides the user along an avoidance route to evade the animal. If the user selects the "Track" button B2, the navigation device 20 guides the user along a tracking route to follow the animal.

[0034] The option to avoid or track can be set in the user settings beforehand. The details of these settings will be explained later using Figures 11 and 12.

[0035] Next, let's assume the user selects the "Avoid" button B1 in the example shown in Figure 3. Then, as shown in Figure 4, the navigation device 20 automatically calculates and sets the avoidance route R2 to the destination and displays it on the HMI unit 23. At the same time, the navigation device 20 displays the area where birds and animals appear A0 and the estimated current location BP0 on the HMI unit 23.

[0036] The appearance range A0 is calculated by the center device 100 when it receives multiple animal detection data for the same type of animal from multiple vehicles within a predetermined range within a predetermined time period. The center device 100 determines that the animals are the same species and calculates the range A0 based on the multiple animal detection data. This prevents the duplicate sharing of information about the same animal. The estimated current location BP0 is calculated by the center device 100 based on the latest animal detection data from multiple vehicles. The animal appearance sharing information is updated as needed based on animal detection data from each vehicle, and the appearance range A0 and estimated current location BP0 are also updated as needed based on the updated animal appearance sharing information.

[0037] Then, for example, when the animal in question is exterminated, the status of the animal sighting information is updated to "exterminated." As shown in Figure 5, the navigation device 20 then displays a pop-up screen M3 on the HMI unit 23 to inform the user of this. The navigation device 20 also automatically cancels the avoidance route R2 after the animal has been exterminated and restores the original set route R1, as shown in Figure 5. At this time, if the navigation device 20 can travel to the destination faster by continuing on the avoidance route R2 after the animal has been exterminated, it does not need to revert to the original set route R1.

[0038] While Figures 4 and 5 illustrate examples of actions taken when avoiding birds and animals, examples of actions taken when tracking birds and animals will be described later using Figures 15 to 18.

[0039] As described above, in the driving assistance method according to this embodiment, the controller 103 of the center device 100 collects wildlife detection data from the in-vehicle device 5, which is data at the time of detection of wildlife based on the image of the camera 12a. The controller 103 also generates and manages wildlife sighting sharing information to be shared among multiple vehicles based on the wildlife detection data. The controller 103 also causes the in-vehicle device 5 to provide route guidance for the vehicle to avoid or track wildlife based on the wildlife sighting sharing information.

[0040] Therefore, according to the driving assistance method of this embodiment, the controller 103 collects wildlife detection data from the in-vehicle device 5, which is data at the time of wildlife detection, and manages it as wildlife sighting shared information that can be shared among multiple vehicles. The controller 103 also causes the in-vehicle device 5 to provide route guidance so that the vehicle can avoid or track wildlife. This enables real-time avoidance or tracking of wildlife, making it possible to provide effective driving assistance for general users as well as pest control companies, for example. In other words, according to one aspect of the embodiment, information regarding wildlife can be shared among multiple vehicles and utilized more effectively.

[0041] The following describes in more detail an example of the configuration of the driver assistance system 1, which includes a center device 100 to which the driver assistance method according to the above embodiment is applied. In the following, the shared information on the appearance of wild animals may be simply referred to as "shared information."

[0042] Figure 6 shows an example of the configuration of the driver assistance system 1 according to the embodiment. As shown in Figure 6, the driver assistance system 1 includes in-vehicle devices 5-1, 5-2, ... 5-m and a center device 100.

[0043] Each in-vehicle device 5 and the central device 100 are connected to each other via a network N1, such as the Internet, a mobile phone network, or a C-V2X (Cellular Vehicle to Everything) communication network, enabling them to communicate with one another.

[0044] Each in-vehicle device 5 includes a drive recorder 10, a navigation system 20, and an in-vehicle sensor 30. The drive recorder 10, the navigation system 20, and the in-vehicle sensor 30 are connected to each other via an in-vehicle network such as CAN (Controller Area Network) to enable communication.

[0045] The drive recorder 10 has already been explained, so its explanation will be omitted here. The navigation device 20 provides route guidance that avoids or tracks birds and animals based on user settings and shared information received from the center device 100.

[0046] The on-board sensor 30 is a group of various sensors mounted on the vehicle. The on-board sensor 30 includes, for example, an accelerator sensor, a brake sensor, a vehicle speed sensor, and a G-sensor. The on-board sensor 30 provides various sensing data indicating the vehicle's status during events or route guidance to the drive recorder 10 and the navigation system 20.

[0047] The central device 100 is implemented, for example, as a public cloud. The central device 100 is managed by administrative agencies such as the Ministry of the Environment and local governments, or by businesses that have been commissioned by administrative agencies to provide services for managing shared information on wildlife sightings. The central device 100 collects wildlife detection data transmitted from each in-vehicle device 5.

[0048] Furthermore, the center device 100 generates and manages shared information based on the collected bird and animal detection data. The center device 100 also transmits the relevant shared information to the in-vehicle device 5 in response to acquisition requests from the in-vehicle device 5. In addition, if the shared information already received by the in-vehicle device 5 is updated, the center device 100 transmits the updated information to the in-vehicle device 5.

[0049] Next, an example of the configuration of the drive recorder 10 will be described. Figure 7 is a diagram showing an example of the configuration of the drive recorder 10 according to the embodiment. As shown in Figure 7, the drive recorder 10 has a communication unit 11, a sensor unit 12, an HMI unit 13, a storage unit 14, and a controller 15.

[0050] The communication unit 11 is implemented by a network adapter or the like. The communication unit 11 is wirelessly connected to the network N1 and transmits and receives information to and from the center device 100 via the network N1.

[0051] The sensor unit 12 is a group of various sensors mounted on the drive recorder 10. The sensor unit 12 includes, for example, a camera 12a, a G-sensor 12b, and a GPS (Global Positioning System) sensor 12c.

[0052] Camera 12a is installed to capture images of at least the area around the vehicle. Camera 12a is mounted near the windshield, near the dashboard, near the rear window, etc. Camera 12a may also be installed to capture images of the interior of the vehicle. In this case, camera 12a may be implemented as, for example, a 360-degree camera.

[0053] The G-sensor 12b measures the acceleration applied to the drive recorder 10. The GPS sensor 12c determines the GPS position of the vehicle.

[0054] The HMI unit 13 is a component that provides interface components for input and output to a user, such as a driver, who operates the drive recorder 10. The HMI unit 13 includes an input interface that accepts input operations from the user. The input interface is implemented, for example, by a touch panel. Alternatively, the input interface may be implemented by a microphone or the like. Furthermore, the input interface may be implemented by software components.

[0055] Furthermore, the HMI unit 13 includes an output interface for presenting visual and auditory information to the user. The output interface is implemented, for example, by a display or speaker. The HMI unit 13 may also provide the input interface and output interface to the user as an integrated unit, for example, by using a touch panel display.

[0056] The memory unit 14 is implemented by a memory device such as ROM (Read Only Memory), RAM (Random Access Memory), or flash memory. In the example shown in Figure 7, the memory unit 14 stores the image recognition model 14a and the operation record data 14b. The operation record data 14b includes bird and animal detection data 14ba.

[0057] The image recognition model 14a corresponds to the AI ​​model for image recognition described above. After being loaded as an AI model into the controller 15, the image recognition model 14a is configured to detect various objects in each frame when each frame of video captured by the camera 12a is input to the controller 15.

[0058] In this embodiment, the image recognition model 14a is configured to detect, for example, specific birds and animals and their species that appear in each frame when each frame of an external video image is input. Furthermore, the image recognition model 14a may be configured to detect vehicles, lanes, traffic lights, the color of the illuminated traffic lights, etc.

[0059] The operation record data 14b is the aforementioned operation record data recorded by the drive recorder 10. The wildlife detection data 14ba is event data when wildlife is detected as a specific event.

[0060] The controller 15 corresponds to a so-called processor. The controller 15 is implemented by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphical Processing Unit), etc. The controller 15 executes a program according to an embodiment not shown, stored in the memory unit 14, using RAM as the working area. The controller 15 can also be implemented by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0061] The controller 15 executes a portion of the information processing performed by the in-vehicle device 5 in the processing sequence shown in Figure 10. An explanation using Figure 10 will be provided later.

[0062] Next, an example of the configuration of the navigation device 20 will be described. Figure 8 is a diagram showing an example of the configuration of the navigation device 20 according to the embodiment. As shown in Figure 8, the navigation device 20 includes a communication unit 21, a sensor unit 22, an HMI unit 23, a storage unit 24, and a controller 25.

[0063] The communication unit 21 is implemented by a network adapter or the like, similar to the communication unit 11 described above. The communication unit 21 is wirelessly connected to the network N1 and transmits and receives information to and from the center device 100 via the network N1. The communication unit 21 may be used in conjunction with the communication unit 11 of the drive recorder 10, or they may be the same unit.

[0064] The sensor unit 22 is a group of various sensors mounted on the navigation device 20. The sensor unit 22 includes, for example, a GPS sensor 22a. The GPS sensor 22a determines the GPS position of the vehicle. The GPS sensor 22a may be used in conjunction with the GPS sensor 12c of the drive recorder 10, or they may be the same.

[0065] The HMI unit 23 is a component that provides interface components for input and output to a user such as a driver operating the navigation device 20. However, since it is the same as the HMI unit 13 of the drive recorder 10, its explanation is omitted here. Note that the HMI unit 23 may be used in conjunction with the HMI unit 13 of the drive recorder 10, or they may be common to each other.

[0066] The memory unit 24, like the memory unit 14 described above, is implemented using a memory device such as ROM, RAM, or flash memory. In the example shown in Figure 8, the memory unit 24 stores map information DB 24a, route setting information 24b, and shared information on the appearance of wild animals 24c. The memory unit 24 may be used in conjunction with the memory unit 14 of the drive recorder 10, or they may be common to both.

[0067] Map information DB24a is a database of map information used by the navigation device 20 when providing route guidance. Route setting information 24b includes information such as the user-set route R1, the aforementioned avoidance route R2, and the tracking route R3 (see Figure 17) described later. Wildlife sighting shared information 24c stores wildlife sighting shared information received from the center device 100.

[0068] The controller 25, like the controller 15 described above, corresponds to a so-called processor. The controller 25 is implemented by a CPU, MPU, GPU, etc. The controller 25 executes a program according to an embodiment not shown, stored in the memory unit 24, using RAM as its working area. The controller 25 can also be implemented by an integrated circuit such as an ASIC or FPGA.

[0069] The controller 25 executes some or all of the information processing performed by the in-vehicle device 5 in the processing sequences shown in Figures 10, 13-15, and 17. An explanation using Figures 10, 13-15, and 17 will be provided later.

[0070] Next, an example of the configuration of the center device 100 will be described. Figure 9 is a diagram showing an example of the configuration of the center device 100 according to the embodiment. As shown in Figure 9, the center device 100 has a communication unit 101, a storage unit 102, and a controller 103. The center device 100 is also connected to an HMI unit 150.

[0071] The HMI unit 150 is a component that provides interface components for input and output to operators, etc., who operate the center device 100. The HMI unit 150 includes an input interface that receives input operations from operators, etc. The input interface is implemented, for example, by a touch panel. Alternatively, the input interface may be implemented by a keyboard, mouse, pen tablet, microphone, etc. Furthermore, the input interface may be implemented by software components.

[0072] Furthermore, the HMI unit 150 includes an output interface for presenting visual and audio information to the operator. The output interface is implemented, for example, by a display or speaker. The HMI unit 150 may also provide the input interface and output interface to the operator as an integrated unit, for example, by using a touch panel display.

[0073] The communication unit 101 is implemented by a network adapter or the like, similar to the communication units 11 and 21 described above. The communication unit 101 is connected to the network N1 by wire or wireless connection and transmits and receives information to and from each in-vehicle device 5 via the network N1.

[0074] The memory unit 102 is implemented by a storage device such as ROM, RAM, flash memory, or HDD (Hard Disk Drive). In the example shown in Figure 9, the memory unit 102 stores the collected data DB 102a and the shared information DB 102b regarding the appearance of birds and animals.

[0075] The collected data DB102a is a database that stores event data, including wildlife detection data collected from each drive recorder 10. The wildlife sighting shared information DB102b is a database of wildlife sighting shared information that is generated based on wildlife detection data and updated as needed.

[0076] Although not shown in the diagram, the memory unit 102 may store an AI model corresponding to the image recognition model 14a described above. When generating and updating the shared information on the appearance of wild animals, the controller 103 may include the image recognition results recognized by the center device 100 in the shared information on the appearance of wild animals by performing image recognition processing using such an AI model.

[0077] Controller 103, like controllers 15 and 25 described above, corresponds to a so-called processor. Controller 103 is implemented by a CPU, MPU, GPU, etc. Controller 103 executes a program according to an embodiment not shown, stored in the memory unit 102, using RAM as a working area. Controller 103 can also be implemented by an integrated circuit such as an ASIC or FPGA.

[0078] The controller 103 executes information processing by the center device 100 in the processing sequences shown in Figures 10, 13 to 15, and 17.

[0079] Next, we will explain the information processing performed by these processing sequences. First, Figure 10 shows the basic processing sequence executed by the driving support system 1 according to the embodiment. In each processing sequence described below, we will use the case where the user has set the option of avoidance or tracking in advance through the usage settings as an example. Figure 11 shows an example of the usage setting screen for wildlife sighting information. Also, Figure 12 shows an example of the authentication screen when the user selects tracking.

[0080] As shown in Figure 10, in the in-vehicle device 5, the controller 25 of the navigation device 20 first sets the usage settings for bird and animal sighting information based on user operation (step S101). In this usage setting, the controller 25 displays the usage setting screen M4 shown in Figure 11 on the HMI unit 23.

[0081] As shown in Figure 11, the user settings screen presents a UI (User Interface) that allows the user to select from three modes, such as "Avoid," "Tracking," and "Optional." Figure 11 shows an example where radio buttons B3, B4, and B5, corresponding to "Avoid," "Tracking," and "Optional," are displayed.

[0082] As shown in Figure 11, if the user selects "Avoid," the controller 25 will automatically change to the avoidance route R2 according to the sighting information around the destination / set route R1.

[0083] Furthermore, if the user selects "track," the controller 25 sets a destination from the list of animal sighting information and automatically changes the destination according to updates during tracking. This enables continuous tracking that matches the movement of the animals.

[0084] Furthermore, if the user selects "arbitrary settings," the controller 25 will operate to allow the user to make arbitrary selections when setting routes and receiving new information (see detailed screen M2 in Figure 3).

[0085] Furthermore, when multiple vehicles share information about animal sightings, it's possible that ordinary users, even those who aren't pest control professionals, might want to go and see the animals out of curiosity. This tendency is particularly evident among content creators on video streaming services these days.

[0086] To deter such dangerous behavior, it is desirable that the controller 25 displays an authentication screen M5, as shown in Figure 12, on the HMI unit 23 when "tracking" is selected, as shown in Figure 11.

[0087] Figure 12 shows an example where the controller 25 displays an authentication screen M5 on the HMI unit 23, which can only be authenticated by users who are, for example, registered with the Ministry of the Environment as certified wildlife capture operators. By making the tracking mode available only to users who have been authenticated via such an authentication screen M5, it is possible to restrict the number of people who can approach wildlife, thereby contributing to the suppression of damage caused by wildlife.

[0088] Returning to the explanation of Figure 10, the controller 15 of the drive recorder 10 then records the vehicle's operation while it is running (step S102). The controller 15 also determines whether or not a specific event, such as a bird or animal, has been detected while recording the operation (step S103).

[0089] If birds or animals are detected (Step S103, Yes), the controller 15 transmits bird and animal detection data for a certain period of time before and after the detection to the central device 100 (Step S104). If birds or animals are not detected (Step S103, No), the controller 15 repeats the process from Step S102.

[0090] The controller 103 of the central device 100 collects the bird and animal detection data in step S104, generates shared information, and manages the status (step S105).

[0091] In step S105, the controller 103 generates shared information for each presumed identical bird or animal, while performing data cleansing and other processes to ensure that shared information about the same bird or animal does not overlap. This makes it possible to provide highly accurate route guidance when avoiding or tracking birds or animals. The controller 103 may also conceal the license plates of other vehicles, pedestrians, etc., that appear in the video footage included in the shared information by blurring or other methods to protect privacy.

[0092] Furthermore, if the controller 103 receives wildlife detection data from the in-vehicle device 5 for wildlife that corresponds to already generated shared information, it updates the shared information based on that wildlife detection data.

[0093] Then, in the in-vehicle device 5, the controller 25 of the navigation device 20 sends a request to the central device 100 to acquire shared information at any time (step S111). Any time could be, for example, when the vehicle starts up or when setting a route.

[0094] In response to such acquisition request, the controller 103 of the center device 100 transmits the shared information corresponding to the request to the in-vehicle device 5 as new information if the in-vehicle device 5 has not yet received it (step S112). Then, the controller 103 determines whether or not there is updated information for the shared information already received by the in-vehicle device 5 (step S113).

[0095] If there is no update information (step S113, No), the controller 103 repeats step S113. If there is update information (step S113, Yes), the controller 103 sends the update information to the in-vehicle device 5 (step S114) and repeats the process from step S113.

[0096] Furthermore, the driver assistance system 1 can execute the processes of steps S102 to S105 and steps S111 to S114 in parallel.

[0097] Next, we will explain the processing sequence when the user has already selected "Avoid" on the usage setting screen M4 shown in Figure 11. Figure 13 is a diagram (part 1) showing the processing sequence when avoidance usage is set, which is executed by the driver support system 1 according to the embodiment. Figure 14 is a diagram (part 2) showing the processing sequence when avoidance usage is set, which is executed by the driver support system 1 according to the embodiment.

[0098] If "Avoid" is selected on the usage settings screen M4, as shown in Figure 13, when the vehicle starts up (step S201), the controller 25 of the navigation device 20 in the in-vehicle device 5 reads the surrounding shared information stored in the shared bird and animal sighting information 24c (step S202).

[0099] Furthermore, the controller 25 sends a startup notification to the central device 100 (step S203), and in response receives the latest information on shared peripheral information transmitted from the central device 100 (step S204).

[0100] The controller 25 then sets the destination and route based on the user's operation (step S205) and sends a request to the center device 100 to acquire shared information about the area around the destination / around the set route R1 (step S206).

[0101] The controller 103 of the center device 100 extracts any shared information corresponding to this request from the shared information DB 102b for the appearance of wild animals and transmits it to the in-vehicle device 5 (step S207).

[0102] Then, in the in-vehicle device 5, the controller 25 of the navigation device 20 determines whether or not there are any unexterminated birds or animals based on the status included in this shared information (step S208). If there are any unexterminated birds or animals (step S208, Yes), the controller 25 calculates and sets an avoidance route R2 (step S209). For example, the controller 25 analyzes the shared information to calculate the probability of encountering the relevant bird or animal and calculates an avoidance route R2 in which the probability of encounter is below a threshold. This makes it possible to set a safe route with a low probability of encounter as the avoidance route R2.

[0103] If there are no birds or animals that have not been exterminated (Step S208, No), the controller 25 maintains the configured route R1. Then, the controller 25 starts route guidance according to the configured route R1 or the avoidance route R2 (Step S210).

[0104] Furthermore, the controller 25 sends a request to the center device 100 to acquire shared information about the area around the driving route at any time after route guidance has started (step S211). Then, if there is any new shared information (new information) that has not yet been received, the controller 25 receives it from the center device 100 (step S212).

[0105] Furthermore, as shown in Figure 14, in the in-vehicle device 5, the controller 25 of the navigation device 20 repeats the processing of steps S301 to S305, or steps S311 to S312, after route guidance has started. In step S301, the controller 25 determines whether or not it has received update information from the center device 100.

[0106] If update information is received (step S301, Yes), the controller 25 displays the received update information to the HMI unit 23 (step S302). The controller 25 also determines whether or not the birds and animals that were targeted for route avoidance have been exterminated (step S303).

[0107] If the animals have been exterminated (Step S303, Yes), the controller 25 cancels the avoidance route R2 (Step S304) and provides route guidance using the original set route R1 (Step S305). As mentioned above, even if the animals have been exterminated, the controller 25 does not need to switch back to the original set route R1 if continuing on the avoidance route R2 shortens the time required to reach the destination.

[0108] Furthermore, if the controller 25 does not receive update information (step S301, No), or if the birds or animals targeted for route avoidance have not been exterminated (step S303, No), it does nothing.

[0109] Furthermore, as shown in Figure 14, in step S311, the controller 25 determines whether or not new information has been received (step S311).

[0110] If new information is received (Step S311, Yes), the controller 25 calculates and sets a new avoidance route R2 (Step S312). If no new information is received (Step S311, No), the controller 25 does not calculate a new avoidance route R2.

[0111] Next, we will explain the processing sequence when the user has already selected "Tracking" on the usage settings screen M4 shown in Figure 11. Figure 15 is a diagram (part 1) showing the processing sequence when tracking usage is set up, which is performed by the driving support system 1 according to the embodiment.

[0112] Figure 16 is a diagram (part 1) showing an example of the operation performed by the driver assistance system according to the embodiment when setting up tracking use. Figure 17 is a diagram (part 2) showing an example of the operation performed by the driver assistance system according to the embodiment when setting up tracking use. Figure 18 is a diagram (part 2) showing the processing sequence performed by the driver assistance system according to the embodiment when setting up tracking use.

[0113] If "Tracking" is selected on the usage settings screen M4, as shown in Figure 15, when the vehicle starts up (step S401), the controller 25 of the navigation device 20 in the in-vehicle device 5 reads the surrounding shared information stored in the shared bird and animal sighting information 24c (step S402).

[0114] Furthermore, the controller 25 sends a startup notification to the central device 100 (step S403), and in response receives the latest information on shared peripheral information transmitted from the central device 100 (step S404).

[0115] The controller 25 then accepts the designation of an arbitrary location based on user operation, for example (step S405). It then sends a request to the center device 100 to acquire shared information around the designated location (step S406).

[0116] The controller 103 of the center device 100 extracts any shared information corresponding to this request from the shared information DB 102b for the appearance of wild animals and transmits it to the in-vehicle device 5 (step S407).

[0117] Then, in the in-vehicle device 5, the controller 25 of the navigation device 20 generates and displays this list of shared information to the HMI unit 23 (step S408). The controller 25 then accepts the user's input to specify a tracking target from the list (step S409).

[0118] Then, the controller 25 sets the destination and route based on the location of the target to be tracked specified by the user (step S410). Here, we will explain in more detail an example of the operation of steps S405 to S410.

[0119] In step S405, as shown in Figure 16, the controller 25 receives a designation of an arbitrary location P1 from the user via the HMI unit 23. Then, a request to acquire shared information around the designated location is sent to the center device 100, and if there is shared information corresponding to this request, the controller 25 receives it from the center device 100 (steps S406 to S407).

[0120] Then, as also shown in Figure 16, the controller 25 generates a list of shared information, which is the wildlife sighting information list L1, and displays it to the HMI unit 23 (step S408). The controller 25 accepts the designation of the target to be tracked from each entry E1, E2, E3... of this wildlife sighting information list L1 (step S409).

[0121] Figure 16 shows an example where the UI is presented in such a way that the user can select entries E1 and E2, which are among the entries E1, E2, E3, etc., whose status is not "exterminated".

[0122] For example, when the wildlife sighting information list L1 is initially displayed, the controller 25 displays buttons B11 and B21 labeled "Avoid" and buttons B12 and B22 labeled "Exterminate," corresponding to entries E1 and E2, respectively. Also, when the display is initially set up, the controller 25 displays buttons B12 and B22 labeled "Exterminate" in a deactivated state (see the dashed line on button B22 in the diagram).

[0123] When buttons B11 and B21 display "Avoid," they indicate that the birds and animals in entries E1 and E2 are targets to be avoided. As shown in Figure 16, if the user operates button B11, which corresponds to entry E1, the display changes from "Avoid" to "Track," and the controller 25 sets the bird or animal in entry E1 as a target to be tracked.

[0124] Simultaneously, button B12 in entry E1 is activated (see the solid line on button B12 in the diagram). This activated button B12 will be used as a UI to track the target animal and report that it has been exterminated when it is successfully eliminated.

[0125] In addition, the wildlife sighting information list L1 also displays other items, such as a "Change Display Order" button B6. By operating this "Change Display Order" button B6, the user can switch the sort order of the list each time they touch it, for example, by date and time proximity, distance proximity, etc.

[0126] Then, based on the location of the bird or animal set as the target of tracking, the controller 25 calculates and sets the shortest tracking route R3 to reach, for example, the bird or animal's habitat range A0 or estimated current location BP0 as the destination, as shown in Figure 17 (step S410). This allows the user to automatically set a route to a location where the target bird or animal is likely to be found, with just a simple operation of selecting from the bird or animal sighting information list L1.

[0127] Returning to the explanation of Figure 15, the controller 25 then starts route guidance according to the configured tracking route R3 (step S411).

[0128] Furthermore, as shown in Figure 18, in the in-vehicle device 5, the controller 25 of the navigation device 20 repeats the processing of steps S501 to S502 after route guidance by the tracking route R3 has started. In step S501, the controller 25 determines whether or not it has received update information regarding the bird or animal being tracked from the center device 100.

[0129] If update information is received (Step S501, Yes), the controller 25 automatically changes the destination to the updated location if the location of the tracked bird or animal has been updated (Step S502). If the controller 25 does not receive update information (Step S501, No), it does nothing.

[0130] Furthermore, the controller 25 constantly determines whether or not there has been an "eradicated" input (step S511). An "eradicated" input corresponds, for example, to the operation of button B12 shown in Figure 16. If there is no "eradicated" input (step S511, No), the controller 25 repeats step S511.

[0131] If there is an input indicating that the animals have been exterminated (step S511, Yes), the controller 25 sends the exterminated data to the central device 100 (step S512). The controller 103 of the central device 100 updates the corresponding shared information to "exterminated" based on this exterminated data (step S513). This allows for information sharing that the animals have been exterminated, and for example, it is possible to share in real time that it is no longer necessary to avoid the animals.

[0132] In the explanation above, we have given an example in which shared information is transmitted from the central device 100 to the in-vehicle device 5, and the in-vehicle device 5 calculates each route, such as the avoidance route R2 and the tracking route R3, based on the shared information. However, this may also be done by the central device 100.

[0133] In this case, the UI elements such as the screens M1-M3 related to the appearance of birds and animals, the appearance range A0, the estimated current location BP0, and the bird and animal appearance information list L1 may be generated by the central device 100 and transmitted to the in-vehicle device 5 for display.

[0134] In such cases, the in-vehicle device 5 will basically transmit bird and animal detection data, the set route R1 if a route has been set, and the current location in real time or periodically to the center device 100.

[0135] The center device 100 exclusively manages the shared information based on the wildlife detection data transmitted from the vehicle-mounted device 5, and basically only stores the minimum necessary shared information on the vehicle-mounted device 5, not sharing information that should be protected. The center device 100 also calculates the probability of encountering the target wildlife based on the changing shared information, the changing current position of the vehicle-mounted device 5, and the set route R1. Then, based on the calculated encounter probability, the center device 100 calculates the avoidance route R2 and tracking route R3 to be set on the vehicle-mounted device 5.

[0136] The central device 100 then transmits the calculated routes to the in-vehicle device 5, which automatically sets them. Alternatively, instead of setting them automatically, the central device 100 allows the user to select the route on the in-vehicle device 5.

[0137] This method, in which the central device 100 is responsible for calculating each route, has the advantage of protecting information while also reducing the processing load on the in-vehicle device 5.

[0138] As described above, the center device 100 according to the embodiment includes a controller 103. The controller 103 collects wildlife detection data from the in-vehicle device 5, which is data at the time of wildlife detection based on the image from the camera 12a (corresponding to an example of an "in-vehicle camera"). The controller 103 also generates and updates wildlife sighting shared information (corresponding to an example of "shared information") to be shared among multiple vehicles based on the wildlife detection data. The controller 103 also causes the in-vehicle device 5 to provide route guidance for the vehicle to avoid or track wildlife based on the wildlife sighting shared information.

[0139] Therefore, according to the center device 100 of this embodiment, the controller 103 collects wildlife detection data, which is data from the in-vehicle device 5 when wildlife is detected, and manages it as shared information that can be shared among multiple vehicles. The controller also causes the in-vehicle device 5 to provide route guidance so that the vehicle can avoid or track wildlife based on the shared information. This enables real-time avoidance of wildlife or tracking of wildlife, making it possible to provide effective driving assistance for general users as well as pest control companies, for example. In other words, according to one aspect of the embodiment, information regarding wildlife can be shared among multiple vehicles and utilized more effectively.

[0140] In the above-described embodiment, the detected birds and animals were identified as potentially harmful pests, and the example given was one in which they were tracked to avoid or exterminate them. However, the driving assistance method according to the embodiment can also be applied to other use cases.

[0141] For example, one use case is searching for a missing pet. In this case, the image recognition model 14a should be trained to detect at least the same species as the missing pet, and a tracking route R3 for tracking the pet should be automatically set based on shared information derived from the detected animal data. This use case is a variation of tracking animals.

[0142] Another use case involves searching for tourist-oriented wildlife such as monkeys and ptarmigan. In this case as well, the image recognition model 14a should be trained to detect the wildlife to be searched, and a tracking route R3 should be automatically set based on shared information derived from the detected wildlife data. This use case is also a variation of the approach used when tracking wildlife.

[0143] Another use case is to avoid situations where, for example, the carcasses of birds and animals are blocking at least part of a road and obstructing passage. In this case, the image recognition model 14a should be trained to be able to detect whether the birds and animals are alive or dead, or an algorithm should be added to determine whether the detected birds and animals are alive or dead. Then, based on the shared information derived from the detected bird and animal data, an avoidance route R2 should be automatically set to avoid the carcasses. Such a use case is a variation of the bird and animal avoidance scenario.

[0144] Furthermore, in the embodiment described above, the driver assistance system 1 includes a plurality of in-vehicle devices 5 and a central device 100, but it may also include a plurality of terminal devices such as smartphones used by the user. In this case, the central device 100 treats the terminal devices as also being targets for transmitting shared information, and enables the sharing of bird and animal sighting information among such terminal devices, the in-vehicle devices 5, and the central device 100.

[0145] Further effects and modifications can be readily derived by those skilled in the art. Therefore, broader aspects of the present invention are not limited to the specific details and representative embodiments expressed and described above. Accordingly, various modifications are possible without departing from the spirit or scope of the overall concept of the invention as defined by the appended claims and their equivalents. [Explanation of symbols]

[0146] 1. Driver assistance system 10. Dashcam 11 Communications Department 12 Sensor section 13 HMI section 14 Storage section 15 Controllers 20 Navigation System 21 Communications Department 22 Recovery Unit 23 HMI section 24 Memory section 25 Controllers 30 Automotive Sensors 100 Center device 101 Communications Department 102 Storage section 103 Controller 150 HMI section

Claims

1. Based on the images from the in-vehicle camera, animal detection data, which is data from when animals are detected, is collected from the in-vehicle device. Based on the aforementioned wildlife detection data, shared information to be shared among multiple vehicles is generated and updated. A controller that causes the in-vehicle device to provide route guidance for the vehicle to avoid or track the birds and animals based on the shared information. A center device equipped with [a specific feature].

2. The aforementioned controller, The system generates the shared information which includes at least the date and time, location, video footage, and type of bird or animal at the time of detection. The center device according to claim 1.

3. The controller manages the shared information for each of the same birds and animals. The center device according to claim 1 or 2.

4. The aforementioned controller, When multiple in-vehicle devices receive animal detection data of the same type of animal within a predetermined time and range, it is determined that the animals indicated by the animal detection data are the same. The center device according to claim 3.

5. The aforementioned controller, Based on the shared information, the vehicle is instructed to provide route guidance to avoid the animal or bird using an avoidance route that results in an encounter probability between the vehicle and the animal or bird being below a threshold. The center device according to claim 1.

6. The aforementioned controller, Based on the shared information, the vehicle is instructed to provide route guidance to the vehicle to track the animal by selecting the location of the animal from the list of animal sighting information generated by the user. The center device according to claim 1.

7. The aforementioned controller, If the location of the animal in the shared information is updated while the vehicle is tracking the animal, the destination is changed to the updated location of the animal. The center device according to claim 6.

8. The shared information includes status information indicating whether or not the birds and animals have been exterminated. The aforementioned controller, When the vehicle-mounted device receives data indicating that the birds and animals have been exterminated, the status information is updated to reflect that the animals have been exterminated. The center device according to claim 1.

9. It comprises an in-vehicle device and a center device, The aforementioned in-vehicle device is Based on the images from the in-vehicle camera, birds and animals are detected. The animal detection data, which is the data obtained when the animal is detected, is transmitted to the center device. The aforementioned center device is The in-vehicle device collects the bird and animal detection data, Based on the aforementioned wildlife detection data, shared information to be shared among multiple vehicles is generated and updated. Based on the shared information, the in-vehicle device is instructed to provide route guidance for the vehicle to avoid or track the birds and animals. Driver assistance system.

10. A driver assistance method performed by the controller, This involves collecting wildlife detection data from the vehicle's equipment, which is data obtained when wildlife is detected based on images from the vehicle's camera, and Based on the aforementioned wildlife detection data, the system generates and updates shared information to be shared among multiple vehicles, Based on the shared information, the in-vehicle device is instructed to provide route guidance for the vehicle to avoid or track the birds and animals. Driving assistance methods including

Images