System and information generation method

Abstract

To reduce cost for generating guidance information to a charging facility.SOLUTION: An information processing device according to an aspect of the present disclosure acquires, from a driving recorder of a target vehicle that has used a charging facility in a target on-road rest facility, target image data captured in a section from when the target vehicle starts entering the target on-road rest facility to when the target vehicle arrives at the charging facility, performs image analysis on the acquired target image data, generates a guidance sentence for the charging facility in the target on-road rest facility according to a result of the image analysis, and stores the generated guidance sentence in order to provide the guidance sentence to a user who uses the target on-road rest facility.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a system and method for generating information. [Background technology]

[0002] Patent Document 1 proposes an in-vehicle device that provides facility information, including the locations of chargers, when entering a parking area of ​​a facility via an access road branching off from a main road. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-153339 Summary of the Invention [Problem to be solved by the invention]

[0004] One object of the present disclosure is to provide a technique for reducing the cost of generating guidance information to charging facilities. [Means for solving the problem]

[0005] An information processing device according to a first aspect of the present disclosure includes a control unit configured to acquire, from a driving recorder of a target vehicle that has used a charging facility at a target roadside rest facility, target image data captured during a section from when the target vehicle starts entering the target roadside rest facility to when the target vehicle arrives at the charging facility, perform image analysis on the acquired target image data, generate guidance text for the charging facility at the target roadside rest facility based on the results of the image analysis, and store the generated guidance text to provide to a user who uses the target roadside rest facility.

[0006] An information generation method (information processing method) according to a second aspect of the present disclosure is executed by a computer. The information generation method includes acquiring, from a driving recorder of a target vehicle that uses a charging facility at a target roadside rest facility, target image data captured during a section from when the target vehicle starts entering the target roadside rest facility to when the target vehicle arrives at the charging facility, performing image analysis on the acquired target image data, generating guidance text for the charging facility at the target roadside rest facility according to the results of the image analysis, and saving the generated guidance text to provide to a user who uses the target roadside rest facility.

[0007] A system according to a third aspect of the present disclosure includes a server device and an on-board device installed in a target vehicle. The on-board device is configured to extract, from image data stored in the driving recorder of the target vehicle when the target vehicle charges at a charging facility at the target roadside rest facility, target image data captured during a section from when the target vehicle begins entering the target roadside rest facility to when the target vehicle arrives at the charging facility, and transmit the extracted target image data to the server device. The server device is configured to receive the target image data from the on-board device, perform image analysis on the received target image data, generate guidance text for the charging facility at the target roadside rest facility based on the results of the image analysis, and store the generated guidance text to provide to users of the target roadside rest facility. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to reduce the cost of generating guidance information to charging facilities. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 schematically illustrates an example of a situation to which the present disclosure is applied. [Figure 2] FIG. 2 shows a schematic diagram of an example of a road rest facility. [Figure 3]FIG. 3 shows an example of the time lapse from the vicinity of the entrance to the arrival at the charging facility and the extracted section. [Figure 4] FIG. 4 shows an example of a processing procedure for generating a guide sentence based on rules. [Figure 5A] FIG. 5A schematically illustrates an example of the hardware configuration of the server device. [Figure 5B] FIG. 5B schematically illustrates an example of the hardware configuration of the in-vehicle device. [Figure 6] FIG. 6 is a sequence diagram showing an example of a processing procedure for generating a guidance text by the system of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0010] Guidance information for roadside rest facilities such as service areas and parking areas is generally generated manually. However, with the spread of electric vehicles such as BEVs (Battery Electric Vehicles), charging facilities at roadside rest facilities are frequently updated (e.g., newly installed). Manually updating the guidance information in response to frequent updates to charging facilities would be very costly.

[0011] In contrast, an information processing device according to a first aspect of the present disclosure includes a control unit configured to acquire, from a driving recorder of a target vehicle that has used a charging facility at a target roadside rest facility, target image data captured during a section from when the target vehicle starts entering the target roadside rest facility to when the target vehicle arrives at the charging facility, perform image analysis on the acquired target image data, generate guidance text for the charging facility at the target roadside rest facility according to the results of the image analysis, and store the generated guidance text to provide to a user who uses the target roadside rest facility.

[0012] According to the first aspect of the present disclosure, by utilizing target image data obtained by a driving recorder of a target vehicle that has visited a target road rest facility, it is possible to automate at least a part of the task of generating guidance information for charging facilities, which is expected to reduce the cost of generating guidance information.

[0013] As another form of the information processing device (computer) according to the above embodiment, one aspect of the present disclosure may be an information processing method, an information processing system, a program, or a machine-readable storage medium, such as a computer, that implements all or part of the above components. Here, the machine-readable storage medium may be a non-transitory medium that stores information such as a program through electrical, magnetic, optical, mechanical, or chemical action. Non-transitory storage media may include storage media (e.g., CDs, DVDs, semiconductor memories), auxiliary storage devices for computers, external storage devices connected to computers, etc. The information processing system may be composed of multiple computers, such as server devices and in-vehicle devices.

[0014] For example, an information generation method (information processing method) according to a second aspect of the present disclosure is executed by a computer. The information generation method includes acquiring, from a driving recorder of a target vehicle that uses a charging facility at a target roadside rest facility, target image data captured during a section from when the target vehicle starts to enter the target roadside rest facility until the target vehicle arrives at the charging facility, performing image analysis on the acquired target image data, generating guidance text for the charging facility at the target roadside rest facility according to the results of the image analysis, and saving the generated guidance text to provide to a user who uses the target roadside rest facility.

[0015] Furthermore, for example, the system according to the third aspect of the present disclosure includes a server device and a target vehicle. The vehicle-mounted device is configured to extract, when a target vehicle charges at a charging facility at a target roadside rest facility, target image data captured during a section from when the target vehicle starts entering the target roadside rest facility to when the target vehicle arrives at the charging facility from image data stored in the driving recorder of the target vehicle, and to transmit the extracted target image data to a server device. The server device is configured to receive the target image data from the vehicle-mounted device, perform image analysis on the received target image data, generate guidance text for the charging facility at the target roadside rest facility based on the results of the image analysis, and store the generated guidance text to provide to users of the target roadside rest facility.

[0016] [1 Application example] 1 is a schematic diagram illustrating an example of a scenario in which the present disclosure is applied. A system 100 according to this embodiment includes a server device 1 and an in-vehicle device 2 installed in a target vehicle VE. The number of in-vehicle devices 2 (target vehicles VE) included in the system 100 may be determined appropriately depending on the embodiment.

[0017] The in-vehicle device 2 according to this embodiment is one or more computers configured to provide image data obtained by a driving recorder to a server device. In this embodiment, when a target vehicle VE charges at a charging facility 39 at a target road rest facility 3, the in-vehicle device 2 extracts target image data 50 captured during a section from when the target vehicle VE starts entering the target road rest facility 3 to when the target vehicle VE arrives at the charging facility 39 from image data 40 stored in the driving recorder DR of the target vehicle VE. The in-vehicle device 2 transmits the extracted target image data 50 to the server device 1. The in-vehicle device 2 may be an in-vehicle device permanently installed in the vehicle, or may be a user terminal at least temporarily installed in the vehicle. The type of in-vehicle device 2 may be selected appropriately depending on the embodiment.

[0018] On the other hand, the server device 1 according to this embodiment is one or more computers configured to generate guidance text for a facility from image data. The server device 1 is an example of an information processing device according to the present disclosure. In this embodiment, the server device 1 receives target image data 50 from the in-vehicle device 2. Receiving the target image data 50 is an example of acquiring the target image data 50. The server device 1 performs image analysis on the received target image data 50. The server device 1 generates guidance text 55 for the charging facility 39 at the target road rest facility 3 according to the results of the image analysis. The server device 1 stores the generated guidance text 55 to provide to users who use the target road rest facility 3.

[0019] According to the system 100 of this embodiment, by utilizing target image data 50 captured by the driving recorder DR of the target vehicle VE that has visited the target road rest facility 3, it is possible to automate at least a part of the work of generating guidance information (guidance text 55) to the charging facility 39. This is expected to reduce the cost of generating guidance information.

[0020] (Driving Recorder) The driving recorder DR is configured to observe the situation outside the target vehicle VE and record the observation results in the form of image data. The type of driving recorder DR is not particularly limited and may be appropriately selected depending on the embodiment, as long as it is capable of detecting objects present during the target vehicle VE's movement from the entrance 30 of the road rest facility 3 to the charging facility 39. The driving recorder DR may include any sensor that acquires data in the form of an image or an image representation. In a typical example, the driving recorder DR may be a video recorder.

[0021] In addition, as long as the situation outside the target vehicle VE can be observed, the location of the driving recorder DR is not particularly limited and may be determined appropriately depending on the embodiment. In a typical example, the driving recorder DR captures an image of the situation in front of the target vehicle VE through the windshield. The driving recorder DR may be installed near the center of the upper part of the windshield of the target vehicle VE so as to be in a position where the driving recorder DR is always installed in the target vehicle VE, or may be installed temporarily. The in-vehicle device 2 may be connected to the driving recorder DR temporarily when extracting the target image data 50, or may be connected to the driving recorder DR at all times.

[0022] (Roadside rest facility) The road rest facility 3 is a rest facility provided adjacent to a road, such as a service area or a parking area. The road is, for example, a highway.

[0023] Figure 2 shows a schematic diagram of an example of a roadway rest facility 3. The example in Figure 2 assumes a road where traffic is restricted to keeping to the left, with a first carriageway R1 and a second carriageway R2 separated by a center strip, and a roadway rest facility 3 provided on the left side of the first carriageway R1. Generally, the roadway rest facility 3 includes an approach road 31, a main area 33, and an exit road 37.

[0024] The approach road 31 is used when entering the main area 33 from the main road (first carriageway R1 in the case of Figure 2). An approach entrance 30 may be provided at the boundary between the approach road 31 and the main road. The approach entrance 30 constitutes the entrance to the road rest facility 3. Cushion drums 301 may be placed on structures (guardrails, walls, etc.) near the approach entrance 30. In addition, signs 302 may be placed around the approach entrance 30 to inform drivers of the presence of the road rest facility 3. The cushion drums 301 and the signs 302 are examples of markers for the approach entrance 30.

[0025] The main area 33 is an area used by users for resting. Various facilities may be provided in the main area 33. For example, the main area 33 may include a parking lot 34, a predetermined facility 35, and a charging facility 39. The parking lot 34 is a space for parking vehicles. In one example, the parking lot 34 area may be divided by vehicle type, such as motorcycles, compact cars, large vehicles, and trailers. The predetermined facility 35 may be a building used by visitors. The type of predetermined facility 35 may be selected arbitrarily. The predetermined facility 35 may include, for example, a commercial facility, a rest facility, a gas station, etc. The commercial facility may include, for example, a restaurant, a convenience store, etc. The rest facility may include, for example, a washroom, etc. The charging facility 39 is used to charge batteries installed in electric vehicles. The electric vehicle may include a plug-in hybrid vehicle. The type of charging facility 39 may be selected arbitrarily. The charging facility 39 may include, for example, a standard charger, a rapid charger, etc. The installation location of the charging facility 39 may be determined as appropriate depending on the embodiment. In one example, the charging facility 39 may be placed in the vicinity of a predetermined facility 35, such as in front of the facility 35, beside the facility 35, or beyond the facility 35.

[0026] Furthermore, the main area 33 may be provided with multiple lanes extending in the directions of each area of ​​the parking lot 34, specific facilities 35, charging facilities 39, and other locations. Accordingly, a branch point 32 may be provided near where the approach road 31 merges with the main area 33. The branch point 32 may be a point where one lane (the approach road 31) branches into multiple lanes. Predetermined display objects may be placed at and around the branch point 32. The display objects may be configured as appropriate to display any information, such as directions to each direction (branch destination). In one example, the predetermined display object may include at least one of road markings (321, 322) and a signboard 323.

[0027] Road markings (321, 322) are applied to the road surface. The road markings (321, 322) may include, for example, signs guiding in various directions, road markings, dividing lines, etc. The signboard 323 may be, for example, signs guiding in various directions, road signs, etc. The signs guiding in various directions may include signs indicating each area of ​​the parking lot ("small vehicles", "large vehicles", etc.). In the example of FIG. 2, the road markings (321, 322) are placed around the start of each lane after the branch, and the signboard 323 is placed on the outside near the branch point 32. However, the placement locations of the road markings (321, 322) and the signboard 323 need not be limited to this example. Road markings (321, 322) The road markings (321, 322) and the sign board 323 may be placed at the junction 32 and any location around the junction 32, respectively. The road markings (321, 322) and the sign board 323 are examples of indicators. The indicators are examples of markers for the junction 32.

[0028] The exit road 37 is used when exiting from the main area 33 to the main road. An exit exit may be provided at the boundary between the exit road 37 and the main road. The exit exit constitutes the exit of the road rest facility 3. Each vehicle, including the target vehicle VE, may enter the main area 33 from the main road via the approach road 31. Users may park their vehicles in the parking lot 34 of the main area 33 and take a rest as needed, for example, by using a predetermined facility 35. Thereafter, each vehicle may rejoin the main road from the main area 33 via the exit road 37. Note that the configuration of the road rest facility 3 is not limited to the example of Figure 2 and may be changed as desired. Regarding the configuration of the road rest facility 3, omissions, substitutions, and additions of components may be made as appropriate.

[0029] (Determining whether charging equipment was used) Whether or not the target vehicle VE has used the charging facility 39 at the target road rest facility 3 may be determined by any method. In one example, it may be determined that the target vehicle VE has used the charging facility 39 based on a user's declaration. The user's declaration may be made, for example, by operating the in-vehicle device 2. In another example, it may be determined whether or not the target vehicle VE has used the charging facility 39 based on any information processing.

[0030] As an example of the determination process, it may be determined whether the target vehicle VE has used the charging facility 39 by analyzing vehicle data. For example, the position of the target vehicle VE may be measured by a positioning module. The positioning module may be configured with a GPS (Global Positioning Satellite) module, a GNSS (Global Navigation Satellite System) module, etc. The positioning module may be provided in the target vehicle VE, the in-vehicle device 2, or another computer (such as a user terminal other than the in-vehicle device 2). Depending on the position information obtained by the positioning module, it may be determined whether the target vehicle VE has entered the target road rest facility 3. In addition, it may be determined whether the charge amount has increased based on battery information (such as State Of Charge (SOC)) of the target vehicle VE. Depending on whether or not the target vehicle VE has used the charging facility 39, it may be determined whether or not the target vehicle VE has used the charging facility 39.

[0031] As another example of the determination process, whether or not the target vehicle VE has used the charging facility 39 may be determined by analyzing image data obtained by the driving recorder DR. For example, whether or not the target vehicle VE has used the charging facility 39 may be determined depending on whether or not an object present at the target road rest facility 3 is detected in the image data. The object at the target road rest facility 3 may include, for example, a landmark at the entrance 30 (cushion drum 301, signboard 302, etc.), a landmark at the junction 32 (display, etc.), a predetermined facility 35, the charging facility 39, etc.

[0032] The entity that executes the determination process is not particularly limited and may be selected appropriately depending on the embodiment. Each of the above determination processes may be executed by any computer.

[0033] In one example, the in-vehicle device 2 may execute the above-described determination process to determine whether the target vehicle VE has used the charging facility 39. The in-vehicle device 2 may execute the above-described determination process autonomously or in response to a request from the server device 1. When it is determined that the target vehicle VE has used the charging facility 39, the in-vehicle device 2 may extract, from the stored image data 40, target image data 50 captured by the driving recorder DR of the target vehicle VE during a section from when the target vehicle VE starts to enter the target road rest facility 3 to when it arrives at the charging facility 39. The in-vehicle device 2 may then transmit the extracted target image data 50 to the server device 1. When it is determined that the target vehicle VE has not used the charging facility 39, the in-vehicle device 2 may omit the process of transmitting the target image data 50.

[0034] In another example, the server device 1 may acquire data from the target vehicle VE (on-board device 2). The server device 1 may use the acquired data to execute the above-mentioned determination process to determine whether the target vehicle VE has used the charging facility 39. The server device 1 may transmit a signal to the target vehicle VE (or on-board device 2) determined to have used the charging facility 39, requesting the provision of target image data 50. In response to the request signal, the on-board device 2 may extract the target image data 50 from the image data 40 and transmit the extracted target image data 50 to the server device 1. Note that the process of extracting the target image data 50 does not necessarily have to be performed by the on-board device 2. The on-board device 2 or the driving recorder DR may transmit at least a portion of the image data 40 including the target image data 50 to the server device 1. The server device 1 may extract the target image data 50 from at least a portion of the acquired image data 40.

[0035] In yet another example, a computer (such as another server device) other than the server device 1 and the in-vehicle device 2 may execute the above-described determination process to determine whether the target vehicle VE has used the charging facility 39. The other computer may issue a command to the target vehicle VE (or the in-vehicle device 2) determined to have used the charging facility 39 to transmit the target image data 50 to the server device 1. In response to a command from the external computer, the in-vehicle device 2 may extract the target image data 50 from the image data 40 and transmit the extracted target image data 50 to the server device 1. The external computer may issue a command to the server device 1 requesting the in-vehicle device 2 to transmit the target image data 50. Furthermore, if the in-vehicle device 2 or the driving recorder DR has previously provided the external computer with at least a portion of the image data 40 including the target image data 50, the external computer may extract the target image data 50 and provide the extracted target image data 50 to the server device 1. Alternatively, the external computer may provide at least a portion of the image data 40 stored in the external computer to the server device 1, causing the server device 1 to extract the target image data 50.

[0036] (Target image data) The target image data 50 is composed of image data captured by the driving recorder DR for at least a portion of the time from when the target vehicle VE starts to enter the target road rest facility 3 to when it arrives at the charging facility 39. In one example, the target image data 50 may be composed of image data for the entire time from when the target vehicle VE starts to enter the target road rest facility 3 to when it arrives at the charging facility 39. In another example, the target image data 50 may be composed of image data for a portion of the time (section).

[0037] The time at which the target vehicle VE starts entering the target road rest facility 3 may be defined as any time at which the target vehicle VE travels around the entrance 30. In one example, the time at which the target vehicle VE starts entering the target road rest facility 3 may be defined as the time at which the target vehicle VE passes the entrance 30, or as the time at which the target vehicle VE arrives in front of the entrance 30. The time in front of the entrance 30 may be defined relative to the entrance 30, such as X seconds before passing the entrance 30 or a position Dx [m] before the entrance 30. The reference value (X seconds, Dx [m]) may be set arbitrarily. The reference value may be a fixed value or a variable value (e.g., a value determined according to the traveling speed of the target vehicle VE). The time at which the target vehicle VE arrives at the charging facility 39 may be defined relative to the charging facility 39, such as the time at which the target vehicle VE stops in front of the charging facility 39.

[0038] The configuration of the target image data 50 is not particularly limited, and may be determined appropriately depending on the embodiment, as long as it includes image data for at least a part of the time from when the target vehicle VE starts to enter the target road rest facility 3 to when it arrives at the charging facility 39. In one example, the target image data 50 may include image data (which may be referred to as section image data) for each of one or more sections defined between when the target vehicle VE starts to enter the target road rest facility 3 and when it arrives at the charging facility 39. The target image data 50 may also include image data for sections other than the time from when the target vehicle VE starts to enter the target road rest facility 3 to when it arrives at the charging facility 39, such as before the target vehicle VE enters the target road rest facility 3. The target image data 50 may further include image data obtained by the driving recorder DR. The target image data 50 may be composed of a plurality of still images or may be composed of moving images. If the driving recorder DR is equipped with a microphone, the target image data 50 may further include sound data obtained together with the image data. The sound data may be omitted.

[0039] The target image data 50 may be extracted by any method from the image data 40 stored in the driving recorder DR. In one example, at least one of the target vehicle VE and the in-vehicle device 2 may be equipped with a timer, and vehicle data related to the travel of the target vehicle VE may be recorded together with the time. Accordingly, a reference time for extraction may be identified from the vehicle data. For example, the occurrence of an event related to the use of the charging facility 39 may be identified from the vehicle data. The event may include characteristic occurrences occurring between the start of entry into the target road rest facility 3 and the arrival at the charging facility 39, such as the target vehicle VE being positioned in front of the entrance 30, passing through the entrance 30, passing through the junction 32, or parking the target vehicle VE in front of the charging facility 39. Events related to the location of objects other than the charging facility 39, such as the target vehicle VE being positioned in front of the entrance 30, passing through the entrance 30, or passing through the junction 32, may be identified from the position information obtained by the positioning module. For example, the time at which the target vehicle VE is identified from the location information as being located at the entrance 30 may be identified as the time at which the target vehicle VE passed through the entrance 30. A charging-related event, such as parking the target vehicle VE in front of the charging facility 39, may be identified from the battery information. For example, the time immediately before the charge level of the target vehicle VE increases may be identified from the battery information, and the identified time may be identified as the time at which the target vehicle VE was parked in front of the charging facility 39. The method for identifying the occurrence time of each event is not limited to these examples and may be changed as appropriate depending on the embodiment. For example, the time at which the ignition was turned off in front of the charging facility 39 may be identified as the time at which the target vehicle VE was parked in front of the charging facility 39. Then, a section from which target image data 50 is extracted from the image data 40 may be identified based on the occurrence time of the identified event. In another example, the reference time for extraction may be identified from the image data 40 itself. For example, the occurrence of each of the above events may be identified by image analysis, and a section to be extracted as target image data 50 may be identified based on the occurrence time of the identified event.

[0040] The process of extracting the target image data 50 may be performed in association with the above-described determination process of determining whether the target vehicle VE has used the charging facility 39 at the target road rest facility 3, or may be performed independently of the above-described determination process. In one example, the process of extracting the target image data 50 may be performed at least partially in conjunction with the above-described determination process. For example, whether the target vehicle VE has used the charging facility 39 at the target road rest facility 3 may be determined based on whether the events of passing through the entrance 30 and charging at the charging facility 39 are detected. Furthermore, the target image data 50 may be extracted from the accumulated image data 40 based on the occurrence time of each detected event. In another example, the process of extracting the target image data 50 may be performed separately from the above-described determination process.

[0041] The method by which the in-vehicle device 2 transmits the target image data 50 to the server device 1 is not particularly limited and may be selected appropriately depending on the embodiment. In one example, the in-vehicle device 2 may directly transmit the target image data 50 to the server device 1. As a result, the server device 1 may directly acquire the target image data 50 from the in-vehicle device 2. In another example, the in-vehicle device 2 may transmit the target image data 50 to a storage area other than the server device 1, such as a storage medium or another computer. The other computer may include a data server such as a NAS (Network Attached Storage). In response to this, the server device 1 may indirectly acquire the target image data 50 via a storage medium, another computer, or the like. A similar method may be used for transmitting at least a portion of the image data 40 including the target image data 50 from the driving recorder DR or the in-vehicle device 2 to the server device 1 or another computer when the in-vehicle device 2 does not execute a process to extract the target image data 50.

[0042] (Sorting) The target vehicle VE may be any vehicle that has used the charging facility 39 at the target road rest facility 3. That is, the target image data 50 may be obtained from any vehicle that has used the charging facility 39 at the target road rest facility 3. In a typical example, the target image data 50 may be obtained from a vehicle that has used a charging facility (charging facility 39) that does not exist in the map information, or that exists in the map information but for which no guidance information exists.

[0043] In addition, in one example of this embodiment, it may be determined whether or not the imaging conditions in the vehicle satisfy predetermined conditions. The target image data 50 used to generate the guidance sentence 55 may be obtained from a vehicle whose imaging conditions are determined to satisfy the predetermined conditions. That is, the target vehicle VE from which the target image data 50 for generating the guidance sentence 55 is obtained may be selected depending on whether or not the imaging conditions satisfy the predetermined conditions. The predetermined conditions may be defined as appropriate to suit the image processing when generating the guidance sentence 55 (for example, whether or not the target object is easy to detect). In one example, the predetermined conditions may include at least one of the visibility conditions of the driving recorder DR, the driving conditions of the vehicle, and the vehicle type conditions.

[0044] The visibility conditions of the driving recorder DR may be defined as appropriate to extract vehicles with good visibility for the driving recorder DR. In one example, the visibility conditions of the driving recorder DR may be defined according to at least one of the weather when the target road rest facility 3 is visited and the imaging environment of the driving recorder DR. For example, the visibility conditions related to weather may be defined to extract vehicles that visit the target road rest facility 3 in weather other than weather that obstructs visibility. Weather that obstructs visibility may be, for example, rain, fog (including thick fog), snow, hail, etc. The weather conditions may be determined based on weather information in the area of ​​the target road rest facility 3. The weather information may be obtained as appropriate from an external server that provides weather information.

[0045] The visibility conditions related to the imaging environment may be defined to extract vehicles in which imaging by the driving recorder DR is performed in an environment other than an environment that obstructs visibility. The environment that obstructs visibility may include, for example, a dirty lens or lens cover of the driving recorder DR, or the presence of an obstacle that obstructs visibility in the imaging range of the driving recorder DR, etc. When the driving recorder DR is positioned to capture images of the outside of the vehicle through the windshield, determining whether an obstacle that obstructs visibility exists may include determining whether the windshield is dirty. The conditions related to the imaging environment may be determined from images obtained by the driving recorder DR.

[0046] Furthermore, if the vehicle is traveling at a high speed, blurring of the object occurs in the image acquired by the driving recorder DR, which may result in a decrease in the accuracy of the image analysis. Therefore, the vehicle traveling conditions may be defined so as to extract vehicles traveling at a speed below a threshold during the time period in which the target image data 50 is acquired. The speed threshold may be set arbitrarily. The vehicle traveling conditions may be determined according to the vehicle's traveling speed data. The traveling speed data may be acquired appropriately from the target vehicle VE.

[0047] Furthermore, many vehicles that use roadside rest facilities are compact vehicles. By using image data obtained from compact vehicles to generate the guidance text 55, it is possible to generate guidance text 55 that can provide guidance from a perspective similar to that of many vehicles that use roadside rest facilities. Therefore, the vehicle type condition may be defined to extract compact vehicles. The vehicle type condition may be determined according to vehicle attribute information. The vehicle attribute information may be acquired as appropriate. Note that compact vehicles may be appropriately defined to include general vehicles. For example, a compact vehicle may be defined as a vehicle that is 4.7 m or less in length, 1.7 m or less in width, and 2.0 m or less in height.

[0048] The predetermined conditions may include at least one of the above conditions. In one example, the predetermined conditions may include all of the visibility conditions of the driving recorder DR, the vehicle's driving conditions, and the vehicle type conditions. In this case, the target image data 50 used to generate the guidance text 55 can be obtained from a vehicle whose visibility conditions satisfy the good visibility condition, whose driving speed is below a threshold, and which is a small vehicle. By including at least one of the visibility conditions and the driving conditions in the predetermined conditions, it is possible to obtain target image data 50 suitable for image analysis. This can be expected to improve the accuracy of generating the guidance text 55. Furthermore, by including the predetermined conditions in the vehicle type conditions, it is possible to obtain target image data 50 with a visibility similar to that of many vehicles using the target road rest facility 3. This can be expected to generate guidance text 55 suitable for many users.

[0049] The process of selecting the target vehicle VE based on the above-mentioned imaging conditions may be performed at any timing. In one example, selecting the target vehicle VE from which to acquire the target image data 50 for generating the guidance text 55 may be configured by acquiring image data from all vehicles and then selecting image data that meets the conditions from the acquired image data as the target image data 50. In another example, selecting the target vehicle VE may be configured by selecting vehicles that meet the conditions from vehicles that have visited the target road rest facility 3 before acquiring the image data, and acquiring the target image data 50 from the selected vehicles.

[0050] The process of selecting the target vehicle VE based on the above-mentioned imaging conditions may be executed by any computer. Similar to the above-mentioned determination process of determining whether the target vehicle VE has used the charging facility 39 at the target road rest facility 3, the process of selecting the target vehicle VE may be executed by at least one of the in-vehicle device 2, the server device 1, and another computer.

[0051] Note that the process of selecting the target vehicle VE based on the imaging conditions may be omitted. In another example, the target vehicle VE from which the target image data 50 is acquired may be randomly selected from vehicles that have used the charging facility 39 at the target road rest facility 3. In yet another example, the target image data 50 may be acquired from all vehicles that have used the charging facility 39 at the target road rest facility 3.

[0052] (Image analysis) The content of the image analysis of the target image data 50 may be determined appropriately depending on the embodiment. In one example, the image analysis of the target image data 50 may include detecting an object appearing in the target image data 50, estimating the position of the object, estimating the distance to the object, estimating the positional relationship between multiple objects, identifying the type of object, etc. The object may be, for example, a landmark at the entrance 30 (cushion drum 301, signboard 302, etc.), a landmark at the junction 32 (display, etc.), a predetermined facility 35, a charging facility 39, etc.

[0053] Furthermore, the method of image analysis of the target image data 50 may be appropriately selected depending on the embodiment. In one example, the server device 1 may analyze the target image data 50 using a general image analysis method such as edge detection or pattern matching. In another example, the server device 1 may perform image analysis of the target image data 50 using a trained machine learning model that has acquired the ability to analyze images. The machine learning model is configured to have one or more calculation parameters that can be adjusted by machine learning. The one or more calculation parameters are used to calculate the desired inference (image analysis, etc.). The machine learning model may be configured using, for example, a neural network, a support vector machine, or other functional formulas (calculation models). The machine learning method may be appropriately selected depending on the machine learning model to be adopted (e.g., backpropagation, etc.). Training a machine learning model means adjusting (optimizing) the values ​​of the calculation parameters using training samples. The machine learning model may be appropriately trained so that, when given images of training samples, it derives the true value of the corresponding analysis result. The trained machine learning model may be a large-scale visual language model (VLM), etc. A large-scale model may be used. Note that, when the target image data 50 includes sound data, the image analysis of the target image data 50 may include sound analysis of the sound data.

[0054] (How to generate guidance text) The guidance sentence 55 may be generated as appropriate based on the results of image analysis of the target image data 50. In a typical example, the server device 1 may detect an object captured while the target vehicle VE is moving from the entrance 30 to the charging facility 39. The server device 1 may generate the guidance sentence 55 providing guidance about the charging facility 39 based on the detected object. That is, the guidance sentence 55 may be generated to provide guidance about the charging facility 39 based on an object observed by the driving recorder DR from the entrance 30 to the charging facility 39. The object (object) serving as the basis for guidance may be selected as appropriate depending on the embodiment. Furthermore, the results of the image analysis may be applied as appropriate to the generation of the guidance sentence 55. For example, when an object is identified in the image analysis, the server device 1 may generate the guidance sentence 55 including the name of the identified object, such as, "The charging facility is located just before the AAA facility." Furthermore, for example, when estimating the distance to an object in image analysis, server device 1 may generate guidance sentence 55 including the estimated distance, such as "The charging facility is located approximately XX m before the AAA facility." Furthermore, for example, when estimating the positional relationship between charging facility 39 and an object in image analysis, server device 1 may generate guidance sentence 55 including the estimated positional relationship, such as "The charging facility is located to the right of the AAA facility."

[0055] The method for generating the guidance text 55 is not particularly limited and may be selected appropriately depending on the embodiment. For example, the guidance text 55 may be generated rule-based. When a rule-based method is adopted, the server device 1 may generate the guidance text 55 from the results of image analysis according to the rules. The generation rules may be set appropriately. A template such as a fixed phrase may be used when generating the guidance text 55. Generating the guidance text 55 using a template may include at least one of selecting a fixed phrase to use from a plurality of prepared fixed phrases based on the results of image analysis and inputting the results of image analysis into a specified portion of the fixed phrase. The prepared fixed phrases may include, for example, a fixed phrase that can be used as is, such as "Charging facilities are...". The template may be stored in the memory resources of the server device 1, provided from an external computer, or incorporated into the program of the server device 1.

[0056] In another example, a trained machine learning model may be used for both image analysis and generation of the guidance text 55. The machine learning model may be configured as described above. When a trained machine learning model is used for both image analysis and generation of the guidance text 55, the machine learning model used for image analysis and the machine learning model used for generation of the guidance text 55 may be separate or the same.

[0057] When the former is adopted, the first machine learning model used for image analysis may be configured to accept input of image data (target image data 50) and output the result of image analysis of the input image data. The second machine learning model used for generating the guidance text 55 may be configured to accept input of the result of image analysis and output the result of generating the guidance text 55 according to the result of the input image analysis. Each machine learning model may be configured to further accept input of any information in addition to the above input. Furthermore, a dedicated model may be adopted for each machine learning model. Each dedicated model may be appropriately trained to acquire the ability to perform each of the above inference processes (image analysis, generation of the guidance text 55). Alternatively, each machine learning model may be a large-scale visual language model, a large-scale language model (LLM), etc. In one example, a large-scale visual and language model may be used as the first machine learning model, and a large-scale language model may be used as the second machine learning model. Large-scale models such as a large-scale visual and language model may be used for in-context learning. The server device 1 has acquired the ability to perform in-context learning. When using a machine learning model capable of performing such in-context learning, the server device 1 may provide a pre-prompt, such as an instruction on inference content, to the machine learning model along with the input information. As an example, when a large-scale visual language model is used as the first machine learning model, the server device 1 may provide inference instructions, such as "detect objects present around the charging facility" or "identify the type of object," to the first machine learning model along with the target image data 50. When a large-scale language model is used as the second machine learning model, the server device 1 may provide inference instructions, such as "generate a guide text to the charging facility from the provided information," to the second machine learning model along with the image analysis results. Each inference instruction may be provided as appropriate using a template or the like.

[0058] On the other hand, when the latter is adopted, the machine learning model may be configured to accept input of image data (target image data 50) and output the generated guidance text 55 corresponding to the input image data. That is, the machine learning model may be configured as an end-to-end model. In this case, image analysis of the target image data 50 and generation of the guidance text 55 based on the results of the image analysis may be simultaneously performed by the machine learning model. As described above, the machine learning model may be a dedicated model or a general-purpose model. The machine learning model may be configured by a model capable of performing in-context learning, such as a large-scale visual language model. When using a machine learning model capable of performing in-context learning, the server device 1 may provide a pre-prompt, such as an instruction for inference content, to the machine learning model along with the input information. For example, when a large-scale visual language model is adopted as the machine learning model, the server device 1 may provide an inference instruction, such as "(I) Detect objects present around the charging facility. (II) Generate a guidance text for the charging facility based on the detection result of (I)," to the machine learning model along with the target image data 50. The inference instruction may be provided appropriately using a template or the like.

[0059] In the road rest facility 3 illustrated in FIG. 2 , a vehicle typically enters the road rest facility 3 from the entrance 30 and heads toward the charging facility 39 via the junction 32. The generated guidance sentence 55 may include at least one of a first partial sentence providing guidance toward the charging facility 39 from the area from the entrance 30 to around the junction 32, and a second partial sentence providing guidance about the location of the charging facility 39 from the area after the junction 32. The location of the charging facility 39 may be indicated, for example, by its position relative to a predetermined facility 35. In both the case where the rule base is adopted and the case where a machine learning model is used, the server device 1 may generate at least one of the first partial sentence and the second partial sentence. The first partial sentence may be generated from the result of image analysis of image data up to around the junction 32. The second partial sentence may be generated from the result of image analysis of image data from passing the junction 32 to arriving at the charging facility 39. The first sub-sentence may be used to guide the user to the location of the charging facility 39, based on a position slightly distant from the charging facility 39. The second sub-sentence may be used to indicate the location of the charging facility 39, based on a position close to the charging facility 39.

[0060] When providing the generated guidance text 55 to a user who will use the target road rest facility 3 after the target vehicle VE, the timing of outputting the generated guidance text 55 may be defined arbitrarily. In one example, the timing of outputting the guidance text 55 may be uniformly defined, such as the timing of passing through the entrance 30. However, the size of the road rest facility 3 may differ depending on the location. In particular, the length of the entrance road 31 may differ. Even if the guidance text 55 is output at a location far away from the main area 33, there may be a large discrepancy between the information provided by the guidance text 55 and the current location, making it difficult to determine the location of the charging facility 39. Therefore, the server device 1 may set the output timing of the guidance text 55 according to the length of any section in the target image data 50. In other words, generating the guidance text 55 may include setting the output timing of the guidance text 55 when providing the guidance text 55 to the user. The section that serves as the basis for the output timing may be defined appropriately depending on the embodiment. For example, The section that serves as the timing reference may be defined as at least a part of the section from just before the entrance 30 to around the branch point 32.

[0061] As described above, the guidance sentence 55 (each partial sentence) may be generated according to a partial section (specific section) of the journey from when the target vehicle VE enters the target road rest facility 3 from the entrance 30 to when it arrives at the charging facility 39. Similarly, the output timing setting may also be specified according to a partial section. Therefore, one or more specific sections may be extracted in the time the target vehicle VE travels from just before the entrance 30 to the charging facility 39. Each section may be extracted by any method. In one example, each section may be extracted by appropriately dividing the travel time from the entrance 30, or by identifying a range that satisfies a condition.

[0062] 3 is a schematic diagram showing an example of the time lapse from near the entrance 30 to arriving at the charging facility 39 and an example of the extracted section. In the example of FIG. 3, a target vehicle VE is assumed to enter the road rest facility 3 illustrated in FIG. 2 from the entrance 30, and travel to the charging facility 39 via the entrance road 31 and the branch point 32.

[0063] In one example, during the process (time) of moving from just before the entrance 30 to the charging facility 39, a first section from just before the entrance 30 of the target road rest facility 3 to passing through the entrance 30 may be extracted. The first section may be extracted by any method. For example, as described above, the time when the target vehicle VE passes through the entrance 30 may be identified using position information, image analysis, etc. The time when the target vehicle VE is located just before the entrance 30 may be appropriately identified based on the identified time when the target vehicle VE passes through the entrance 30. In one example, as shown in FIG. 3 , the time just before the entrance 30 may be identified based on the time of passing through the entrance 30 (X seconds before passing through the entrance 30). In another example, the time just before the entrance 30 may be identified based on a criterion other than time (e.g., distance, etc.) using the time of passing through the entrance 30 as a reference. The time just before the entrance 30 may also be identified without using the time of passing through the entrance 30 as a reference. For example, the time point just before the entrance 30 may be identified by position information, image analysis (whether the entrance 30 is captured in the image, etc.), etc. This makes it possible to extract the first section.

[0064] When extracting the first section, the target image data 50 may include first image data 501 captured in the first section. The first image data 501 may be referred to as first section image data. Image analysis of the target image data 50 may include determining whether a predetermined facility 35 is captured in the first image data 501. If the predetermined facility 35 is captured in the first image data 501, the generated guidance sentence 55 may include a sentence (first-1 partial sentence 5511) providing guidance toward the charging facility 39 based on the predetermined facility 35. The first-1 partial sentence 5511 is an example of the first partial sentence. The first-1 partial sentence 5511 may be composed of one or more phrases that guide the user toward the charging facility 39 based on the predetermined facility 35, such as "Proceed toward the AAA facility." Charging facilities are often located in the vicinity of predetermined facilities. According to one example of this embodiment, in a case where a predetermined facility 35 is visible within an initial range of entering a target road rest facility 3, it is possible to generate guidance sentence 55 (first-first partial sentence 5511) that provides guidance to charging facility 39 based on the predetermined facility 35. This makes it possible to expect the generation of guidance sentence 55 that can appropriately guide the user to charging facility 39.

[0065] In one example, a third section including the junction 32 of the target road rest facility 3 may be extracted in the process of moving from just before the entrance 30 to the charging facility 39. The third section may be defined as appropriate to include the range around the junction 32 in which the display object may be captured. As shown in FIG. 3, in one example, the third section may be defined as the range from the point just before the junction 32 to the point at which the junction 32 is passed. The third section may also include a range beyond the junction 32. The third section may be extracted by any method. For example, as described above, the point at which the target vehicle VE passes the junction 32 may be identified by position information, image analysis, etc. The time point at which the target vehicle VE is located before the junction 32 may be appropriately determined based on the determined time point at which the target vehicle VE passes the junction 32. In one example, as shown in FIG. 3 , the time point before the junction 32 may be determined based on the time at which the target vehicle VE passes the junction 32 (Y seconds before passing the junction 32). In another example, the time point before the junction 32 may be determined based on a criterion other than time (e.g., distance, etc.) and on the time point at which the target vehicle VE passes the junction 32. The time point before the junction 32 may also be determined without using the time point at which the target vehicle VE passes the junction 32 as a reference. For example, the time point before the junction 32 may be determined by position information, image analysis (detection of a displayed object, etc.), etc. This makes it possible to extract the third section.

[0066] When extracting the third section, the target image data 50 may further include third image data 503 captured in the third section. The third image data 503 may be referred to as third section image data. Image analysis of the target image data 50 may include determining whether a predetermined object is captured in the third image data 503. The predetermined object may be, for example, the road markings (321, 322) or the sign board 323. If the first image data 501 does not capture the predetermined facility 35 but the third image data 503 captures the predetermined object, the generated guidance text 55 may include text (first-second partial text 5512) providing guidance in the direction of the charging facility 39 based on the predetermined object. The first-second partial text 5512 is an example of the first partial text. The first-second partial sentence 5512 may be composed of one or more phrases that guide the user toward the charging facility 39 by indicating a lane among multiple lanes extending from the junction 32 that leads to the charging facility 39, such as "Proceed toward the BBB area." When multiple display objects are present, a display appropriate for the traveling direction of the target vehicle VE toward the charging facility 39 may be appropriately selected from the displays indicated by each display object, such as a display located in the traveling direction of the target vehicle VE or a display guiding the traveling direction of the target vehicle VE. At the junction 32, the route splits into multiple lanes. Therefore, if the predetermined facility 35 is not visible within the initial range of approach, a user approaching the junction 32 may be confused about which lane to take to reach the charging facility 39. In contrast, according to an example of the present embodiment, when the predetermined facility 35 is not visible within the initial range of approach, a guidance sentence 55 (the first-second partial sentence 5512) that provides guidance toward the branch destination at the junction 32 can be generated. This makes it possible to expect the generation of a guidance sentence 55 that can appropriately guide the user to the charging facility 39.

[0067] If the predetermined facility 35 is not detected in the first image data 501 and the display object is not detected in the third image data 503, the server device 1 may detect any object (target object) from images obtained from the start point of the first section to the end point of the third section. The target object may be selected appropriately depending on the embodiment. For example, the server device 1 may detect the predetermined facility 35 as a target object from images of a range other than the first section. The server device 1 may detect objects other than the display object and the predetermined facility 35 as targets from images obtained from the start point of the first section to the end point of the third section. The image analysis of the target image data 50 may include determining whether a target object is captured in the image data from the start point of the first section to the end point of the third section. The generated guidance sentence 55 may include a sentence (first-third partial sentence 5513) providing guidance in the direction of the charging facility 39 based on the detected target object. Alternatively, the server device 1 may omit the process of generating the first partial sentence (first-third partial sentence 5513).

[0068] In one example, in the process of moving from just before the entrance 30 to the charging facility 39, a fourth section from just before the charging facility 39 to when the target vehicle VE is parked at the charging facility 39 may be extracted. The fourth section may be extracted by any method. For example, as described above, the time when the target vehicle VE is parked at the charging facility 39 may be identified using battery information, image analysis, etc. The time when the target vehicle VE is located in front of the charging facility 39 may be appropriately identified based on the time when the target vehicle VE is parked at the charging facility 39. In one example, as shown in FIG. 3, the time just before the charging facility 39 may be identified based on the time when the target vehicle VE is parked at the charging facility 39 (Z seconds before the target vehicle VE is parked at the charging facility 39). In another example, the time when the target vehicle VE is parked at the charging facility 39 may be identified based on a criterion other than time (for example, distance, etc.). A point in time just before the charging facility 39 may be identified based on the point in time. The point in time just before the charging facility 39 may be identified by image analysis (whether or not the charging facility 39 is captured in the image, etc.). This makes it possible to extract the fourth section.

[0069] When extracting the fourth section, the target image data 50 may include fourth image data 504 captured in the fourth section. The fourth image data 504 may be referred to as fourth section image data. Image analysis of the target image data 50 may include estimating the positional relationship between the specified facility 35 and the charging equipment 39 captured in the fourth image data 504. The generated guidance text 55 may include text (second partial text 552) providing guidance about the positional relationship between the specified facility 35 and the charging equipment 39. The positional relationship may be indicated by the direction in which the charging equipment 39 is located relative to the object, such as to the right of the facility or to the left of the facility. The positional relationship may also be indicated by a relative position relative to the object, such as in front of the facility, beside the facility, or beyond the facility. If at least one of the specified facility 35 and the charging equipment 39 is not detected from the fourth image data 504, the process of generating the second partial text 552 may be omitted.

[0070] The content of the second partial sentence 552 may be selected as appropriate depending on the positional relationship between the predetermined facility 35 and the charging facility 39. For example, when it is estimated that the charging facility 39 is located in front of the predetermined facility 35 (the first relationship is established), the server device 1 may generate, as an example of the second partial sentence 552, a second-first partial sentence 5521 informing the user that the charging facility 39 is located in front of the predetermined facility 35. Being located in front of the predetermined facility 35 may mean that the charging facility 39 is located away from the predetermined facility 35 on the entrance 30 (branch 32) side. Furthermore, when it is estimated that the charging facility 39 is located near the predetermined facility 35 (the second relationship is established), the server device 1 may generate, as an example of the second partial sentence 552, a second-second partial sentence 5522 informing the user that the charging facility 39 is located near the predetermined facility 35. Being located near the predetermined facility 35 may mean that the charging facility 39 is located in the vicinity of the predetermined facility 35. Furthermore, for example, when it is estimated that the charging facility 39 is located beyond the predetermined facility 35 (the third relationship is established), the server device 1 may generate, as an example of the second partial sentence 552, a second-third partial sentence 5523 informing the user that the charging facility 39 is located beyond the predetermined facility 35. The term "located beyond the predetermined facility 35" may mean that the charging facility 39 is located away from the predetermined facility 35 at a position past the predetermined facility 35 as viewed from the entrance 30 (branch 32). Whether the charging facility 39 is located near or far from the predetermined facility 35 may be determined by any method. For example, whether the charging facility 39 is located near or far from the predetermined facility 35 may be determined based on the location information of the target vehicle VE at the time of stopping and the location information of the predetermined facility 35. Furthermore, for example, whether the charging facility 39 is located near or far from the predetermined facility 35 may be determined based on whether the predetermined facility 35 and the charging facility 39 are simultaneously detected in the fourth image data 504. The types of second partial sentences 552 according to the positional relationship are not limited to the above three, and may be changed as appropriate depending on the embodiment. The types of second partial sentences 552 may be set to two or less, or may be set to four or more. According to one example of the present embodiment, it is possible to generate guidance sentences 55 (second partial sentences 552) that provide guidance on the positional relationship with a predetermined facility 35.This is expected to generate guidance text 55 that can appropriately guide the user to charging facility 39.

[0071] Also, in one example, in the process of moving from just before the entrance 30 to the charging facility 39, a second section may be extracted from the entrance 30 of the target road rest facility 3 after passing the entrance 30 and before passing the branch point 32. The second section may be appropriately defined between the entrance 30 and the branch point 32 (i.e., the range of the approach road 31). As shown in FIG. 3, in one example, the second section may be defined as the section between the end point of the first section and the start point of the third section. As a result, the second section may be continuous with the first section and the third section. In this case, the second section can be easily detected after the first section and the third section have been detected. Note that the definition of the second section is not limited to this example. The second section may be partially connected to at least one of the first section and the third section. The second section may overlap the third section. The second section may include the third section. The second section may not be contiguous with at least one of the first section and the third section. That is, the second section may be defined so as to be away from at least one of the end point of the first section and the start point of the third section.

[0072] When extracting the second section, the target image data 50 may include time information 520 indicating the time length of second image data 502 captured in the second section. The second image data 502 may be referred to as second section image data. The target image data 50 may or may not include the second image data 502. Generating the guidance text 55 may include setting the output timing of the guidance text 55 when providing the guidance text 55 to the user, in accordance with the time length indicated by the time information 520.

[0073] The relationship between the duration of the second section and the setting of the output timing may be defined as appropriate depending on the embodiment. For example, if the duration of the second section is less than the first threshold, the output timing may be set to the time when the vehicle passes through the entrance 30. If the duration of the second section exceeds the first threshold but is less than the second threshold, the output timing may be set to the time when the vehicle passes through the entrance 30 by D1 [m]. If the duration of the second section is equal to the first threshold, the output timing may be set to either the time when the vehicle passes through the entrance 30 or the time when the vehicle passes through the entrance 30 by D1 [m]. If the duration of the second section exceeds the second threshold, the output timing may be set to the time when the vehicle passes through the entrance 30 by D2 [m]. If the duration of the second section is equal to the second threshold, the output timing may be set to either the time when the vehicle passes through the entrance 30 by D1 [m] or the time when the vehicle passes through the entrance 30 by D2 [m]. The first threshold may be set as appropriate so that outputting the guidance text 55 at the entrance 30 is suitable for guiding the vehicle to the charging facility 39. The second threshold may be appropriately set to a value greater than the first threshold. D1 [m] may be appropriately set according to the second threshold. D2 [m] may be appropriately set to a value greater than D1 [m]. The types of output timing according to the time length of the second section are not limited to the above three and may be appropriately changed depending on the embodiment. The types of output timing may be set to two or less, or four or more. The criterion for determining the output timing is not limited to the entrance 30 but may be appropriately set for the target road rest facility 3 and its surroundings. Furthermore, the indicator of the output timing is not limited to distance. For example, the output timing may be calculated using other indicators such as time. The time length of the second section may correspond to the driving time on the entrance road 31. The longer the time length of the second section, the farther the entrance 30 is from the main area 33. The farther the entrance 30 is from the main area 33, the more difficult it may be to determine the location of the charging facility 39 even if the guidance message 55 is output near the entrance 30. In contrast, according to one example of this embodiment, by setting the output timing of the guidance text 55 using the time length of the second section as an index, it is possible to expect that the output of the guidance text 55 will be controlled at a timing that is appropriate for understanding the location of the charging equipment 39.

[0074] 3 is adopted, the target image data 50 may be configured to include first image data 501, time information 520 of the second image data 502, third image data 503, and fourth image data 504. When the first section is omitted, the first image data 501 may be omitted from the target image data 50. When the second section is omitted, the time information 520 may be omitted from the target image data 50. When the third section is omitted, the third image data 503 may be omitted from the target image data 50. When the fourth section is omitted, the fourth image data 504 may be omitted from the target image data 50.

[0075] Furthermore, the target image data 50 may or may not include image data for sections other than the above sections. For example, since the charging facility 39 is located away from the branch point 32, a section may be created between the end point of the third section and the start point of the fourth section. The target image data 50 may or may not include image data for this section. At least a part of the guidance text 55 may be generated from image data for other sections. In the example of FIG. 3, the third section and the fourth section are separated from each other. However, the third section and the fourth section may be separated from each other by this The example is not limited to the example shown above. The third and fourth sections may be continuous or may partially overlap each other by arranging the charging facility 39 close to the branch point 32. Each section may be extracted by any calculation process such as extraction of target image data 50 or image analysis.

[0076] (Example of a process for generating guidance text) 4 shows an example of a processing procedure for generating a guidance sentence 55 on a rule basis. In the example of FIG. 4, a situation is assumed in which the guidance sentence 55 is generated on a rule basis after adopting all of the configurations shown in FIG. 3 above. Before executing the process from step S10A, the server device 1 may acquire target image data 50 including first image data 501, time information 520 of the second image data 502, third image data 503, and fourth image data 504 from the in-vehicle device 2. The server device 1 may perform image analysis on the acquired target image data 50.

[0077] In step S10A, the server device 1 may determine whether a predetermined facility 35 has been detected from the results of image analysis of the first image data 501, as a determination of the first section. The detection of the predetermined facility 35 corresponds to the predetermined facility 35 being captured in the first image data 501. If the predetermined facility 35 has been detected, the server device 1 proceeds to the process of step S11. On the other hand, if the predetermined facility 35 has not been detected, the server device 1 proceeds to the process of step S10B. In step S10B, the server device 1 may determine whether a predetermined object has been detected from the results of image analysis of the third image data 503, as a determination of the third section. The detection of the predetermined object corresponds to the predetermined object being captured in the third image data 503. If the predetermined object has been detected, the server device 1 proceeds to the process of step S12. On the other hand, if the predetermined object has not been detected, the server device 1 proceeds to the process of step S13. In step S11, the server device 1 selects the 1-1 partial sentence 5511 as the first partial sentence 551. In step S12, the server device 1 selects the 1-2 partial sentence 5512 as the first partial sentence 551. In step S13, the server device 1 selects the 1-3 partial sentence 5513 as the first partial sentence 551. Once the first partial sentence 551 to be used in the guidance sentence 55 has been selected in any of steps S11 to S13, the server device 1 proceeds to the next step S20.

[0078] In step S20, the server device 1 determines the positional relationship between the predetermined facility 35 and the charging facility 39 from the result of image analysis of the fourth image data 504 to determine the fourth section. If the charging facility 39 is located in front of the predetermined facility 35 (the first relationship is established), the server device 1 proceeds to process step S21. If the charging facility 39 is located near the predetermined facility 35 (the second relationship is established), the server device 1 proceeds to process step S22. If the charging facility 39 is located beyond the predetermined facility 35 (the third relationship is established), the server device 1 proceeds to process step S23. In step S21, the server device 1 selects the 2-1 partial sentence 5521 as the second partial sentence 552. In step S22, the server device 1 selects the 2-2 partial sentence 5522 as the second partial sentence 552. In step S23, the server device 1 selects the 2-3 partial sentence 5523 as the second partial sentence 552. When the second sentence portion 552 to be used in the guidance sentence 55 is selected in any one of steps S21 to S23, the server device 1 advances the process to the next step S30.

[0079] In one example of this embodiment, the server device 1 can generate the guidance sentence 55 at the point in time when the selection of the first partial sentence 551 and the second partial sentence 552 to be used is completed. For example, the server device 1 can generate the guidance sentence 55 by concatenating the opening fixed sentence 550, the selected first partial sentence 551, the selected second partial sentence 552, and the closing fixed sentence 559. At least a part of the first partial sentence 551 and at least a part of the second partial sentence 552 may be given by a template. Furthermore, the opening fixed sentence 550 and the closing fixed sentence 559 may also be given by a template. The opening fixed sentence 550 may be composed of one or more words placed at the beginning of a sentence, such as "The charging facility is." The closing fixed sentence 559 may be The information may be composed of one or more phrases placed at the end of a sentence, such as "It is located at." The server device 1 may acquire the initial template sentence 550 and the final template sentence 559 as appropriate. As a specific example, when the 1-1 partial sentence 5511 is selected as the first partial sentence 551 and the 2-2 partial sentence 5522 is selected as the second partial sentence 552, the server device 1 can generate the guidance sentence 55, "The charging facility is located near the AAA facility as you proceed toward the AAA facility." However, the content of each sentence (550, 551, 552, 559) is not limited to this example and may be changed as appropriate depending on the embodiment.

[0080] In step S30, the server device 1 determines the length of the second section (the duration of the second image data 502) by referring to the time information 520. If the length of the second section is less than the first threshold, the server device 1 proceeds to step S31. If the length of the second section exceeds the first threshold and is less than the second threshold, the server device 1 proceeds to step S32. If the length of the second section exceeds the second threshold, the server device 1 proceeds to step S33. In step S31, the server device 1 sets the output timing of the guidance sentence 55 to the time when the entrance 30 is passed. In step S32, the server device 1 sets the output timing of the guidance sentence 55 to the time when the entrance 30 is passed by D1 [m]. In step S33, the server device 1 sets the output timing of the guidance sentence 55 to the time when the entrance 30 is passed by D2 [m]. The setting information for the output timing may be stored together with the guidance sentence 55 as appropriate. Once the output timing is set in any one of steps S31 to S33, the server device 1 may end the process related to the generation of the guidance text 55.

[0081] The above processing procedure for generating the guidance sentence 55 may be modified as appropriate depending on the embodiment. For example, the order of the processing of steps S10A to S13, the processing of steps S20 to S23, and the processing of steps S30 to S33 is not limited to the example of FIG. 4 and may be arbitrarily changed. Each processing may be performed at least partially in parallel. If the first partial sentence 551 is not used, the processing of steps S10A to S13 may be omitted. If the first-third partial sentence 5513 is not used, the processing of step S13 may be omitted. If the second partial sentence 552 is not used, the processing of steps S20 to S23 may be omitted. If the output timing is not set, the processing of steps S30 to S33 may be omitted. The number of branches in at least one of steps S20 and S30 may be modified as appropriate.

[0082] Furthermore, assuming that at least one of the road markings (321, 322) and the sign board 323 is present at the junction 32, the server device 1 may detect only one of the road markings (321, 322) and the sign board 323 as a display object in the image analysis of the third image data 503. Accordingly, in step S10B, the server device 1 may determine whether or not one of the road markings (321, 322) and the sign board 323 has been detected. If one of the objects has been detected, in step S12, the server device 1 may select, as the first partial sentence 551, a sentence that provides guidance toward the charging facility 39 based on the one of the detected objects. If one of the objects has not been detected, in step S13, the server device 1 may select, as the first partial sentence 551, a sentence that provides guidance toward the charging facility 39 based on the other object.

[0083] (Saving of guidance text) The generated guidance text 55 may be stored in any storage area. In one example, the generated guidance text 55 may be stored in a storage area accessible to the terminal of a user who will use the target road rest facility 3 after the target vehicle VE. The user's terminal may be, for example, an in-vehicle device, another user terminal, etc. The arbitrary storage area may be, for example, a memory resource of the server device 1, an external computer (external server, etc.), a memory resource of a terminal, etc. In another example, the generated guidance text 55 is stored in a storage area that is inaccessible to the user's terminal. After that, the information may be provided to the user's terminal as appropriate.

[0084] The generated guidance text 55 may be provided to the user in any manner. In one example, there may be an external server that provides map information. The server device 1 may provide the generated guidance text 55 to the external server. The external server may provide the guidance text 55 together with the map information in response to a request from the user's terminal. In another example, the server device 1 may also serve as a server device that provides map information by storing the map information. In this case, the server device 1 may provide the guidance text 55 together with the map information to each user's terminal.

[0085] (Data format of the guidance text) The data format of the generated guidance sentence 55 may be selected arbitrarily. In one example, the guidance sentence 55 may be composed of only text data, text data and audio data, or only audio data. The audio data constituting the guidance sentence 55 may be generated as appropriate. In one example, in the process of generating the guidance sentence 55, the server device 1 may directly generate audio data. In another example, the server device 1 may generate text data and then convert the generated text data into audio, thereby indirectly generating audio data.

[0086] On the terminal of a user who uses the generated guidance sentence 55, the guidance sentence 55 may be played back as is, or may be played back after any preprocessing has been applied. By adopting the latter, the data format of the guidance sentence 55 may be different when it is generated by the server device 1 and when it is played back on the terminal. For example, the server device 1 may generate the guidance sentence 55 using text data. As preprocessing, the terminal may convert the generated text data into voice. The terminal may play back the voice data obtained by the voice conversion as the guidance sentence 55.

[0087] [2 Configuration Examples] (Server device) 5A schematically illustrates an example of the hardware configuration of the server device 1 according to this embodiment. The server device 1 according to this embodiment is a computer to which a control unit 11, a storage unit 12, a communication interface 13, and a drive 14 are electrically connected.

[0088] The control unit 11 includes a CPU (Central Processing Unit), a RAM (Random Access Memory ), ROM (Read Only Memory), etc., and is configured to execute any information processing based on programs and various data. The control unit 11 (CPU) is an example of a processor resource. The storage unit 12 may be configured with any storage device, such as a hard disk drive, a solid state drive, or a semiconductor memory. The storage unit 12 (as well as RAM and ROM) are examples of memory resources. In this embodiment, the storage unit 12 stores various information related to a program 81, etc. The program 81 is a program for causing the server device 1 to execute information processing (see FIG. 6 described below) related to the generation of the guidance text 55. The program 81 includes a series of instructions for the information processing.

[0089] The communication interface 13 is configured to perform wired or wireless data communication via a network. The communication interface 13 may be configured, for example, by a wired LAN (Local Area Network) module, a wireless LAN module, etc. In this embodiment, the server device 1 may use the communication interface 13 to perform data communication with another computer (for example, the in-vehicle device 2) via the network.

[0090] The drive 14 is a device for reading various information such as programs stored in a storage medium 91. The program 81 may be stored in the storage medium 91 instead of or together with the storage unit 12. The storage medium 91 is a storage medium that can store various information (programs, etc.) by electrical, magnetic, optical, mechanical or chemical action so that a machine such as a computer can read the information. The drive 14 is configured to store the information by the above. The storage medium 91 is an example of a non-transitory storage medium. The storage medium 91 may be a disk-type storage medium such as a CD or a DVD, or may be a non-disk-type storage medium such as a semiconductor memory (for example, a flash memory). The type of the drive 14 may be selected appropriately depending on the type of the storage medium 91.

[0091] Regarding the specific hardware configuration of the server device 1, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 11 may include multiple hardware processors. The hardware processors may be a microprocessor, an FPGA (field-programmable gate array), a DSP (digital signal processor), an EC The server device 1 may be configured with a U (Electronic Control Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), etc. The drive 14 may be omitted. The program 81 may be stored in an external storage device such as a NAS. The external storage device is also an example of a non-transitory storage medium. The server device 1 may further include an input device and an output device. The server device 1 may be a dedicated computer, a general-purpose server device, a general-purpose PC (Personal Computer), etc.

[0092] (In-vehicle device) 5B schematically shows an example of the hardware configuration of the in-vehicle device 2 according to this embodiment. The in-vehicle device 2 according to this embodiment is a computer to which a control unit 21, a storage unit 22, a communication module 23, and a drive 24 are electrically connected. The control unit 21 to the drive 24 and the storage medium 92 of the in-vehicle device 2 may be configured similarly to the control unit 11 to the drive 14 and the storage medium 91 of the server device 1, respectively.

[0093] The control unit 21 (CPU) is an example of a processor resource of the in-vehicle device 2. The storage unit 22 (and RAM, ROM) is an example of a memory resource of the in-vehicle device 2. In this embodiment, the storage unit 22 stores various information such as a program 82. The program 82 is a program for causing the in-vehicle device 2 to execute information processing (see FIG. 6 described below) related to the generation of the guidance text 55. The program 82 includes a series of instructions for the information processing. The program 82 may be stored in a storage medium 92 instead of or together with the storage unit 22.

[0094] It should be noted that, with regard to the specific hardware configuration of the in-vehicle device 2, components can be omitted, replaced, or added as appropriate depending on the embodiment. For example, the control unit 21 may include multiple hardware processors. The drive 24 may be omitted. The program 82 may be stored in an external storage device. The in-vehicle device 2 may further include an input device and an output device. The in-vehicle device 2 may be a dedicated computer, a mobile terminal (including a smartphone), a tablet terminal, a notebook PC, or other general-purpose PC.

[0095] [3 Example of operation] FIG. 6 is a sequence diagram showing an example of a processing procedure for generating a guidance sentence 55 by the system 100 according to this embodiment. The control unit 11 of the server device 1 executes instructions included in a program 81 using a CPU. The control unit 21 of the in-vehicle device 2 also executes instructions included in a program 82 using a CPU. As a result, the server device 1 and the in-vehicle device 2 each operate as a computer capable of executing the information processing shown in FIG. 6 below. The following processing procedure of at least one of the server device 1 and the in-vehicle device 2 is an example of an information generation method (information processing method) executed by a computer. However, the following processing procedure is merely an example, and each step may be changed as much as possible. Furthermore, steps in the following processing procedure may be omitted, replaced, or added as appropriate depending on the embodiment.

[0096] In step S101, the control unit 21 performs a process of recording the image data stored by the driving recorder DR when the target vehicle VE is charged at the charging facility 39 in the target road rest facility 3. In step S102, the control unit 21 extracts target image data 50 from the server device 1. In one example, the vehicle-mounted device 2 may autonomously execute the processes of steps S101 and S102 in response to having used a charging facility 39 at the target road rest facility 3. In another example, the vehicle-mounted device 2 may determine whether or not the charging facility 39 has been used in response to a request from the server device 1. If it is determined that the charging facility 39 has been used, the vehicle-mounted device 2 may execute the processes of steps S101 and S102.

[0097] In step S201, the control unit 11 receives the target image data 50 from the in-vehicle device 2. In step S202, the control unit 11 performs image analysis on the target image data 50. In step S203, the control unit 11 generates guidance text 55 for the charging facility 39 according to the results of the image analysis. In one example, the control unit 11 may generate the guidance text 55 using the method of FIG. 3 above. When a rule-based system is adopted, the control unit 11 may generate the guidance text 55 using the processing procedure of FIG. 4. In one example, the control unit 11 may set the output timing of the generated guidance text 55. In another example, when an end-to-end model is used as the machine learning model, the control unit 11 may simultaneously perform the processes of steps S202 and S203 by providing the target image data 50 to a trained machine learning model and executing arithmetic processing of the trained machine learning model.

[0098] In step S204, the control unit 11 saves the generated guidance text 55. The save destination may be selected appropriately depending on the embodiment. For example, the save destination may be selected from RAM, the memory unit 12, a storage device of an external computer, a storage device of the terminal using the guidance text 55, etc. In one example, the control unit 11 may distribute the guidance text 55 to a user's terminal located near the target road rest facility 3 as part of the storage process or separately from the storage process. The control unit 11 may distribute the guidance text 55 directly to the terminal or indirectly to the terminal via an external server, etc. The control unit 11 may distribute the generated guidance text 55 to a user's terminal located near the target road rest facility 3, thereby issuing a command to output the guidance text 55. The control unit 11 may distribute the guidance text 55 including a setting for output timing, thereby issuing a command to the terminal to output the guidance text 55 at the set output timing. The terminal may include a positioning module and an output device. The terminal may measure its current location using the positioning module, and when the measured current location satisfies the setting of the output timing, output the guidance sentence 55 to the output device. After saving the generated guidance sentence 55, the system 100 ends the processing procedure according to this operation example.

[0099] [Features] In this embodiment, by the processing of steps S202 to S203, it is possible to automatically generate at least a part of the guidance information (guidance text 55) for the charging facility 39 by utilizing the target image data 50 captured by the driving recorder DR of the target vehicle VE that has visited the target road rest facility 3. This is expected to reduce the cost of generating guidance information.

[0100] [4 Variations] Although the embodiments of the present disclosure have been described in detail above, the above description is merely an example of the present disclosure in every respect. The processes and means described in the present disclosure can be freely combined and implemented as long as no technical contradiction occurs. Various improvements or modifications may be made to the above embodiments as appropriate.

[0101] For example, in the above embodiment, the server device 1 is an example of an information processing device. However, the form of the information processing device is not limited to this example. In another example, another computer such as the in-vehicle device 2 may operate as an example of information processing. In the above embodiment, Therefore, at least a part of the information processing of the server device 1 may be executed by the in-vehicle device 2. At least a part of the information processing of the in-vehicle device 2 may be executed by the server device 1. One of the server device 1 and the in-vehicle device 2 may execute a series of information processing from extracting the target image data 50 to saving the guidance sentence 55. In this case, extracting the target image data 50 is an example of acquiring the target image data 50. When generating the guidance sentence 55 by the in-vehicle device 2, the in-vehicle device 2 may save the generated guidance sentence 55 in any memory area. For example, the in-vehicle device 2 may transmit the generated guidance sentence 55 to an external server that accumulates information used for guidance, such as map information. [Explanation of symbols]

[0102] 100...System, 1...server device, 11...control unit, 12...storage unit, 2...In-vehicle device, 21...control unit, 22...storage unit, 3...Road rest facilities, 39...Charging facilities, 40...image data, 50...target image data, 55...Information text, VE: Target vehicle, DR: Driving recorder

Claims

1. a server device, and an on-board device installed in the target vehicle; A system comprising: The in-vehicle device When the target vehicle is charged at a charging facility at a target road rest facility, extracting target image data captured in a section from when the target vehicle starts entering the target road rest facility to when the target vehicle arrives at the charging facility from image data accumulated in a driving recorder of the target vehicle; and transmitting the extracted target image data to the server device; and The server device receiving the target image data from the in-vehicle device; performing image analysis on the received target image data; generating a guide text to the charging facility at the target road rest facility according to the result of the image analysis; and storing the generated guidance text for providing to users who use the target road rest facility; configured to perform system.

2. The computer Acquiring target image data captured from a driving recorder of a target vehicle that uses a charging facility at a target road rest facility during a section from when the target vehicle starts entering the target road rest facility to when the target vehicle arrives at the charging facility; performing image analysis on the acquired target image data; generating a guide text to the charging facility at the target road rest facility according to the result of the image analysis; and storing the generated guidance text for providing to users who use the target road rest facility; To execute Information generation method.

3. The acquired target image data includes first image data captured in a first section from just before the entrance to the target road rest facility to passing through the entrance, When a predetermined facility is depicted in the first image data, the generated guidance sentence includes a sentence providing guidance in the direction of the charging facility based on the predetermined facility. The information generating method according to claim 2 .

4. The acquired target image data further includes third image data captured in a third section including a branch point of the target road rest facility, When the first image data does not include the predetermined facility and the third image data includes a predetermined display object, the generated guidance sentence includes a sentence providing guidance toward the charging facility based on the predetermined display object. The information generating method according to claim 3 .

5. The acquired target image data includes time information indicating a time length of second image data captured in a second section after passing through an entrance to the target road rest facility and before passing through a branch point, generating the guidance sentence includes setting an output timing of the guidance sentence when providing the guidance sentence to the user, in accordance with the time length indicated by the time information; The information generating method according to claim 2 .

Images