System and information generation method

Abstract

The present disclosure provides a system and an information generation method for reducing the cost of generating guidance information to a charging device. An information processing apparatus according to one aspect of the present disclosure performs processing of using an automobile data recorder of an object vehicle using an overcharging device in a road rest facility of an object, object image data captured in a section from when the object vehicle starts to enter the object road rest facility to when the object vehicle arrives at the charging device is acquired, image analysis is performed on the acquired object image data, and on the basis of the result of the image analysis, the object vehicle is charged. A guidance article is generated to a charging device in a target road rest facility, and the generated guidance article is stored in order to provide the guidance article to a user using the target road rest facility.

Description

Technical Field

[0001] This disclosure relates to a system and an information generation method. Background Technology

[0002] Patent document 1 proposes an onboard device that provides facility information, including the location of a charger, when entering the parking area of ​​a facility via a lead road branching off from a main road.

[0003] Prior art literature

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2014-153339 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] One of the objectives of this disclosure is to provide a technique for reducing the cost of generating guidance information for charging devices.

[0008] Methods for solving problems

[0009] The information processing apparatus according to the first aspect of this disclosure includes a control unit. The control unit is configured to perform the following processing: acquiring object image data captured by a vehicle's dashcam during the period from when the object vehicle enters the object's roadside rest facility until it reaches the charging device, using the object's roadside rest facility's charging device; performing image analysis on the acquired object image data; generating guidance text for the charging device in the object's roadside rest facility based on the result of the image analysis; and saving the generated guidance text for provision to users utilizing the object's roadside rest facility.

[0010] The information generation method (information processing method) disclosed in the second aspect is executed by a computer. The information generation method includes the following processes: acquiring image data of the target vehicle taken by a dashcam using a charging device at the target roadside rest facility, covering the period from when the vehicle enters the roadside rest facility until it reaches the charging device; performing image analysis on the acquired image data; generating guidance text for the charging device at the roadside rest facility based on the image analysis results; and saving the generated guidance text for provision to users utilizing the roadside rest facility.

[0011] The system involved in the third aspect of this disclosure includes a server device and an in-vehicle device equipped on the target vehicle. The in-vehicle device is configured to perform the following processing: when the target vehicle is charging using the charging equipment in the target roadside rest facility, extracting target image data captured from image data accumulated in the target vehicle's dashcam during the interval from when the target vehicle enters the target roadside rest facility until it reaches the charging equipment; and sending the extracted target image data to the server device. The server device is configured to perform the following processing: receiving target image data from the in-vehicle device; performing image analysis on the received target image data; generating guidance text for the charging equipment in the target roadside rest facility based on the image analysis results; and saving the generated guidance text for provision to users utilizing the target roadside rest facility.

[0012] Invention Effects

[0013] According to this disclosure, it is possible to reduce the cost of generating guidance information for charging devices. Attached Figure Description

[0014] Figure 1 This is an example illustrating the application of this disclosure.

[0015] Figure 2 An example of a roadside rest facility is shown schematically.

[0016] Figure 3 This schematically illustrates an example of the time elapsed from near the entrance to the charging device and the interval being extracted.

[0017] Figure 4 This represents an example of the processing steps for generating guide articles based on rules.

[0018] Figure 5A An example illustrating the hardware structure of a server device.

[0019] Figure 5B An example illustrating the hardware structure of an onboard device.

[0020] Figure 6 A timing diagram illustrating an example of the processing steps associated with the generation of a guided article implemented by the system of this disclosure. Detailed Implementation

[0021] Directional information at roadside rest facilities such as service areas and parking areas is typically generated manually. However, with the increasing prevalence of electric vehicles such as BEVs (Battery Electric Vehicles), charging facilities at these facilities are frequently being updated (e.g., newly installed). Updating the directional information manually to reflect these frequent updates would be extremely costly.

[0022] In contrast, the information processing apparatus according to the first aspect of this disclosure includes a control unit. The control unit is configured to perform the following processing: acquiring object image data captured by a dashcam of an object vehicle using an overcharging device at the object's roadside rest facility, covering the period from when the object vehicle enters the object's roadside rest facility until it reaches the charging device; performing image analysis on the acquired object image data; generating guidance text for the charging device at the object's roadside rest facility based on the result of the image analysis; and saving the generated guidance text for provision to users utilizing the object's roadside rest facility.

[0023] According to the first aspect of this disclosure, by effectively utilizing image data of the object obtained from the dashcam of the object's vehicle that has visited the object's roadside rest facility, at least a portion of the operation of generating guidance information to the charging device can be automated. Therefore, a reduction in the cost of generating the guidance information can be expected.

[0024] Furthermore, as another aspect of the information processing apparatus (computer) involved in the above-described manner, one aspect of this disclosure can be an information processing method that implements all or part of the above structural elements, an information processing system, a program, or a mechanically readable storage medium such as a computer storing such a program. Here, the mechanically readable storage medium can be a non-transitory medium that stores information such as programs through electrical, magnetic, optical, mechanical, or chemical action. Non-transitory storage media can include storage media (CDs, DVDs, semiconductor memories, etc.), auxiliary storage devices for computers, external storage devices connected to computers, etc. An information processing system can, for example, consist of multiple computers such as server devices or vehicle-mounted devices.

[0025] For example, the information generation method (information processing method) involved in the second aspect of this disclosure is executed by a computer. The information generation method includes the following processes: acquiring object image data captured by a dashcam of an object vehicle using an overcharging device at the object's roadside rest facility, covering the period from when the object vehicle enters the object's roadside rest facility until it reaches the charging device; performing image analysis on the acquired object image data; generating guidance text for the charging device at the object's roadside rest facility based on the result of the image analysis; and saving the generated guidance text for provision to users utilizing the object's roadside rest facility.

[0026] Furthermore, for example, the system involved in the third aspect of this disclosure includes a server device and an in-vehicle device equipped on the target vehicle. The in-vehicle device is configured to perform the following processing: when the target vehicle is charging using the charging equipment in the target roadside rest facility, extracting target image data captured from image data accumulated in the target vehicle's dashcam during the interval from when the target vehicle enters the target roadside rest facility until it reaches the charging equipment; and sending the extracted target image data to the server device. The server device is configured to perform the following processing: receiving target image data from the in-vehicle device; performing image analysis on the received target image data; generating guidance text for the charging equipment in the target roadside rest facility based on the result of the image analysis; and saving the generated guidance text for provision to users using the target roadside rest facility.

[0027] [1 Application Example]

[0028] Figure 1 This illustration schematically depicts an example of an application of the present disclosure. The system 100 of this embodiment includes a server device 1 and an on-board unit 2 mounted on a target vehicle VE. The number of on-board units 2 (target vehicle VE) included in the system 100 can be appropriately determined according to the embodiment.

[0029] The vehicle-mounted device 2 in this embodiment is one or more computers configured to provide image data obtained by a dashcam to a server device. In this embodiment, when the target vehicle VE is charging using the charging device 39 in the target roadside rest facility 3, the vehicle-mounted device 2 extracts target image data 50 captured from the image data 40 stored in the dashcam DR of the target vehicle VE, covering the period from when the target vehicle VE enters the target roadside rest facility 3 until it reaches the charging device 39. The vehicle-mounted device 2 then sends the extracted target image data 50 to the server device 1. Furthermore, the vehicle-mounted device 2 can be either a vehicle-mounted unit that is always installed in the vehicle, or a user terminal that is at least temporarily installed in the vehicle. The type of vehicle-mounted device 2 can be appropriately selected according to the embodiment.

[0030] On the other hand, the server device 1 involved in this embodiment is one or more computers configured to generate guidance text for the facility based on image data. Server device 1 is an example of the information processing apparatus of this disclosure. In this embodiment, server device 1 receives object image data 50 from vehicle-mounted device 2. The process of receiving object image data 50 is an example of the process of obtaining object image data 50. Server device 1 performs image analysis on the received object image data 50. Based on the result of the image analysis, server device 1 generates guidance text 55 for the charging device 39 in the roadside rest facility 3. Server device 1 saves the generated guidance text 55 in order to provide it to users utilizing the roadside rest facility 3.

[0031] According to the system 100 of this embodiment, by effectively utilizing the object image data 50 captured by the dashcam DR of the object vehicle VE that has visited the roadside rest facility 3, at least a portion of the operation of generating guidance information (guidance text 55) to the charging device 39 can be automated. Therefore, a reduction in the cost of generating guidance information can be expected.

[0032] (Dashcam)

[0033] A dashcam (DR) is configured to observe the conditions outside a vehicle (VE) and record the observations using image data. The type of dashcam (DR) is not particularly limited, but can be appropriately selected depending on the implementation method, as long as it can detect objects present during the period from the entrance 30 of the roadside rest facility 3 to the charging device 39. The dashcam (DR) may include all sensors that acquire data in the form of images or image representations. In a typical example, the dashcam (DR) may be a video recorder.

[0034] Furthermore, as long as the situation outside the target vehicle VE can be observed, the configuration of the dashcam DR is not particularly limited, but can be appropriately determined according to the implementation method. In a typical example, the dashcam DR can be positioned near the upper center of the windshield inside the target vehicle VE to record the situation in front of the target vehicle VE through the windshield. The dashcam DR can be always equipped on the target vehicle VE, or it can be temporarily equipped on the target vehicle VE. The vehicle-mounted device 2 can be temporarily connected to the dashcam DR when extracting the target image data 50, or it can be always connected to the dashcam DR.

[0035] (Roadside rest facilities)

[0036] Roadside rest facilities 3 include, for example, rest areas or parking areas located adjacent to roads. Roads include, for example, highways.

[0037] Figure 2 This schematically illustrates an example of a roadside rest facility 3. Figure 2 In the example, a scenario is envisioned where a road designated for left-hand traffic has a first lane R1 and a second lane R2 separated by a central strip, and a roadside rest facility 3 is provided in the first lane R1 on the left. Typically, the roadside rest facility 3 includes an entrance road 31, a main area 33, and an exit road 37.

[0038] Entering Road 31 from the main road (in Figure 2 This is used when entering the main area 33 from the first lane (R1). An entrance 30 may be provided at the boundary of the road leading to the main road 31. The entrance 30 constitutes the entrance to the roadside rest facility 3. Crash barriers 301 may be installed at structures (guardrails, walls, etc.) near the entrance 30. Furthermore, signs 302 may be installed around the entrance 30 to indicate the presence of the roadside rest facility 3 to drivers. Crash barriers 301 and signs 302 are examples of markings for the entrance 30.

[0039] The main area 33 is a rest area for users. Various facilities can be installed in the main area 33. For example, the main area 33 may include a parking lot 34, a designated facility 35, and a charging device 39. The parking lot 34 is a space for parking vehicles. In one example, the parking lot 34 area may be divided according to each type of vehicle, such as two-wheeled vehicles, small cars, large vehicles, and trailers. The designated facility 35 can be a building for users to use. The type of designated facility 35 can be arbitrarily selected. For example, the designated facility 35 may include commercial facilities, rest facilities, gas stations, etc. Commercial facilities may include restaurants, sales outlets, etc. Rest facilities may include restrooms, etc. The charging device 39 is used to charge the battery installed in an electric vehicle. The electric vehicle may include a plug-in hybrid vehicle. The type of charging device 39 can be arbitrarily selected. The charging device 39 may include, for example, a regular charger, a fast charger, etc. The location of the charging device 39 can be appropriately determined according to the implementation method. In one example, the charging device 39 may be configured around the predetermined facility 35, such as near the facility 35, next to the facility 35, in front of the facility 35, etc.

[0040] Furthermore, within the main area 33, multiple lanes can be provided extending towards various areas of the parking lot 34, predetermined facilities 35, charging equipment 39, and other locations. Correspondingly, a fork in the road 32 can be provided near the location where the entrance road 31 merges with the main area 33. The fork in the road 32 can be a location where one lane (entry road 31) branches into multiple lanes. Predetermined displays can be arranged at and around the fork in the road 32. The displays can be appropriately configured to display any information, such as guidance to each direction (branch destination). In one example, the predetermined displays may include at least one of road surface displays (321, 322) and display panels 323.

[0041] Road surface displays (321, 322) are applied to the road surface. Road surface displays (321, 322) may include, for example, displays providing directional guidance, road signs, lane markings, etc. Display panel 323 may be, for example, a sign providing directional guidance, signage, etc. The directional guidance displays may include displays indicating different areas of the parking lot ("small vehicles", "large vehicles", etc.). Figure 2In the example, road surface displays (321, 322) are positioned near the starting point of each lane after the branch, and display panel 323 is positioned on the outer side near the fork in the road 32. However, the placement of the road surface displays (321, 322) and display panel 323 is not limited to this example. The road surface displays (321, 322) and display panel 323 can be positioned at the fork in the road 32 and at any location around the fork in the road 32. The road surface displays (321, 322) and display panel 323 are one example of a display. The display is an example of a marker at the fork in the road 32.

[0042] Exit road 37 is used when exiting from the main area 33 onto the main road. An exit exit point may be provided at the boundary between exit road 37 and the main road. The exit exit point constitutes the exit of the road rest facility 3. Vehicles, including the target vehicle VE, can enter the main area 33 from the main road via entry road 31. Users can park their vehicles in the parking lot 34 of the main area 33 and rest appropriately using the predetermined facilities 35, etc. Afterwards, vehicles can rejoin the main road from the main area 33 via exit road 37. Furthermore, the structure of the road rest facility 3 is not limited to... Figure 2 The example is not fixed, but can be arbitrarily changed. Regarding the structure of road rest facility 3, structural elements can be omitted, replaced, or added as appropriate.

[0043] (Determination of whether overcharged devices were used)

[0044] In the roadside rest facility 3, whether the target vehicle VE uses the overcharging device 39 can be determined by any method. In one example, the use of the overcharging device 39 by the user can be specified by a user's report. The user's report can be implemented, for example, through the operation of the on-board device 2. In another example, whether the target vehicle VE uses the overcharging device 39 can be determined by any information processing.

[0045] As an example of the judgment process, it is also possible to determine whether the target vehicle VE has used the overcharging device 39 by parsing vehicle data. For example, the location of the target vehicle VE can be measured by a positioning module. The positioning module can be composed of a GPS (Global Positioning Satellite) module, a GNSS (Global Navigation Satellite System) module, etc. The positioning module can be installed on the target vehicle VE, the on-board device 2, or other computers (user terminals other than the on-board device 2, etc.). It is possible to determine whether the target vehicle VE has entered the target roadside rest facility 3 based on the location information obtained by the positioning module. In addition, it is possible to determine whether the target vehicle VE has used the overcharging device 39 by checking whether the charging amount has increased based on the battery information (State of Charge (SOC), etc.) of the target vehicle VE.

[0046] As another example of the judgment process, it is also possible to determine whether the target vehicle VE is using the overcharging device 39 by parsing the image data obtained by the dashcam DR. For example, it is also possible to determine whether the target vehicle VE is using the overcharging device 39 based on whether an object present in the roadside rest facility 3 of the target is detected in the image data. The object of the roadside rest facility 3 may include, for example, markings at the entrance 30 (crash cones 301, signs 302, etc.), markings at the intersection 32 (displays, etc.), the predetermined facility 35, the charging device 39, etc.

[0047] The entity performing the decision-making process is not particularly limited, but can be appropriately selected according to the implementation method. Each of the above decision-making processes can be executed by any computer.

[0048] In one example, the vehicle-mounted device 2 can also determine whether the target vehicle VE has used the overcharging device 39 by performing the above-described determination process. The vehicle-mounted device 2 can also perform the determination process automatically or upon request from the server device 1. If it is determined that the target vehicle VE has used the overcharging device 39, the vehicle-mounted device 2 can extract target image data 50 from the image data 40 accumulated by the vehicle's dashcam DR, covering the period from when the target vehicle VE enters the roadside rest facility 3 until it reaches the charging device 39. Furthermore, the vehicle-mounted device 2 can send the extracted target image data 50 to the server device 1. If it is determined that the target vehicle VE has not used the overcharging device 39, the vehicle-mounted device 2 can omit the transmission of the target image data 50.

[0049] In another example, server device 1 can obtain data from the target vehicle VE (vehicle-mounted device 2). Server device 1 can also determine whether the target vehicle VE has used the overcharging device 39 by performing the aforementioned judgment process using the obtained data. Server device 1 can send a signal requesting the provision of target image data 50 for the target vehicle VE (or vehicle-mounted device 2) determined to have used the overcharging device 39. Based on the request signal, vehicle-mounted device 2 can extract the target image data 50 from the image data 40 and send the extracted target image data 50 to server device 1. Furthermore, the process of extracting the target image data 50 does not necessarily have to be performed by vehicle-mounted device 2. Vehicle-mounted device 2 or dashcam DR can send at least a portion of the image data 40, including the target image data 50, to server device 1. Server device 1 can extract the target image data 50 from at least a portion of the obtained image data 40.

[0050] In another example, the determination of whether the target vehicle VE has used the overcharging device 39 can also be performed by a computer other than server device 1 and vehicle-mounted device 2 (another server device, etc.). The other computer can also issue a command to send the target image data 50 to server device 1 for the target vehicle VE (or vehicle-mounted device 2) determined to have used the overcharging device 39. Based on the command from the external computer, vehicle-mounted device 2 can also extract the target image data 50 from the image data 40 and send the extracted target image data 50 to server device 1. The external computer can also issue a command to server device 1 authorizing vehicle-mounted device 2 to send the target image data 50. Furthermore, if vehicle-mounted device 2 or dashcam DR has previously provided at least a portion of the image data 40, including the target image data 50, to an external computer, the external computer can extract the target image data 50 and send the extracted target image data 50 to server device 1. Alternatively, an external computer may provide at least a portion of the held image data 40 to the server device 1, and cause the server device 1 to perform the extraction of the object image data 50.

[0051] (Object image data)

[0052] The object image data 50 is constituted by image data captured by the dashcam DR during at least a portion of the time from when the object vehicle VE begins entering the roadside rest facility 3 until it reaches the charging device 39. In one example, the object image data 50 may be constituted by image data for all the time from the start of entry into the roadside rest facility 3 to the arrival at the charging device 39. In another example, the object image data 50 may also be constituted by image data for a portion of time (interval).

[0053] The point at which the vehicle begins to enter the roadside rest facility 3 can be defined as any point in time when the vehicle VE is traveling around the entrance 30. In one example, the point at which the vehicle begins to enter the roadside rest facility 3 can be defined either as the point at which the vehicle VE passes the entrance 30, or as the point at which the vehicle VE arrives near the entrance 30. "Near" the entrance 30 can be defined, for example, as X seconds before passing the entrance 30, or as a position Dx m closer to the entrance 30 than the entrance 30, using the entrance 30 as a reference. The reference values ​​(X seconds, Dx m) can be arbitrarily set. The reference values ​​can be fixed or variable (e.g., values ​​determined based on the speed of the vehicle VE). The point at which the vehicle arrives at the charging device 39 can be defined, for example, as the point at which the vehicle VE stops in front of the charging device 39, using the charging device 39 as a reference.

[0054] The composition of the object image data 50 is not particularly limited, but can be appropriately determined according to the implementation method, as long as it includes image data from at least a portion of the time from when the object vehicle VE enters the roadside rest facility 3 to when it arrives at the charging device 39. In one example, the object image data 50 may include image data (also referred to as interval image data) in one or more defined intervals during the period from when the object vehicle VE enters the roadside rest facility 3 to when it arrives at the charging device 39. In addition, the object image data 50 may also include image data in intervals other than the time from the start of entry to arrival, such as before the object vehicle VE enters the roadside rest facility 3. The object image data 50 may be composed of multiple still images or moving images. If the dashcam DR is equipped with a microphone, the object image data 50 may further include audio data acquired together with the image data. The audio data may also be omitted.

[0055] The object image data 50 can be extracted from the image data 40 stored in the dashcam DR by any method. In one example, at least one of the object vehicle VE and the on-board unit 2 may have a timer, and vehicle data related to the movement of the object vehicle VE can be recorded along with the time. Accordingly, the time that serves as the reference for extraction can be specified based on the vehicle data. For example, the occurrence of events related to the use of the charging device 39 can be identified based on the vehicle data. Events may include, for example, the object vehicle VE being located near the entrance 30, passing through the entrance 30, passing through the intersection 32, stopping the object vehicle VE in front of the charging device 39, etc., characteristic events during the period from the start of entering the road rest facility 3 to arriving at the charging device 39. Events related to the position of objects other than the charging device 39, such as the object vehicle VE being located near the entrance 30, passing through the entrance 30, passing through the intersection 32, etc., can be identified based on the position information obtained by the positioning module. For example, the moment when the target vehicle VE is identified as being located at the entrance 30 based on location information can be specified as the moment when the target vehicle VE passes through the entrance 30. Charging-related events, such as the target vehicle VE stopping in front of the charging device 39, can be identified based on battery information. For example, the moment before the charge level of the target vehicle VE is about to increase can be identified based on battery information, and the identified moment can be specified as the moment when the target vehicle VE stops in front of the charging device 39. Furthermore, the method for specifying the occurrence time of each event is not limited to these examples and can be appropriately modified according to the implementation. For example, the moment when the ignition switch is turned off in front of the charging device 39 can be specified as the moment when the target vehicle VE stops in front of the charging device 39. Moreover, the interval from which the target image data 50 is extracted from the image data 40 can be specified based on the occurrence time of the identified events. In another example, the moment that serves as the basis for extraction can be specified based on the image data 40 itself. For example, the occurrence of the above events can be identified through image analysis, and the interval from which the object image data 50 is extracted can be specified based on the time of occurrence of the identified events.

[0056] Furthermore, the process of extracting object image data 50 can be performed either in conjunction with the aforementioned determination process for determining whether the object vehicle VE has used the charging device 39 in the object's roadside rest facility 3, or it can be performed independently of the aforementioned determination process. In one example, the process of extracting object image data 50 can also be performed at least partially together with the aforementioned determination process. For example, it can be determined whether the object vehicle VE has used the charging device 39 in the object's roadside rest facility 3 based on whether the passage through the entrance 30 is detected and the occurrence of events such as charging in the charging device 39. In addition, it can also be extracted from image data 40, which stores object image data 50, based on the occurrence time of each detected event. In another example, the process of extracting object image data 50 can also be performed independently of the aforementioned determination process.

[0057] The method by which the vehicle-mounted device 2 sends the object image data 50 to the server device 1 is not particularly limited, but can be appropriately selected according to the implementation method. In one example, the vehicle-mounted device 2 can send the object image data 50 directly to the server device 1. Thus, the server device 1 can directly obtain the object image data 50 from the vehicle-mounted device 2. In another example, the vehicle-mounted device 2 can also send the object image data 50 to a storage area other than the server device 1, such as a storage medium or other computers. Other computers may include data servers such as NAS (Network Attached Storage). Accordingly, the server device 1 can indirectly obtain the object image data 50 via a storage medium, other computers, etc. If the processing of extracting the object image data 50 is not performed in the vehicle-mounted device 2, the method of sending at least a portion of the image data 40, including the object image data 50, from the dashcam DR or the vehicle-mounted device 2 to the server device 1 or other computers can also be the same.

[0058] (filter)

[0059] The object vehicle VE can be any vehicle that utilizes the overcharging device 39 at the object's roadside rest facility 3. That is, object image data 50 can be obtained from any vehicle that utilizes the overcharging device 39 at the object's roadside rest facility 3. In a typical example, object image data 50 can be obtained from a vehicle that has utilized a charging device (charging device 39) that is not present in the map information, or that is present in the map information but does not contain guidance information.

[0060] Furthermore, in one example of this embodiment, it can be determined whether the shooting conditions in the vehicle meet predetermined conditions. The object image data 50 used in generating the guide article 55 can be obtained from vehicles whose shooting conditions are determined to meet predetermined conditions. That is, the object vehicles VE for which the object image data 50 for generating the guide article 55 is obtained can be filtered according to whether the shooting conditions meet predetermined conditions. The predetermined conditions can be appropriately defined in a manner suitable for image processing when generating the guide article 55 (e.g., easy to detect objects, etc.). In one example, the predetermined conditions may include at least one of the dashcam DR's field of view conditions, the vehicle's driving conditions, and the vehicle's category conditions.

[0061] The field of view conditions of a dashcam (DR) can be appropriately defined by extracting vehicles with a relatively good field of view from the DR. In one example, the field of view conditions of the DR can be defined based on at least one of the weather conditions at the time of arrival at the roadside rest stop 3 and the recording environment of the DR. For example, weather-related field of view conditions can be defined by extracting vehicles arriving at the roadside rest stop 3 in weather conditions other than those obstructing the view. Obstructing weather conditions could be, for example, rain, fog (including dense fog), snow, hail, etc. Weather-related conditions can be determined by weather information in the area of ​​the roadside rest stop 3. Weather information can be appropriately obtained from an external server that provides weather information.

[0062] The field of view conditions related to the shooting environment can be defined by extracting images of the vehicle captured by the dashcam (DR) in environments other than those obstructing the field of view. Obstructing environments may include, for example, dirt on the dashcam's lens or lens cover, or obstacles obstructing the view within the dashcam's field of view. When the dashcam is configured to capture images of the vehicle's exterior through the windshield, determining the presence of obstructions may include assessing whether the windshield is dirty. These environmental conditions can be determined based on the images acquired by the dashcam.

[0063] Furthermore, if the vehicle is traveling at a high speed, jitter will occur in the image obtained by the dashcam (DR), potentially leading to a decrease in the accuracy of image resolution. Therefore, the vehicle's driving conditions can be defined by extracting vehicles traveling at speeds below a threshold within the time interval of acquiring the object image data (50). The speed threshold can be arbitrarily set. The vehicle's driving conditions can be determined based on the vehicle's speed data. The speed data can be appropriately obtained from the object vehicle (VE).

[0064] Furthermore, most vehicles using roadside rest facilities are small cars. By utilizing image data obtained from small cars in the generation of guidance text 55, it is expected that guidance text 55 can be generated from a viewpoint similar to that of most vehicles using roadside rest facilities. Therefore, vehicle category conditions can be defined by extracting small cars. Vehicle category conditions can be determined based on vehicle attribute information. Vehicle attribute information can be appropriately obtained. In addition, small cars can be appropriately defined in a way that belongs to general vehicles. For example, a small car can be defined as a vehicle with a length of 4.7m or less, a width of 1.7m or less, and a height of 2.0m or less.

[0065] The predetermined conditions may include at least one of the conditions described above. In one example, the predetermined conditions may include all of the dashcam (DR) field-of-view conditions, vehicle driving conditions, and vehicle category conditions. In this case, the object image data 50 used in generating the guidance text 55 can be obtained from vehicles that meet the conditions of good field of view, driving speed less than a threshold, and being small cars. By including at least one of the aforementioned field-of-view conditions and driving conditions in the predetermined conditions, object image data 50 suitable for image parsing can be obtained. Therefore, an improvement in the generation accuracy of the guidance text 55 can be expected. Furthermore, by meeting the vehicle category conditions in the predetermined conditions, object image data 50 with a field of view similar to that of most vehicles using the roadside rest facility 3 can be obtained. Therefore, it is expected that a guidance text 55 suitable for most users can be generated.

[0066] The process of filtering the target vehicles VE based on the above shooting conditions can be performed at any time. In one example, filtering the target vehicles VE for obtaining the target image data 50 used to generate the guide article 55 can be configured by selecting image data that meets the conditions from the obtained image data based on image data obtained from all vehicles, and using this as the target image data 50. In another example, filtering the target vehicles VE can also be configured by selecting vehicles that meet the conditions from those that have visited the roadside rest facility 3 before obtaining the image data, and obtaining the target image data 50 from the selected vehicles.

[0067] Furthermore, the process of filtering the target vehicle VE based on the aforementioned shooting conditions can be performed by any computer. Similarly, the process of determining whether the target vehicle VE has used the charging equipment 39 in the roadside rest facility 3 can be performed by at least one of the vehicle-mounted device 2, the server device 1, and other computers.

[0068] Additionally, the process of filtering the target vehicles VE based on shooting conditions can be omitted. In another example, the target vehicles VE for which the target image data 50 is obtained can also be randomly selected from the vehicles using the overcharging device 39 at the roadside rest facility 3. In yet another example, the target image data 50 can also be obtained from all vehicles using the overcharging device 39 at the roadside rest facility 3.

[0069] (Image Analysis)

[0070] The content of image analysis of object image data 50 can be appropriately determined according to the implementation method. In one example, image analysis of object image data 50 may include detecting objects reflected in object image data 50, inferring the position of objects, inferring the distance of objects, inferring the positional relationship between multiple objects, and identifying the type of objects. Objects may be, for example, markers at entrance 30 (crash cones 301, signs 302, etc.), markers at intersections 32 (displays, etc.), predetermined facilities 35, charging equipment 39, etc.

[0071] Furthermore, the method for image parsing of the object image data 50 can be appropriately selected depending on the implementation method. In one example, the server device 1 can parse the object image data 50 using general image parsing techniques such as edge detection and pattern matching. In another example, the server device 1 can also use a trained machine learning model that has acquired the ability to parse images to perform image parsing on the object image data 50. The machine learning model is configured to have one or more operational parameters that can be adjusted through machine learning. The one or more operational parameters are used in operations aimed at inference (image parsing, etc.). The machine learning model can be configured, for example, by neural networks, support vector machines, other functional (operational models), etc. The machine learning method can be appropriately selected depending on the machine learning model used (e.g., backpropagation). Training the machine learning model means adjusting (optimizing) the values ​​of the operational parameters using training samples. The machine learning model can be appropriately trained in a way that derives the true value of the corresponding parsing result when given an image of a training sample. In the trained machine learning model, a large-scale model such as a large-scale visual language model (VLM) can also be used. In addition, if the object image data 50 includes sound data, the image parsing of the object image data 50 may also include the sound parsing of the sound data.

[0072] (Methods for generating guide articles)

[0073] The guidance text 55 can be appropriately generated based on the results of image analysis of the object image data 50. In a typical example, the server device 1 can detect objects that appear during the period from the object vehicle VE moving from the entrance 30 to the charging device 39. The server device 1 can generate a guidance text 55 that guides the charging device 39 based on the detected objects. That is, the guidance text 55 can be generated in such a way that the charging device 39 is guided based on the objects observed by the dashcam DR from the entrance 30 to the charging device 39. The objects (objects) that serve as the reference for guidance can be appropriately selected according to the implementation. Furthermore, the results of image analysis can be appropriately applied to the generation of the guidance text 55. For example, in the case of object identification during image analysis, the server device 1 can generate a guidance text 55 containing the name of the identified object, such as "The charging device is located near the AAA facility". Furthermore, for example, in image analysis, when inferring the distance to the object, server device 1 can also generate guiding text 55 containing the inferred distance, such as "The charging device is located approximately XX meters in front of the AAA facility." Furthermore, for example, in image analysis, when inferring the positional relationship between the charging device 39 and the object, server device 1 can also generate guiding text 55 containing the inferred positional relationship, such as "The charging device is located to the right of the AAA facility."

[0074] The method for generating the guide text 55 is not particularly limited, but can be appropriately selected according to the implementation method. In one example, the guide text 55 can be generated based on rules. In the case of using a rule-based approach, the server device 1 can generate the guide text 55 according to the rules and the results of image analysis. The generated rules can be appropriately set. When generating the guide text 55, templates such as fixed-format articles can also be used. Generating the guide text 55 using a template may include at least one of selecting a fixed-format article to be used from a plurality of prepared fixed-format articles based on the results of image analysis, and inputting the results of image analysis into a specified part of the fixed-format article. In addition, the plurality of prepared fixed-format articles may include, for example, fixed-format articles such as "charging device is" used therein. The template can be stored in the memory resources of the server device 1, provided from an external computer, or embedded in the program of the server device 1.

[0075] In another example, similar to image parsing, a trained machine learning model can also be used in the generation of the guiding article 55. The machine learning model can be configured as described above. When a trained machine learning model is used in both image parsing and the generation of the guiding article 55, the machine learning model used in image parsing and the machine learning model used in the generation of the guiding article 55 can be either different models or the same model.

[0076] In the former case, the first machine learning model used in image parsing can be configured to accept image data (object image data 50) as input and output the result of image parsing for the input image data. The second machine learning model used in generating the guiding text 55 can be configured to accept the result of image parsing as input and output the generated result of guiding text 55 corresponding to the result of image parsing. Each machine learning model can also be configured to accept input of arbitrary information in addition to the above-mentioned inputs. Furthermore, each machine learning model can also be a dedicated model. Each dedicated model can be appropriately trained to acquire the ability to perform the above-mentioned inference processes (image parsing, generation of guiding text 55). Alternatively, each machine learning model can also be configured from general models such as large-scale visual language models or large-scale language models (LLM). In one example, a large-scale visual language model can be used in the first machine learning model, and a large-scale language model can be used in the second machine learning model. Large-scale models such as large-scale visual language models and large-scale language models acquire the ability to perform in-context learning. When using a machine learning model capable of performing in-context learning, server device 1 can also provide pre-hints, such as instructions for inference content, along with input information to the machine learning model. As an example, if a large-scale visual language model is used in the first machine learning model, server device 1 can provide instructions for inference, such as "Please detect objects present around the charging device" or "Please identify the category of the object," along with object image data 50 to the first machine learning model. If a large-scale language model is used in the second machine learning model, server device 1 can also provide instructions for inference, such as "Please generate a guiding text for the charging device based on the given information," along with the results of image parsing to the second machine learning model. Instructions for each inference can be appropriately provided using templates or the like.

[0077] On the other hand, when the latter approach is adopted, the machine learning model can be configured to accept image data (object image data 50) as input and output a generated guide text 55 corresponding to the input image data. That is, the machine learning model can be configured as an end-to-end model. In this case, image parsing of the object image data 50 and generation of the guide text 55 based on the image parsing results can be performed simultaneously during the computation of the machine learning model. Similarly, in the machine learning model, both specialized and general models can be used. The machine learning model can also be configured as a large-scale visual language model or a model capable of performing in-context learning. When using a machine learning model capable of performing in-context learning, the server device 1 can also provide the machine learning model with pre-hints such as inference content instructions along with the input information. As an example, in the case of employing a large-scale visual language model in the machine learning model, server device 1 can also provide the machine learning model with inference instructions such as "(I) Please detect objects existing around the charging device. (II) Please generate a guide text to the charging device based on the detection results of (I)" along with object image data 50. The inference instructions can be appropriately provided through templates, etc.

[0078] In the above Figure 2 In the illustrated roadside rest facility 3, under normal circumstances, vehicles enter the rest facility 3 from entrance 30 and proceed to charging equipment 39 via junction 32. Therefore, the generated guidance text 55 may include at least one of the following: a first part guiding the orientation of charging equipment 39 based on the range from entrance 30 to near junction 32, and a second part guiding the location of charging equipment 39 after junction 32. The location of charging equipment 39 can be represented, for example, by its positional relationship with a predetermined facility 35. In either the rule-based approach or the machine learning model approach described above, server device 1 can generate at least one of the first and second parts. The first part can be generated based on the results of image analysis of image data up to near junction 32. The second part can be generated based on the results of image analysis of image data from passing junction 32 to reaching charging equipment 39. The first part of the text can be used to guide the location of the charging device 39, based on a position slightly away from it. The second part of the text can be used to indicate the location of the charging device 39, based on a position close to it.

[0079] When providing the generated guide text 55 to a user utilizing the roadside rest facility 3 after the target vehicle VE, the timing of outputting the generated guide text 55 can be arbitrarily defined. In one example, the timing of outputting the guide text 55 can be uniformly defined, such as the timing of passing through the entrance 30. However, the size of the roadside rest facility 3 may vary depending on the location. In particular, the length of the entrance road 31 may vary. Even if the guide text 55 is output from a location significantly away from the main area 33, the information guided by the guide text 55 may deviate greatly from the current location, making it difficult to determine the location of the charging device 39. Therefore, the server device 1 can also set the output timing of the guide text 55 based on the time length of any interval in the target image data 50. That is, generating the guide text 55 may include setting the output timing of the guide text 55 when providing the guide text 55 to the user. The interval that serves as the reference for the output timing can be appropriately defined according to the implementation method. For example, the interval that serves as the reference for output timing can be defined in at least a portion of the interval from near the entrance 30 to near the fork in the road 32.

[0080] As described above, the guide text 55 (each section of the text) can be generated based on a portion of the journey (a specific interval) from when the target vehicle VE enters the roadside rest facility 3 from the entrance 30 and reaches the charging device 39. Similarly, the output timing setting can also be specified based on a portion of the interval. Therefore, more than one specific interval can be extracted during the time when the target vehicle VE moves from near the entrance 30 to the charging device 39. Each interval can be extracted in any way. In one example, each interval can be extracted either by appropriately dividing the movement time from the entrance 30 or by specifying a range that sufficiently satisfies the conditions.

[0081] Figure 3 This schematically illustrates an example of the time elapsed from near the inlet 30 to reaching the charging device 39, and the extracted interval. Figure 3 In one example, the following scenario is envisioned: the object vehicle VE enters from entrance 30. Figure 2 The road rest facility 3 shown is moved to the charging equipment 39 via the access road 31 and the fork in the road 32.

[0082] In one example, during the process (time) of moving from near the entrance 30 to the charging device 39, a first interval can be extracted from near the entrance 30 of the roadside rest facility 3 to the point where the vehicle passes through the entrance 30. This first interval can be extracted using any method. For example, as mentioned above, the time point at which the vehicle VE passes through the entrance 30 can be specified using location information, image analysis, etc. The time point at which the vehicle VE is near the entrance 30 can be appropriately specified based on the specified time point at which the vehicle VE passes through the entrance 30. In one example, such as... Figure 3 As shown, the nearest time point to entrance 30 can be specified based on the elapsed time of entrance 30 (X seconds ago from entrance 30). In another example, a reference other than time (e.g., distance) can also be used, and the nearest time point to entrance 30 can be specified based on the elapsed time of entrance 30. The nearest time point to entrance 30 can also be specified without using the elapsed time of entrance 30 as a reference. For example, the nearest time point to entrance 30 can be specified using location information, image analysis (whether entrance 30 is reflected, etc.). Thus, the first interval can be extracted.

[0083] When extracting from the first interval, the object image data 50 may include first image data 501 captured within the first interval. The first image data 501 may also be referred to as first interval image data. Image analysis of the object image data 50 may include determining whether a predetermined facility 35 is reflected in the first image data 501. If the predetermined facility 35 is reflected in the first image data 501, the generated guidance text 55 may include text guiding the orientation of the charging device 39 based on the predetermined facility 35 (Part 1-1 text 5511). Part 1-1 text 5511 is an example of the aforementioned first part text. Part 1-1 text 5511 may, for example, consist of one or more statements guiding the charging device 39 based on the predetermined facility 35, such as "Head towards the AAA facility." It is common for the charging device to be located near the predetermined facility. According to an example of this embodiment, when a predetermined facility 35 can be seen within the initial range of the roadside rest facility 3 leading to the object, a guidance text 55 (part 1-1 text 5511) can be generated to guide the charging device 39 based on the predetermined facility 35. Thus, it is expected that a guidance text 55 capable of appropriately guiding the charging device 39 can be generated.

[0084] Furthermore, in one example, a third section, including the fork in the road leading to the rest stop 32 containing the object, can be extracted during the movement from near the entrance 30 to the charging device 39. The third section can be appropriately defined in such a way that it includes a range around the fork in the road 32 capable of displaying the object. For example... Figure 3 As shown, in one example, the third interval can be defined as the range from the point in time when the vehicle is near the intersection 32 to the point in time when it has passed through the intersection 32. The third interval can also include areas beyond the intersection 32. The third interval can be extracted using any method. For example, as mentioned above, the point in time when the vehicle VE passes through the intersection 32 can be specified using location information, image parsing, etc. The point in time when the vehicle VE is near the intersection 32 can be appropriately specified based on the specific point in time when the vehicle VE passes through the intersection 32. In one example, as... Figure 3 As shown, the nearest time point to the fork in the road 32 can be specified based on the elapsed time of the fork in the road 32 (Y seconds before passing the fork in the road 32). In another example, a reference other than time (e.g., distance) can be used, and the nearest time point to the fork in the road 32 can be specified based on the elapsed time of the fork in the road 32. The nearest time point to the fork in the road 32 can also be specified without using the elapsed time of the fork in the road 32 as a reference. For example, the nearest time point to the fork in the road 32 can also be specified through location information, image analysis (detection of displayed objects, etc.). Thus, the third interval can be extracted.

[0085] When extracting the third interval, the object image data 50 may further include the third image data 503 captured in the third interval. The third image data 503 may also be referred to as the third interval image data. Image analysis of the object image data 50 may include determining whether a predetermined display is reflected in the third image data 503. The predetermined display may be, for example, the aforementioned road surface display (321, 322), display panel 323, etc. If the predetermined facility 35 is not reflected in the first image data 501, but the predetermined display is reflected in the third image data 503, the generated guidance text 55 may include a text that guides the orientation of the charging device 39 based on the predetermined display (text 5512 in part 1-2). Text 5512 in part 1-2 is an example of the first part of the text described above. Article 5512 in Part 1-2 can be composed of one or more statements that guide the user to the charging device 39, such as "Proceed towards the BBB area," by displaying information related to the lanes leading to the charging device 39 among multiple lanes extending from the fork in the road 32. In the case of multiple displays, the user can appropriately select from the displays shown by each display the direction of travel of the target vehicle VE, the display guiding the target vehicle VE's direction of travel, and other displays suitable for the target vehicle VE's direction of travel to the charging device 39. At the fork in the road 32, the path is divided into multiple lanes. Therefore, if the predetermined facility 35 is not visible within the initial entry area, the user arriving at the fork in the road 32 may become lost in the selection of lanes leading to the charging device 39. In contrast, according to an example of this embodiment, if the predetermined facility 35 is not visible within the initial entry area, a guidance article 55 (Article 5512 in Part 1-2) can be generated to guide the user to the branch destination at the fork in the road 32. Therefore, it is expected that a guiding article 55 can be generated that can properly guide the charging device 39.

[0086] Furthermore, if the predetermined facility 35 is not detected in the first image data 501 and the display object is not detected in the third image data 503, the server device 1 can detect any object (target object) from the images obtained from the beginning of the first interval to the end of the third interval. The target object can be appropriately selected according to the implementation. For example, the server device 1 can also detect the predetermined facility 35 as a target object from the images in the range outside the first interval. The server device 1 can also detect other objects besides the display object and the predetermined facility 35 as targets from the images obtained from the beginning of the first interval to the end of the third interval. Image analysis of the object image data 50 can include determining whether the target object is reflected in the image data from the beginning of the first interval to the end of the third interval. The generated guidance text 55 can also include a text that guides the orientation of the charging device 39 based on the detected target object (text 5513 in parts 1-3). Alternatively, server device 1 may omit the process of generating the first part of the article (parts 1-3, article 5513).

[0087] Furthermore, in one example, during the movement from near the entrance 30 to the charging device 39, a fourth interval can be extracted from near the charging device 39 to stopping at the charging device 39. The fourth interval can be extracted using any method. For example, as mentioned above, the time point at which the target vehicle VE stops at the charging device 39 can be specified using battery information, image analysis, etc. The time point at which the target vehicle VE is near the charging device 39 can be appropriately specified based on the time point at which the target vehicle VE stops at the charging device 39. In one example, such as Figure 3 As shown, the nearest time point of the charging device 39 can be specified based on the moment when the charging device 39 stops at the target vehicle VE (Z seconds before stopping at the charging device 39). In another example, a reference other than time (e.g., distance) can also be used, and the nearest time point of the charging device 39 can be specified based on the stopping time point in the charging device 39. The nearest time point of the charging device 39 can also be specified through image analysis (whether the charging device 39 is reflected, etc.). Thus, the fourth interval can be extracted.

[0088] When extracting the fourth interval, the object image data 50 may include the fourth image data 504 captured within the fourth interval. The fourth image data 504 may also be referred to as the fourth interval image data. Image analysis of the object image data 50 may include inferring the positional relationship between the predetermined facility 35 and the charging device 39 reflected in the fourth image data 504. The generated guide text 55 may include a guide text (second part text 552) that guides the positional relationship between the predetermined facility 35 and the charging device 39. The positional relationship may be represented using the direction in which the charging device 39 exists, such as the right direction of the facility, the left direction of the facility, etc., with the object as the reference. The positional relationship may also be represented by a relative position with the object as the reference, such as near the facility, next to the facility, in front of the facility, etc. If at least one of the predetermined facility 35 and the charging device 39 is not detected from the fourth image data 504, the process of generating the second part text 552 may be omitted.

[0089] The content of the second part of article 552 can be appropriately selected based on the positional relationship between the predetermined facility 35 and the charging device 39. For example, if it is inferred that the charging device 39 exists near the predetermined facility 35 (the first relationship is established), as an example of the second part of article 552, the server device 1 can generate a second-first part of article 5521 that guides the content that the charging device 39 exists near the predetermined facility 35. "Near the predetermined facility 35" can mean that the charging device 39 exists at a location away from the predetermined facility 35 towards the entrance 30 (fork in the road 32). Furthermore, for example, if it is inferred that the charging device 39 exists next to the predetermined facility 35 (the second relationship is established), as an example of the second part of article 552, the server device 1 can generate a second-second part of article 5522 that guides the content that the charging device 39 exists next to the predetermined facility 35. "Next to the predetermined facility 35" can mean that the charging device 39 exists in the vicinity of the predetermined facility 35. Furthermore, for example, if it is inferred that the charging device 39 exists in front of the predetermined facility 35 (the third relationship is established), as an example of Part 2 article 552, the server device 1 can generate Part 2-3 article 5523 that guides the content regarding the existence of the charging device 39 in front of the predetermined facility 35. "Existing in front of the predetermined facility 35" can mean that the charging device 39 exists separately from the predetermined facility 35 when viewed from the entrance 30 (fork in the road 32) side, beyond the position of the predetermined facility 35. Whether the charging device 39 exists near the predetermined facility 35 or is far from the predetermined facility 35 can be determined by any method. For example, it can be determined whether the charging device 39 exists near the predetermined facility 35 or is far from the predetermined facility 35 based on the location information of the parking time of the target vehicle VE and the location information of the predetermined facility 35. Furthermore, for example, it can be determined whether the charging device 39 is located near or away from the predetermined facility 35 based on whether the predetermined facility 35 and the charging device 39 are detected simultaneously in the fourth image data 504. Additionally, the types of the second part text 552 corresponding to the positional relationship are not limited to the three mentioned above, but can be appropriately changed according to the implementation method. The types of the second part text 552 can be set to two or fewer, or even four or more. According to an example of this embodiment, a guidance text 55 (second part text 552) guiding the positional relationship with the predetermined facility 35 can be generated. Thus, it is expected that a guidance text 55 capable of appropriately guiding the charging device 39 can be generated.

[0090] Furthermore, in one example, during the movement from near the entrance 30 to the charging device 39, a second interval can be extracted after passing the entrance 30 of the road rest facility 3 and before passing the fork in the road 32. This second interval can be appropriately defined between the entrance 30 and the fork in the road 32 (i.e., the area leading to the road 31). Figure 3 As shown, in one example, the second interval can be defined as the interval between the end of the first interval and the start of the third interval. Thus, the second interval can be continuous with both the first and third intervals. In this case, the second interval can be easily detected after the first and third intervals have been detected. However, the definition of the second interval is not limited to this example. The second interval may also partially overlap with at least one of the first and third intervals. The second interval may also contain the third interval. The second interval may also not be continuous with at least one of the first and third intervals. That is, the second interval may also be defined in a way that distances it from at least one of the end of the first interval and the start of the third interval.

[0091] When extracting the second interval, the object image data 50 may include time information 520 indicating the time length of the second image data 502 captured in the second interval. The second image data 502 may also be referred to as the second interval image data. The object image data 50 may or may not include the second image data 502. Generating the guide article 55 may include setting the output timing of the guide article 55 when providing the guide article 55 to the user, based on the time length indicated by the time information 520.

[0092] The relationship between the duration of the second interval and the setting of the output timing can be appropriately defined according to the implementation method. For example, if the duration of the second interval is less than a first threshold, the output timing can be set to the time point elapsed from the inlet 30. If the duration of the second interval exceeds the first threshold but is less than the second threshold, the output timing can be set to the time point exceeding the inlet 30 by D1 m. If the duration of the second interval is equal to the first threshold, the output timing can also be set to either the time point elapsed from the inlet 30 or the time point exceeding the inlet 30 by D1 m. If the duration of the second interval exceeds the second threshold, the output timing can be set to the time point exceeding the inlet 30 by D2 m. If the duration of the second interval is equal to the second threshold, the output timing can also be set to either the time point exceeding the inlet 30 by D1 m or the time point exceeding the inlet 30 by D2 m. The first threshold can be appropriately set in a manner that makes the output guide 55 at the inlet 30 suitable for guiding to the charging device 39. The second threshold can be appropriately set to a value larger than the first threshold. D1m can be appropriately set according to the second threshold. D2m can be appropriately set to a value larger than D1m. Furthermore, the types of output timing corresponding to the time length of the second interval are not limited to the three mentioned above, but can be appropriately changed according to the implementation method. The types of output timing can be set to two or fewer, or even four or more. The criterion for judging output timing is not limited to the entrance 30, but can be appropriately set in and around the road rest facility 3. Furthermore, the output timing index is not limited to distance. For example, output timing can also be calculated using other indicators such as time. The time length of the second interval can correspond to the travel time to enter the road 31. The longer the time length of the second interval, the farther the entrance 30 is from the main area 33. The farther the entrance 30 is from the main area 33, the more difficult it may be to determine the location of the charging device 39 even if the guide article 55 is output near the entrance 30. In contrast, according to an example of this embodiment, by setting the output timing of the guide article 55 with the time length of the second interval as an indicator, it is possible to expect output control of the guide article 55 under the master timing of the charging device 39.

[0093] Using the above Figure 3In the case of the complete structure, the object image data 50 can be configured to include first image data 501, time information 520 of second image data 502, third image data 503, and fourth image data 504. If the first interval is omitted, the first image data 501 can be omitted from the object image data 50. If the second interval is omitted, the time information 520 can be omitted from the object image data 50. If the third interval is omitted, the third image data 503 can be omitted from the object image data 50. If the fourth interval is omitted, the fourth image data 504 can be omitted from the object image data 50.

[0094] Furthermore, the object image data 50 may or may not include image data from other intervals besides those mentioned above. For example, by moving the charging device 39 away from the fork in the road 32, an interval can be generated between the end of the third interval and the beginning of the fourth interval. The object image data 50 may or may not include image data from this interval. At least a portion of the guiding text 55 can also be generated based on image data from other intervals. Additionally, in Figure 3 In the example, the third and fourth intervals are separate. However, the third and fourth intervals are not limited to this example. By positioning the charging device 39 closer to the fork in the road 32, the third and fourth intervals can be either continuous or partially overlapping. Each interval can be extracted during any operation, such as extraction of object image data 50 or image parsing.

[0095] (Specific example of the processing steps for generating guide articles)

[0096] Figure 4 This illustrates an example of the processing steps for generating guide article 55 based on rules. Figure 4 In one example, it is envisioned that when using the above... Figure 3 Based on the entire structure, a scenario for guiding article 55 is generated according to rules. Before processing begins from step S10A, server device 1 can obtain object image data 50 from vehicle device 2, including time information 520 of first image data 501, second image data 502, third image data 503, and fourth image data 504. Server device 1 can perform image parsing on the obtained object image data 50.

[0097] In step S10A, as a determination of the first interval, the server device 1 determines whether a predetermined facility 35 has been detected based on the image analysis results of the first image data 501. The detection of a predetermined facility 35 corresponds to the presence of a predetermined facility 35 in the first image data 501. If the predetermined facility 35 has been detected, the server device 1 proceeds to step S11. Conversely, if the predetermined facility 35 has not been detected, the server device 1 proceeds to step S10B. In step S10B, as a determination of the third interval, the server device 1 determines whether a predetermined display object has been detected based on the image analysis results of the third image data 503. The detection of a predetermined display object corresponds to the presence of a predetermined display object in the third image data 503. If the predetermined display object has been detected, the server device 1 proceeds to step S12. Conversely, if the predetermined display object has not been detected, the server device 1 proceeds to step S13. In step S11, server device 1 selects the first part of article 5511 as the first part of article 551. In step S12, server device 1 selects the first part of article 5512 as the first part of article 551. In step S13, server device 1 selects the first part of article 5513 as the first part of article 551. When the first part of article 551 used in the guiding article 55 is selected through any of steps S11 to S13, server device 1 causes the process to proceed to the next step S20.

[0098] In step S20, as a determination of the fourth interval, server device 1 determines the positional relationship between the predetermined facility 35 and the charging device 39 based on the image analysis results of the fourth image data 504. If the charging device 39 is located near the predetermined facility 35 (first relationship established), server device 1 proceeds to step S21. If the charging device 39 is located next to the predetermined facility 35 (second relationship established), server device 1 proceeds to step S22. If the charging device 39 is located in front of the predetermined facility 35 (third relationship established), server device 1 proceeds to step S23. In step S21, server device 1 selects the second-1st part of the article 5521 as the second part of the article 552. In step S22, server device 1 selects the second-2nd part of the article 5522 as the second part of the article 552. In step S23, server device 1 selects the second-3rd part of the article 5523 as the second part of the article 552. When the second part of the article 552 used in the guiding article 55 is selected through any of steps S21 to S23, the server device 1 causes the processing to proceed to the next step S30.

[0099] Furthermore, in one example of this embodiment, at the point in time when the selection of the first part of the article 551 and the second part of the article 552 is completed, the server device 1 can generate a guide article 55. For example, the server device 1 can generate the guide article 55 by connecting the opening format article 550, the selected first part of the article 551, the selected second part of the article 552, and the closing format article 559. At least a portion of the first part of the article 551 and at least a portion of the second part of the article 552 can be provided by a template. In addition, the opening format article 550 and the closing format article 559 can also be provided by a template. The opening format article 550 can be composed of one or more statements configured at the beginning of the article, such as "The charging device is," etc. The closing format article 559 can be composed of one or more statements configured at the end of the article, such as "Located at...". The server device 1 can appropriately obtain the opening format article 550 and the closing format article 559. As a specific example, if Part 1-1 article 5511 is selected as Part 1 article 551 and Part 2-2 article 5522 is selected as Part 2 article 552, the server device 1 can generate the guiding article 55 "The charging device is moving towards the AAA facility and is located next to the AAA facility." However, the content of each article (550, 551, 552, 559) is not limited to this example and can be appropriately modified according to the implementation method.

[0100] In step S30, server device 1 determines the length of the second interval (the time length of the second image data 502) by referring to time information 520. If the length of the second interval is less than a first threshold, server device 1 proceeds to step S31. If the length of the second interval exceeds the first threshold but is less than the second threshold, server device 1 proceeds to step S32. If the length of the second interval exceeds the second threshold, server device 1 proceeds to step S33. In step S31, server device 1 sets the output timing of the guide article 55 to the time point elapsed since the entry point 30. In step S32, server device 1 sets the output timing of the guide article 55 to the time point exceeding the entry point 30 by D1 m. In step S33, server device 1 sets the output timing of the guide article 55 to the time point exceeding the entry point 30 by D2 m. The output timing setting information can be appropriately saved along with the guide article 55. When the output time is set by any of steps S31 to S33, the server device 1 can end the processing related to the generation of the guide article 55.

[0101] Furthermore, the above-described processing steps for generating the guiding article 55 can be appropriately modified according to the implementation method. For example, the order of the processing steps S10A to S13, steps S20 to S23, and steps S30 to S33 is not limited to... Figure 4 The example can be arbitrarily replaced. The processes can also be executed at least partially in parallel. Steps S10A to S13 can be omitted if Part 1 (Article 551) is not used. Step S13 can be omitted if Parts 1-3 (Article 5513) are not used. Steps S20 to S23 can be omitted if Part 2 (Article 552) is not used. Steps S30 to S33 can be omitted if the output timing is not set. The number of branches in at least one of steps S20 and S30 can be appropriately changed.

[0102] Furthermore, assuming that at least one of the road surface displays (321, 322) and display panel 323 exists at the fork in the road 32, during the image analysis of the third image data 503, the server device 1 can detect only either the road surface displays (321, 322) or the display panel 323 as a display object. Accordingly, in step S10B, the server device 1 can determine whether a display object of either the road surface displays (321, 322) or the display panel 323 has been detected. If a display object of one of the two is detected, in step S12, the server device 1 can select an article that guides the orientation of the charging device 39 based on the detected display object as the first part of the article 551. If no display object of one of the two is detected, in step S13, the server device 1 can select an article that guides the orientation of the charging device 39 based on the other display object as the first part of the article 551.

[0103] (Guide to saving this article)

[0104] The generated guide text 55 can be stored in any storage area. In one example, the generated guide text 55 can be stored in a storage area accessible to the user's terminal using the roadside rest facility 3 after the object vehicle VE. The user's terminal can be, for example, an in-vehicle device, another user terminal, etc. The arbitrary storage area can be, for example, the memory resources of the server device 1, an external computer (external server, etc.), the memory resources of the terminal, etc. In another example, the generated guide text 55 can also be stored in a storage area inaccessible to the user's terminal and then appropriately provided to the user's terminal.

[0105] Furthermore, the generated guide text 55 can be provided to the user in any way. In one example, an external server providing map information may exist. Server device 1 can provide the generated guide text 55 to the external server. The external server can provide the guide text 55 along with the map information based on a request from the user's terminal. In another example, server device 1 can also act as a server device providing map information by maintaining the map information. In this case, server device 1 can provide the guide text 55 along with the map information to each user's terminal.

[0106] (Data format for guiding articles)

[0107] The data format of the generated guide text 55 can be arbitrarily selected. In one example, the guide text 55 can consist of only text data, text data and voice data, or only voice data. The voice data constituting the guide text 55 can be generated appropriately. In one example, during the process of generating the guide text 55, the server device 1 can also directly generate the voice data. In another example, the server device 1 can also indirectly generate voice data by performing speech conversion on the generated text data after it has been generated.

[0108] In the user's terminal utilizing the generated introductory text 55, the introductory text 55 can be played either as is or after any preprocessing is applied. By using the latter, the data format of the introductory text 55 can differ between when it is generated by the server device 1 and when it is played by the terminal. For example, the server device 1 can also generate the introductory text 55 using text data. As preprocessing, the terminal can perform speech conversion on the generated text data. The terminal can also play the speech data obtained through speech conversion as the introductory text 55.

[0109] [2 Structural Examples]

[0110] (Server device)

[0111] Figure 5A This schematically illustrates an example of the hardware structure of the server device 1 according to this embodiment. The server device 1 according to this embodiment is a computer electrically connected to a control unit 11, a storage unit 12, a communication interface 13, and a driver 14.

[0112] The control unit 11 includes a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), etc., and is configured to perform arbitrary information processing based on programs and various data. The control unit 11 (CPU) is an example of a processor resource. The storage unit 12 can be configured as any storage device such as a hard disk drive, solid-state drive, semiconductor memory, etc. The storage unit 12 (and RAM and ROM) is an example of a memory resource. In this embodiment, the storage unit 12 stores various information such as program 81. Program 81 is for causing the server device 1 to perform information processing related to the generation of the aforementioned boot article 55 (described later). Figure 6 The program 81 contains a series of commands for this information processing.

[0113] The communication interface 13 is configured to implement wired or wireless data communication via a network. The communication interface 13 may be configured as, for example, a wired LAN (Local Area Network) module, a wireless LAN module, or the like. In this embodiment, the server device 1 can use the communication interface 13 to perform data communication via a network with other computers (e.g., vehicle-mounted device 2).

[0114] The drive 14 is a device for reading various information, such as programs, stored in the storage medium 91. The program 81 may replace the storage unit 12 or be stored in the storage medium 91 together with the storage unit 12. The storage medium 91 is configured to store various information (programs, etc.) through electrical, magnetic, optical, mechanical, or chemical action so that a computer or other machine can read it. The storage medium 91 is an example of a non-transitory storage medium. The storage medium 91 can be a disc-type storage medium such as a CD or DVD, or a non-disc-type storage medium such as a semiconductor memory (e.g., flash memory). The type of drive 14 can be appropriately selected depending on the type of storage medium 91.

[0115] Furthermore, the specific hardware structure of the server device 1 can be appropriately omitted, replaced, or added depending on the implementation method. For example, the control unit 11 may include multiple hardware processors. The hardware processors may be composed of microprocessors, FPGAs (field-programmable gate arrays), DSPs (digital signal processors), ECUs (electronic control units), GPUs (graphics processing units), ASICs (application-specific integrated circuits), etc. The driver 14 may also be omitted. The program 81 may be stored, for example, in an external storage device such as a NAS. An external storage device is also an example of a non-transitory storage medium. The server device 1 may further include input devices and output devices. In addition to a specially designed computer, the server device 1 may also be a general-purpose server device, a general-purpose PC (personal computer), etc.

[0116] (Vehicle-mounted device)

[0117] Figure 5B This schematically illustrates an example of the hardware structure of the vehicle-mounted device 2 according to this embodiment. The vehicle-mounted device 2 according to this embodiment is a computer electrically connected to a control unit 21, a storage unit 22, a communication module 23, a driver 24, and an external interface 25. The control unit 21 to the driver 24 and the storage medium 92 of the vehicle-mounted device 2 can be configured in the same way as the control unit 11 to the driver 14 and the storage medium 91 of the server device 1 described above.

[0118] The control unit 21 (CPU) is an example of the processor resources of the vehicle-mounted device 2. The storage unit 22 (including RAM and ROM) is an example of the memory resources of the vehicle-mounted device 2. In this embodiment, the storage unit 22 stores various information such as program 82. Program 82 is for causing the vehicle-mounted device 2 to perform information processing related to the generation of the aforementioned guide article 55 (described later). Figure 6The program 82 contains a series of commands for information processing. The program 82 may replace the storage unit 22 or be stored together with the storage unit 22 in the storage medium 92. The external interface 25 is configured to connect to an external device via wired or wireless means. The external interface 25 may be configured as, for example, a USB (Universal Serial Bus) port, a dedicated port, a communication port, etc. When the external interface 25 includes a communication port, the communication standard of the communication port can be arbitrarily selected. In this embodiment, the vehicle-mounted device 2 can be connected to the dashcam DR via the external interface 25.

[0119] Furthermore, the specific hardware structure of the vehicle-mounted device 2 can be appropriately omitted, replaced, or added depending on the implementation method. For example, the control unit 21 may include multiple hardware processors. The driver 24 may also be omitted. The program 82 may also be stored in an external storage device. The vehicle-mounted device 2 may further include input devices and output devices. In addition to a specially designed computer, the vehicle-mounted device 2 may also be a portable terminal (including a smartphone), a tablet terminal, a laptop PC, or other general-purpose PCs.

[0120] [3 Action Examples]

[0121] Figure 6 This is a timing diagram illustrating an example of the processing steps related to the generation of the guide article 55 implemented by the system 100 according to this embodiment. The control unit 11 of the server device 1 executes the commands contained in program 81 via the CPU. The control unit 21 of the vehicle-mounted device 2 also executes the commands contained in program 82 via the CPU. Thus, the server device 1 and the vehicle-mounted device 2 are respectively configured to perform the following... Figure 6 The computer performs the information processing operations. The following processing steps of at least one of the server device 1 and the vehicle-mounted device 2 are an example of an information generation method (information processing method) executed by a computer. However, the following processing steps are merely an example, and each step can be modified as much as possible. Furthermore, regarding the following processing steps, steps can be omitted, substituted, or added appropriately according to the implementation method.

[0122] In step S101, when the target vehicle VE has been charged using the charging device 39 at the target roadside rest facility 3, the control unit 21 extracts the target image data 50 from the image data 40 accumulated by the dashcam DR. In step S102, the control unit 21 sends the extracted target image data 50 to the server device 1. In one example, the vehicle-mounted device 2 may spontaneously execute steps S101 and S102 in response to the use of the overcharging device 39 at the target roadside rest facility 3. In another example, the vehicle-mounted device 2 may determine whether the overcharging device 39 has been used based on a request from the server device 1. If it is determined that the overcharging device 39 has been used, the vehicle-mounted device 2 may also execute steps S101 and S102.

[0123] In step S201, the control unit 11 receives the object image data 50 from the vehicle-mounted device 2. In step S202, the control unit 11 performs image analysis on the object image data 50. In step S203, the control unit 11 generates a guide text 55 for the charging device 39 based on the result of the image analysis. In one example, the control unit 11 can also perform the above... Figure 3 The method is used to generate the guiding article 55. In the case of a rule-based approach, the control unit 11 can also... Figure 4 The processing steps are used to generate the guide article 55. In one example, the control unit 11 can set the output timing of the generated guide article 55. In another example, when using an end-to-end model as the machine learning model, the control unit 11 can feed the object image data 50 to the trained machine learning model and execute the processing steps S202 and S203 simultaneously by performing the computational processing of the trained machine learning model.

[0124] In step S204, the control unit 11 saves the generated guide text 55. The save destination can be appropriately selected according to the implementation method. The save destination can be selected from, for example, RAM, storage unit 12, storage device of an external computer, storage device of the terminal using the guide text 55, etc. In one example, the control unit 11 may also, as a save process or in addition to a save process, deliver the guide text 55 to the terminals of users located around the target road rest facility 3. The control unit 11 may deliver the guide text 55 directly to the terminal or indirectly to the terminal via an external server, etc. The control unit 11 may also, by delivering the generated guide text 55, give instructions to the terminals of users located around the target road rest facility 3 to output the guide text 55. The control unit 11 may also, by delivering the guide text 55 including the output timing setting, give instructions to the terminal to output the guide text 55 at the set output timing. The terminal may have a positioning module and an output device. The terminal can measure its current position using the positioning module. When the measured current position meets the output timing setting, the guide text 55 is output to the output device. When the generated guide text 55 is saved, the system 100 ends the processing steps related to this action example.

[0125] [feature]

[0126] In this embodiment, by utilizing the processing in steps S202 to S203, at least a portion of the guidance information (guidance text 55) for the charging device 39 can be automatically generated by effectively utilizing the object image data 50 captured by the dashcam DR of the object vehicle VE that has visited the roadside rest facility 3. Therefore, a reduction in the cost of generating guidance information can be expected.

[0127] [4 Modified Examples]

[0128] The embodiments of this disclosure have been described in detail above, but the foregoing description is merely illustrative in all respects. The processes and units described in this disclosure can be freely combined and implemented as long as they do not create technical contradictions. Various modifications or variations can be appropriately made to the above embodiments.

[0129] For example, in the above embodiment, server device 1 is an example of an information processing device. However, the manner of information processing device is not limited to such an example. In another example, other computers such as vehicle-mounted device 2 can also operate as an example of information processing. In the above embodiment, at least a portion of the information processing of server device 1 can also be performed by vehicle-mounted device 2. At least a portion of the information processing of vehicle-mounted device 2 can also be performed by server device 1. Either server device 1 or vehicle-mounted device 2 can perform a series of information processing steps from extracting object image data 50 to saving the guide text 55. In this case, extracting object image data 50 is an example of obtaining object image data 50. When vehicle-mounted device 2 performs the generation of guide text 55, vehicle-mounted device 2 can save the generated guide text 55 in any storage area. For example, vehicle-mounted device 2 can also send the generated guide text 55 to an external server that stores information used in guidance such as map information.

[0130] Symbol Explanation

[0131] 100…system; 1…server device; 11…control unit; 12…storage unit; 2…vehicle device; 21…control unit; 22…storage unit; 3…road rest facility; 39…charging equipment; 40…image data; 50…object image data; 55…guide text; VE…object vehicle; DR…driving recorder.

Claims

1. A system comprising a server device and an onboard device mounted on a target vehicle, wherein, The vehicle-mounted device is configured to perform the following processes: When the vehicle is charged using the charging equipment at the roadside rest facility, image data of the vehicle captured during the period from when the vehicle enters the roadside rest facility until it reaches the charging equipment is extracted from the image data stored in the vehicle's dashcam; and The extracted object image data is sent to the server device. Furthermore, the server device is configured to perform the following processing: Receive the object image data from the vehicle-mounted device; Perform image parsing on the received object image data; as well as Based on the results of the image analysis, a guide article is generated for the charging equipment in the roadside rest facilities of the object; as well as The generated guide text is saved in order to provide it to users who utilize the roadside rest facilities of the object.

2. An information generation method, wherein a computer performs the following processing: By using the dashcam of the target vehicle at the roadside rest facility of the target vehicle to overcharge the device, image data of the target vehicle is obtained from the time the vehicle enters the roadside rest facility of the target vehicle until it reaches the charging device. Perform image analysis on the acquired object image data; Based on the results of the image analysis, generate a guide article leading to the charging equipment in the roadside rest facilities of the object; and The generated guide text is saved in order to provide it to users who utilize the roadside rest facilities of the object.

3. The information generation method as described in claim 2, wherein, The acquired object image data includes first image data, which is image data captured in a first interval from near the entrance of the roadside rest facility of the object to passing by the entrance. If a predetermined facility is reflected in the first image data, the generated guide text includes a text that guides the orientation of the charging device based on the predetermined facility.

4. The information generation method as described in claim 3, wherein, The acquired object image data also includes third image data, which is image data captured in a third section, including a road junction containing the object and rest facilities. If the predetermined facility is not reflected in the first image data, but the predetermined display is reflected in the third image data, the generated guide text includes a text that guides the orientation of the charging device based on the predetermined display.

5. The information generation method as described in claim 2, wherein, The acquired object image data includes time information, which represents the length of time during which the second image data was captured in a second interval after the entrance to the roadside rest facility passing the object and before the fork in the road. The process of generating the introductory article includes the following steps: setting the output timing of the introductory article when it is provided to the user, based on the time length represented by the time information.

Images