Vehicle information collection system

Abstract

To provide a vehicle information collection system that can comprehensively verify various travel conditions to confirm travel safety in verifying an update model.SOLUTION: A vehicle information collection system comprises: a collection condition acquiring unit 201 which acquires a condition to verify an update model; an external environment acquiring unit 202 which acquires external travel environment information; a verifying place setting unit 203 which sets a verifying place to be a target based on information from the external environment acquiring unit 202 and the collection condition acquiring unit 201; a notification unit 204 which notifies an operator of a vehicle of the verifying place; a response receiving unit 205 which receives a permission response from the operator wishing to move to the verifying place, or a denial response that does not wish to move; and a guiding control unit 206 which guides the vehicle to the verifying place according to the permission response and information on the verifying place. In verifying an update model, the system is able to comprehensively verify various travel conditions to confirm safety during travel.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a vehicle information collection system that collects various types of vehicle information, and particularly relates to a vehicle information collection system provided with an AI (artificial intelligence) function.

Background Art

[0002] Recently, the development of autonomous driving that allows automobiles to travel autonomously has been progressing. To implement this autonomous driving, it is effective to use an AI function to perform traveling control of the automobile. When a new updated model is developed for the current model that is actually mounted on and operated in an automobile to enable autonomous driving of the automobile, it is necessary to verify and analyze whether the updated model can perform operations as intended.

[0003] In the verification and analysis of this updated model, in order to confirm traveling safety, it is necessary to spend a great deal of time comprehensively collecting traveling data under various traveling conditions using an actual automobile. In particular, to enable autonomous driving of automobiles, image data of traveling environments under various traveling conditions is required. For this reason, while actually driving the automobile, image data is collected by checking the operating state of the automobile, and verification and analysis are performed.

[0004] For example, regarding the recognition result of the surrounding environment, which is necessary for autonomous driving of an automobile, it is necessary to verify and analyze the degree of similarity between recognition information inferred from actually captured image data and actual correct data. Furthermore, for collecting image data used for AI learning and verification, shortening of the collection time for a large amount of image data is achieved by having a plurality of automobiles travel, and having each automobile share the task of collecting image data.

[0005] However, in general, the collection of image data often results in the collection of large amounts of image data of easily photographable scenes, while image data of unusual scenes, such as different weather conditions or different times of day, is not collected. As a result, a wide variety of image data is not comprehensively collected, making sufficient verification and analysis difficult. To resolve this situation, a technique for comprehensively collecting image data is known, for example, the technique described in Patent Document 1 (International Publication No. 2022 / 107595).

[0006] Patent Document 1 describes a system for efficiently retraining a recognition model that includes a collection timing control unit that controls the timing of collecting candidate image data to be used for retraining a recognition model, and an image data collection unit that selects image data from the collected image data candidates based on at least one of the characteristics of the image data candidates and the similarity to stored image data.

[0007] The recognition model is used to recognize predetermined recognition targets around the vehicle, and the image data collection unit selects appropriate image data from a list of image data candidates, including image data obtained by photographing the area around the vehicle with imaging means such as cameras installed in the vehicle. [Prior art documents] [Patent Documents]

[0008] [Patent Document 1] International Publication No. 2022 / 107595 [Overview of the project] [Problems that the invention aims to solve]

[0009] Incidentally, the image data collection method described in Patent Document 1 determines whether or not to collect image data based on a certain external event (for example, the occurrence of construction work, the occurrence of an accident, rainfall conditions, snowfall conditions, etc.), and classifies the image data collected based on this external event according to the output result (confidence level) of the recognition model.

[0010] However, with this image data collection method, image data cannot be collected unless the relevant external event (such as construction work, an accident, rainfall, or snowfall) is encountered by chance. Thus, with the passive image data collection method described in Patent Document 1, the probability of encountering the aforementioned external event during normal driving is low, making it uncertain whether sufficient image data can actually be collected, resulting in poor verification and analysis efficiency. It should be noted that similar problems are expected to arise not only with the image data of the updated model, but also with other update data.

[0011] The objective of the present invention is to provide a vehicle information collection system that can proactively collect update data necessary for comprehensively verifying and analyzing various driving conditions to confirm driving safety when collecting update data for use in updated models, based on predetermined collection conditions. [Means for solving the problem]

[0012] The present invention relates to a vehicle information collection system for collecting update data for verifying and analyzing an updated model related to vehicle operation, characterized in that it comprises: a collection condition acquisition unit that acquires predetermined collection conditions for collecting update data for the updated model; an external environment acquisition unit that acquires external environmental information; a verification location setting unit that sets a target verification location based on the information from the external environment acquisition unit and the collection condition acquisition unit; and a guidance control unit that guides the vehicle to the set verification location. [Effects of the Invention]

[0013] According to the present invention, in collecting update data used for verifying and analyzing updated models, it becomes possible to proactively collect update data necessary to comprehensively verify and analyze various driving conditions for confirming driving safety, based on predetermined collection conditions. [Brief explanation of the drawing]

[0014] [Figure 1] This is a configuration diagram illustrating the structure for explaining the concept of the vehicle information collection system according to the present invention. [Figure 2] This is a control block diagram showing the specific configuration of a vehicle information collection system according to the first embodiment of the present invention. [Figure 3] Figure 2 is a control block diagram showing the specific configuration of the data collection condition acquisition unit. [Figure 4] Figure 2 is a control block diagram showing the specific configuration of the verification location setting unit. [Figure 5] Figure 2 is a control block diagram showing the specific configuration of the notification unit. [Figure 6] Figure 2 is a control block diagram showing the specific configuration of the induction control unit. [Figure 7] Figure 2 is a flowchart illustrating the control flow for executing the functions of the control block shown. [Figure 8] This is an explanatory diagram showing an example of a table for determining the operating method in the control flow (step S04) shown in Figure 7. [Figure 9] This is an explanatory diagram illustrating the map information displayed on the screen of a vehicle's navigation system. [Figure 10] This is a control block diagram showing a specific configuration of a vehicle information collection system according to a second embodiment of the present invention. [Figure 11] Figure 10 is a control block diagram showing the specific configuration of the induction control unit. [Figure 12] Figure 10 is a flowchart illustrating the control flow that executes the functions of the control block shown. [Figure 13]It is a control block diagram showing a specific configuration of a vehicle information collection system according to the third embodiment of the present invention. [Figure 14] It is a control block diagram showing a specific configuration of a vehicle information collection system according to the fourth embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the following embodiments, and various modifications and applications are included within the scope of the technical concept of the present invention. Examples

[0016] FIG. 1 shows a configuration for explaining the concept of a vehicle information collection system according to the present invention. The vehicle information collection system generally includes: a server 10 mounted with a server-side control unit having an AI function unit; a vehicle (an automobile in this example) 11 mounted with a vehicle-side control unit also having an AI function unit; and communication means (not shown) provided respectively in the server 10 and the vehicle 11 so that they can communicate with each other wirelessly.

[0017] Here, the AI function unit of the server 10 is mounted with an update model to be verified, and if necessary, a current model used for controlling the vehicle 11. The AI function unit of the vehicle 11 is mounted with the current model used for actual control and the above-described update model. Note that the update model may be mounted only on the server 10 or only on the vehicle 11, or the update model may be configured by the server 10 and the vehicle 11 together.

[0018] Actual traveling control such as automatic driving of the vehicle is executed using the current model that has already been verified, and the update model is not used. For this reason, the update model of the AI function unit mounted on the vehicle-side control unit does not affect the traveling of the vehicle, so the safety of the vehicle is ensured.

[0019] Furthermore, since the objective of this invention is to comprehensively collect update data for verifying and analyzing the update model, detailed explanations of the update model's configuration, the methods for verifying and analyzing the collected images, and its operation will be omitted, and the focus will be on explaining the method for collecting update data. In this case, the update data is image data used for recognizing predetermined recognition targets (objects, etc.) around the vehicle 11, and is used in a recognition model as an example of an update model.

[0020] The objects to be recognized are other vehicles, pedestrians, traffic signs, and other landmarks required for autonomous driving. Of course, other objects can also be targeted. Furthermore, the recognition processing results of the image data from the current model and the updated model are compared with each other to obtain recognition result verification data. For example, if there is a difference between the current model and the updated model in this recognition result verification data, the image data will be considered for verification and analysis.

[0021] In Figure 1, the server-side control unit of server 10 sends a request to the vehicle-side control unit of vehicle 11 to acquire image data (update data). The request to acquire image data is made according to a predetermined list of collection conditions. The collection conditions in the collection conditions list include, for example, "location to be verified (hereinafter referred to as verification location)", "time period to be verified (hereinafter referred to as verification time period)", "weather conditions to be verified (hereinafter referred to as verification weather conditions)", and "vehicle driving conditions to be verified (hereinafter referred to as verification driving conditions)". Of course, it is also possible to set other collection conditions.

[0022] The term "verification location" may refer to a specific location, or it may refer to a verification area encompassing a specific location. The following explanation will focus on verification areas including specific locations. A specific location is, for example, a merging lane or similar area where a large amount of image data is needed to ensure the safe operation of vehicle 11. Of course, other locations are also acceptable.

[0023] Then, when the vehicle-side control unit of vehicle 11 determines that the conditions are met according to the collection condition list—that the vehicle is traveling near the verification location (e.g., a merging lane) during the verification time period (e.g., around 5 PM in the evening) under verification weather conditions (e.g., rainfall)—it displays a request on the vehicle's display screen or notifies the driver of vehicle 11 via voice message to travel to the verification location (in this case, the verification area). In addition to the above, various other conditions can be set for the items in the collection condition list.

[0024] Then, upon receiving permission from the operator who has acknowledged the request to drive in the verification area, the vehicle-side control unit displays a route guidance for the verification area to the verification location on the navigation system's display screen. The operator drives the vehicle 11 to the verification location either by their own control or by using the current model's automated driving function, following the route guidance on the display screen. Remote control from the server 10 (as shown in the embodiment in Figure 14) can also be performed as needed.

[0025] During this driving process, the vehicle-side control device acquires image data of the surrounding environment of the vehicle 11 using imaging means 12 such as a camera. The image data may be still images or moving images (in the case of moving images, time-based image data is used). When the vehicle arrives at the verification location, the collection of image data is completed. Of course, it is also possible to collect image data of the area surrounding the verification location as needed. This can be decided as appropriate depending on the need for the image data to be collected.

[0026] Furthermore, the image data may include vehicle position data, distance data to feature points of landmarks or other objects starting from the imaging means 12, time data, weather data, etc. It may also include driving condition data necessary for analysis, and may even include other data.

[0027] Examples of guided route image data include, as shown in the figure, image data taken while driving on a straight road in the evening, or image data taken while driving on a curved road during rainfall. These image data are those that match the aforementioned collection conditions, such as the verification location, verification time, and verification weather.

[0028] The image data obtained by the imaging means 12, such as a camera, is edited into a dataset by the AI ​​function unit of the vehicle-side control unit and sent to the server-side control unit of the server 10 for upload. The dataset includes information such as vehicle driving status data, location data, weather data, and recognition result verification data, which are linked to the image data. The recognition result verification data, as described above, indicates that if there is a difference between the recognition processing result of the image data of the current model and the recognition processing result of the image data of the updated model, it will be subject to verification and analysis. This data is sent to the server 10, which is the "verification / analysis means".

[0029] The AI ​​function unit of the server-side control unit of server 10 will then perform detailed verification and analysis of the recognition model using the uploaded dataset. Needless to say, this detailed verification and analysis will be performed by the developers.

[0030] As described above, the recognition result verification data compares the recognition processing results using image data from the current model, assuming the same driving conditions, with the recognition processing results using image data from the updated model, and verifies whether there is a difference between the two or no difference. In other words, if there is no difference in the recognition of vehicles or targets in both image data, the current model and the updated model are considered to have the same capabilities or performance. If there is a difference in the recognition of vehicles or targets in both image data, the current model and the updated model are considered to have different capabilities or performance. Based on these verification results, the server-side control unit of server 10 can perform a detailed analysis of the updated model. Furthermore, the server-side control unit of server 10 integrates datasets uploaded from many vehicles 11, verifies and analyzes the recognition model, and reconstructs the recognition model to improve its completeness.

[0031] In this invention, when the collection conditions in the collection condition list described above are met, the vehicle-side control unit requests the driver to drive towards the verification location. If the request is accepted, it provides route guidance to the verification location. The unit then operates to acquire and collect the image data required along the journey to the verification location. As a result, image data necessary for verification and analysis can be comprehensively and proactively collected, enabling efficient verification and analysis of the recognition model in the updated model.

[0032] Next, a specific embodiment will be described. Figure 2 shows the configuration of a specific control block of an update data collection means that constitutes a vehicle information collection system according to the first embodiment of the present invention. This vehicle information collection system is installed in the on-board control unit of the vehicle 11, but it may also be configured in cooperation with a server 10.

[0033] In Figure 1, the collection condition acquisition unit 201 acquires collection conditions from the collection condition list sent from the server 10. These collection conditions, as mentioned above, include, for example, "verification location," "verification time period," "verification weather conditions," and "verification driving conditions." It is sufficient that appropriate collection conditions are set, and it is not necessary for all collection conditions to be satisfied. In addition, collection conditions may be assigned a priority. For example, if there are multiple scenes that match the collection conditions while the vehicle 11 is driving, the priority is set so that the scene that occurs less frequently is selected as the target for image data collection. In other words, the scene that occurs less often is given priority.

[0034] The external environment acquisition unit 202 acquires external data corresponding to at least the collection conditions described above. For example, this includes the current vehicle position, current time, current weather, current driving direction, and driving conditions. The current vehicle position can be obtained from a GPS sensor or server 10, the current time can be obtained from the vehicle's timing function, GPS signal timestamp, or server 10, the current weather can be obtained from server 10, the current driving direction can be obtained from the navigation device, and the driving conditions can be obtained from the power control device or steering control device. In addition, information corresponding to the collection conditions described above can also be obtained from other sources.

[0035] The collection condition data from the collection condition acquisition unit 201 and the external environment data from the external environment acquisition unit 202 are input to the verification location setting unit 203. The verification location setting unit 203 compares the external environment data, which includes the current vehicle position, current time, current weather, current driving direction and driving conditions, with the collection conditions, which include the "verification location," "verification time period," "verification weather conditions," and "verification driving conditions," and makes a determination as to whether all of these conditions match.

[0036] Here, the current vehicle position and the verification location (location) to be verified do not need to be a perfect match; it is sufficient for the vehicle to be in the verification area described above, and even if vehicle 11 is driving near the verification location, the location conditions are considered to be met. For example, if vehicle 11 is within a radius of 5 km centered on the verification location, the location conditions can be considered to be met. When the verification location setting unit 203 determines that the above-described collection conditions and the external environment data match, the verification location setting unit 203 sends notification data to the notification unit 204 of the vehicle's display device and also sends location data of the verification location (destination) to the guidance control unit 206.

[0037] When notification data is received, the notification unit 204 displays a request for verification cooperation via a pop-up function on the navigation device's display screen. In addition to this display, the request for verification cooperation can also be read aloud. The pop-up function displays a collection permission button and a collection refusal button simultaneously with the collection cooperation request.

[0038] When the operator who has received a request for data collection cooperation selects either the data collection permission button or the data collection rejection button on the navigation system's display screen, the response reception unit 205 accepts this selection. If the data collection rejection button is selected, the system returns to its original state (default state). On the other hand, if the data collection permission button is selected, guidance start data is sent to the guidance control unit 206 to guide the vehicle 11. In this state, the pop-up function is stopped, and the system returns to the state of the navigation system's display screen.

[0039] When the data collection permission button is pressed, the guidance control unit 206 displays map data within a predetermined range, including the vehicle 11's location data and route data, based on the location data of the verification location sent from the verification location setting unit 203. Here, the vehicle's location data is obtained from the GPS sensor 208. The guidance control unit 206 then calculates a guidance route from the links and nodes connecting the vehicle 11's location data and the location data of the verification location, and displays this guidance route overlaid on the map data.

[0040] The guidance control unit 206, when the vehicle 11 is being driven manually by the driver operating the steering wheel, guides the driver to the verification location in the same way as a normal navigation system. When the vehicle 11 is being driven autonomously, it autonomously drives the vehicle 11 to the verification location and guides it to the verification location. This allows the vehicle 11 to travel to the verification location and acquire image data. When the vehicle 11 arrives at the verification location, image data is collected as needed. Furthermore, the guidance control unit 206 notifies the collection condition update unit 207 that the vehicle is traveling to the verification location and instructs it to update the collection conditions.

[0041] The image data is mainly image data captured by camera 208, and this image data is linked to position data from GPS sensor 209, as will be described later. The image data from camera 208 is acquired by the external information acquisition unit 210 as still image data or as moving image data and stored in a predetermined memory area.

[0042] Image data acquired by the external information acquisition unit 210 is sent to the update model-side recognition processing unit 211 and the communication unit 214 which transmits data. In the update model-side recognition processing unit 211, the acquired image data is recognized by the AI ​​function unit, and the output of this recognition processing result is sent to the recognition processing result verification unit 213. Here, the update model-side recognition processing unit 211 is the recognition processing in the update model, for example, recognizing people or objects from image data, and the recognition processing result of image data including people or objects is sent to the recognition processing result verification unit 212.

[0043] At the same time, the current model's recognition processing unit 211 is simultaneously performing recognition processing using the same image data. Therefore, the recognition processing results of the image data, including people and objects, performed by the current model's recognition processing unit 211 are also sent to the recognition processing result verification unit 213.

[0044] The recognition processing result verification unit 213 verifies the recognition processing results of the acquired image data. For example, it verifies the capabilities or performance of the updated model by comparing the recognition processing results of the updated model's image data under the same collection conditions with the recognition processing results of the current model's image data.

[0045] For example, if there is no difference, verification and analysis are unnecessary, and if there is a difference, verification and analysis are necessary. In this way, the recognition processing result verification unit 213 derives data on whether or not to perform verification and analysis of the updated recognition model based on the presence or absence of a difference. The recognition processing result verification unit 213 also receives position data from the GPS sensor 209 and links it to the recognition processing verification result data.

[0046] Furthermore, if there is no substantial difference in the recognition processing results, it is assumed that there is no difference in recognition processing between the updated model and the current model. In other words, the capabilities or performance of the current model and the updated model can be considered the same. In this case, the recognition processing result verification data is accompanied by discrimination data indicating that there is no difference.

[0047] On the other hand, if there is a difference in the recognition processing results, it is considered that there is a difference in recognition processing between the updated model and the current model. In other words, it can be considered that the capabilities or performance of the current model and the updated model are different. In this case, the recognition processing result verification data is accompanied by discrimination data indicating that there is a difference. Therefore, in this case, the developer will use server 10 to perform verification and analysis work on the updated recognition model based on the image data at that time.

[0048] Image data from the external information acquisition unit 210, recognition processing result data from the updated model-side recognition processing unit 211, and recognition processing result verification data from the recognition processing result verification unit 213 are sent to the communication unit 214 and transmitted to the server 10 as a dataset via wireless communication. On the server 10, the AI ​​function unit of the server-side control unit performs verification and analysis of the recognition model based on the received image data, recognition processing result data, and recognition processing result verification data, and then reconstructs the recognition model.

[0049] Furthermore, since the image recognition processing of the updated model's recognition processing unit 211 and the specific methods of verification and analysis on the server 10 are well-known technologies and have little direct relation to the present invention, a detailed explanation will be omitted.

[0050] Furthermore, the recognition processing result verification data from the recognition processing result verification unit 213 is sent to the collection condition update unit 207. If the verification is sufficient under the current collection conditions described above in the collection condition list, the verification is considered complete, and the current collection conditions are removed from the next collection condition list. If the recognition processing result verification unit 213 fails to verify the authentication result of the current recognition model and the recognition result of the updated recognition model, this failure result is also sent to the collection condition update unit, where it is updated and reflected as failure information.

[0051] Furthermore, this updated collection condition list is sent back to the collection condition acquisition unit 201, and when new verification locations, time periods, weather conditions, etc. that match the collection conditions appear, the above operation is repeated. In this way, the system actively searches for scenes that should be verified, which improves the efficiency of verifying and analyzing the updated recognition model.

[0052] Next, we will briefly explain the main control blocks among the control blocks described above. Figure 3 shows the configuration of the collection condition acquisition unit 201, Figure 4 shows the configuration of the verification location setting unit 203, Figure 5 shows the configuration of the notification unit 204, and Figure 6 shows the configuration of the induction control unit 206.

[0053] In Figure 3, the collection condition acquisition unit 201 consists of an external collection condition acquisition unit 301 and a collection condition storage unit 302. The external collection condition acquisition unit 301 has the function of receiving a collection condition list from the server 10 and receives the collection conditions, such as "verification location," "verification time period," "verification weather conditions," and "verification driving conditions," as described above.

[0054] The received collection condition list is then stored in the collection condition storage unit 302. The collection condition storage unit 302 consists of a writable / readable memory such as RAM or flash ROM. The collection condition storage unit 302 stores a list of collection conditions, including at least "verification location," "verification time period," "verification weather conditions," and "driving status of the verification vehicle," with collection priority assigned to each condition.

[0055] Furthermore, the collection condition storage unit 302 receives the updated collection condition list from the collection condition update unit 207, in which case the collection condition list in memory is overwritten. The collection conditions stored in the collection condition storage unit 302 are then sent to the next verification location setting unit 203.

[0056] In Figure 4, the verification location setting unit 203 consists of a driving condition classification unit 401, a destination setting unit 402, and a collection condition setting unit 403. The collection conditions from the collection condition acquisition unit 201 are input to the driving condition classification unit 401, the destination setting unit 402, and the collection condition setting unit 403. In addition, external environmental information is input to the collection condition setting unit 403 from the external environment acquisition unit 102. The environmental information includes, for example, the current time, weather conditions, the vehicle's direction of travel, and its driving status, all of which correspond to the collection conditions. The current location can be obtained from a navigation device or the like.

[0057] The driving condition classification unit 401 classifies whether the vehicle will travel to the verification location under the driver's control, under autonomous driving conditions, or under commands from the server 10. The method of this classification will be described later.

[0058] Furthermore, the destination setting unit 402 sets the vehicle 11 as a destination to guide it to the verification location when the vehicle 11 is driving near the verification location to collect image data. For this purpose, it monitors the relationship between the current vehicle position and the location of the verification location, and when the vehicle approaches the vicinity of the verification location, it provisionally sets the verification location as the destination. The guidance route between the vehicle and the destination is the same as the well-known route search performed by navigation devices. When the collection consent button is selected, the provisional setting becomes a final setting and the destination is set.

[0059] Furthermore, the collection condition setting unit 403 acquires environmental information corresponding to the collection conditions acquired from the collection condition acquisition unit 101. When the collection conditions and environmental information match, it notifies the notification unit 104 that the collection conditions and environmental information match, prompting the operator to decide whether to permit or refuse collection. Similarly, it functions to send the driving conditions set by the driving condition classification unit 401 and the destination set by the destination setting unit 402 to the guidance control unit 106.

[0060] In Figure 5, the notification unit 204 consists of a notification content selection unit 501 and a notification content display unit 502. When the vehicle approaches the vicinity of the verification location, the verification location setting unit 203 selects a message to convey to the driver. This message selection involves selecting text data such as "Request for cooperation in data collection," and icon data such as a "Collection permission button" and a "Collection refusal button." Once a message to the driver is selected, this message data is sent to the notification content display unit 502.

[0061] Furthermore, to assist the operator in deciding whether to permit or refuse data collection, the system can filter and display information necessary for the operator to make this decision. This information is displayed on the screen as needed. Examples include the location of the verification site and the time required to travel to the verification site. This selected information is sent to the notification content display unit 502. The location of the verification site and the time required to travel to the verification site are important pieces of information for deciding whether to permit or refuse the collection of image data. For example, if the travel time is considerably long, the operator may decide to refuse the collection of image data.

[0062] The notification content display unit 502 displays the aforementioned messages and decision-making support information, for example, on the display screen of a navigation device. This display is achieved, for example, by providing a pop-up function to the navigation device and displaying the aforementioned messages and decision-making support information using the pop-up function.

[0063] Here, the navigation device's display screen has a function display area for icon data such as a data collection permission button and a data collection rejection button. The notification content display unit 502 functions to send the input status of whether data collection is permitted or not for the data collection permission button and data collection rejection button to the response reception unit 105. The data collection permission information received by the response reception unit 205 is sent to the guidance control unit 206.

[0064] The guidance control unit 206 consists of a pilot operation reception unit 601, a rejection operation reception unit 602, a cancellation decision unit 603, a vehicle guidance operation unit 604, and a collection state switching unit 605.

[0065] The operator input reception unit 601 has a function to receive input from the operator to stop the image data collection process if the operator wishes to stop the process midway through the image data collection process. In addition, the operator input reception unit 601 has a function to determine that an override state is occurring when an operation signal such as an accelerator pedal or brake pedal is input, or when steering wheel operation is performed, during autonomous driving, and to receive input to stop the image data collection process. Furthermore, during manual driving, if the vehicle deviates from a pre-displayed guidance path, the operator input to stop the image data collection process is determined to be an override state.

[0066] In this embodiment, the operator operation reception unit 601 is operated by input from the collection operation stop button displayed on the navigation device's display screen. The rejection operation reception unit 602 receives input from the notification content display unit 502, which is input via the response reception unit 105, to reject the collection of image data.

[0067] The input signals for canceling the data collection operation and the input signals for refusing data collection are sent to the cancellation decision unit 603. When the cancellation decision unit 603 determines that either input signal is present, it inputs a data collection cancellation signal to the data collection state switching unit 605, which will not perform data collection. The data collection state switching unit 605 receives either the data collection permission signal from the response reception unit 105 or the data collection cancellation signal from the cancellation decision unit 603, and then executes a function to set whether to automatically drive the vehicle to the verification location, provide guidance to the driver, or return to the original driving route. The setting status from this setting function is sent to the vehicle guidance operation unit 604 and the data collection condition update unit 207.

[0068] When the vehicle guidance control unit 604 is set to guide the vehicle to the verification location, it performs various control operations related to guidance. For example, when the vehicle is to head to the verification location by the operator's command, the navigation device displays a map on its display screen that includes the vehicle's position and the verification location, overlays the guidance route on this map, and provides route guidance using voice or other means.

[0069] On the other hand, when heading to a test location using autonomous driving, the navigation system's display screen shows a map including the vehicle's position and the test location. The guidance route is then overlaid on this map to provide route guidance, while the steering system and powertrain (internal combustion engine or electric motor) are operated to automatically drive the vehicle to the test location.

[0070] In this embodiment of the vehicle information collection system, when the collection conditions in the collection condition list are met, the vehicle-side control unit requests the driver to drive towards the verification location. Once the request is accepted, it guides the vehicle along the route to the verification location. The system then operates to acquire and collect the necessary image data until the vehicle arrives at the verification location. As a result, it is possible to comprehensively and proactively collect image data necessary for verification and analysis, enabling efficient verification of the recognition model.

[0071] Although the vehicle information collection system shown in Figure 2 is represented by functional blocks, its actual operation is performed by control software on a computer mounted on the vehicle-side control device of vehicle 11. The operations performed by this control software are explained below using the flowchart shown in Figure 7.

[0072] ≪Step S01≫ In step S01, the server-side control unit of server 10 obtains collection conditions according to a predetermined collection condition list. The collection conditions in the collection condition list include, for example, "verification location," "verification time period," "verification weather conditions," and "verification vehicle driving status." Once the collection conditions are obtained, the process proceeds to step S02.

[0073] ≪Step S02≫ In step S02, current external environmental information corresponding to the collection conditions in the collection conditions list is acquired. For example, the vehicle position, verification time period, verification weather conditions, and verification vehicle driving status in the verification area corresponding to the verification location shown in step S01 are acquired. Once the environmental information is acquired, the process proceeds to step S03. ≪Step S03≫ In step S03, the collection conditions obtained in step S01 are compared with the environmental information obtained in step S02 to determine the matching status and extract the corresponding collection conditions. In other words, it is determined whether a matching state has occurred between the collection conditions obtained in step S01 and the environmental information obtained in step S02, and if a matching state is determined, it is determined that collection can be performed.

[0074] For example, if the following conditions are met, such as vehicle 11 is driving near a verification location which is a collection condition, the current time is within the verification time period which is a collection condition, the current weather is a rainy state which is a collection condition, and the current direction of travel is towards a predetermined landmark which is a collection condition, then it is determined that image data can be collected and extracted as a collection condition. Once the collection conditions for collecting image data are extracted, the process proceeds to step S04.

[0075] ≪Step S04≫ In step S04, the contents of the collection conditions extracted in step S03 are determined, and the vehicle driving method formula is determined. Here, vehicle 11 is (1) Pattern 1: The current model is unable to perform image data recognition processing, but the updated model is able to perform image data recognition processing. (2) Pattern 2, in which the current model can perform image data recognition processing and the updated model can similarly perform image data recognition processing, (3) Pattern 3, in which the current model can perform image data recognition processing, but the updated model cannot perform image data recognition processing, (4) There are vehicles in Pattern 4 where the current model is unable to perform image data recognition processing, and similarly, the updated model is also unable to perform image data recognition processing.

[0076] The driving method is then determined based on the recognition status of these models. Figure 8 shows a table that determines the driving method corresponding to the recognition status of these models.

[0077] In Figure 8, Pattern 1 represents a vehicle where the current model cannot perform recognition processing, and only the updated model can perform image data recognition. In this case, vehicle 11 is guided to the verification location, but since the current model cannot perform recognition processing, either a manual driving mode by the operator or a remote driving mode instructed by the server 10 is selected. Which driving mode is selected is determined according to predetermined conditions.

[0078] In this case, it is not possible to compare and verify the recognition results of the current model and the updated model, so the analysis will be based on the recognition results of the updated model. However, even in this case, the image data and recognition results of the required scenes can be obtained, so although a comparison and verification of the recognition results is not possible, the image data and recognition results are still actively collected, which is advantageous.

[0079] Pattern 2 involves vehicles where both the current model and the updated model are capable of performing image data recognition processing. In this case, vehicle 11 is guided to the verification location, and since the current model is capable of performing image data recognition processing, either a manual driving mode by the operator or an automated driving mode by the current model is selected. Which driving mode is selected is determined according to predetermined conditions.

[0080] Pattern 3 is a vehicle in which the current model can perform recognition processing, but the updated model cannot perform image data recognition. In this case, since the updated model cannot perform image data recognition processing, it is excluded from image data collection. Therefore, vehicle 11 will not be guided to the verification location.

[0081] Pattern 4 is a vehicle in which both the current model and the updated model are unable to perform image data recognition processing. In this case, since the updated model cannot perform image data recognition processing, it is excluded from image data collection. Therefore, vehicle 11 will not be guided to the verification location.

[0082] Once the collection conditions are determined and the vehicle driving method is also determined, the process proceeds to step S05.

[0083] ≪Step S05≫ In step S05, the notification content (message) to be sent to the operator is selected. This selection involves choosing text data such as "Request for cooperation in data collection," and icon data such as a "Collection permission button" and a "Collection refusal button." In addition, to assist the operator in deciding whether to grant or refuse permission for image data collection, the operator selects (filters) information necessary for making that decision. Examples include the location of the verification site and the time required to travel to the verification site. Once the notification content (message) to the operator is selected, the process proceeds to step S06.

[0084] ≪Step S06≫ In step S06, the notification content selected in step S05 is communicated to the operator. For example, the notification content is displayed on the navigation system's display screen. This display can be achieved, for example, by providing the navigation system with a pop-up function, which then displays the notification content. Furthermore, a request for cooperation in data collection can also be communicated via voice. Once the operator is notified, the process proceeds to step S07.

[0085] ≪Step S07≫ In step S07, it is determined whether the operator selected the "Allow Data Collection" button or the "Deny Data Collection" button displayed on the navigation system's display screen in step S06. The operator can also decide whether or not to cooperate with the collection of image data based on the verification location and the estimated travel time to the verification location displayed on the screen. If it is determined that the "Deny Data Collection" button was selected (No decision), the process proceeds to step S08; if it is determined that the "Allow Data Collection" button was selected (Yes decision), the process proceeds to step S10.

[0086] ≪Step S08≫ In step S08, since the operator has refused to collect image data for the recognition model to be updated, the execution of the image data collection operation is canceled. This allows the operator to continue driving normally. In addition, the notification content displayed on the navigation system's display screen is cleared. Once the collection operation is canceled, the process proceeds to step S09.

[0087] ≪Step S09≫ In step S09, the collection conditions list is updated. However, since the collection operation has been canceled by the operator, the collection conditions list is updated to indicate that no image data was collected under the current collection conditions and that these collection conditions will not be presented again. This completes the current control flow.

[0088] ≪Step S10≫ In step S10, since it is determined that the data collection permission button was selected in step S07, the data collection process for the current recognition model and the recognition model to be updated is performed. In order to collect the image data of the recognition model, it is necessary to guide the vehicle to the verification location. In this case, the vehicle 11 is guided to the verification location using the driving method determined in step S04.

[0089] The vehicle-side control unit of vehicle 11 displays route guidance to the verification location on the navigation device's display screen. The operator drives vehicle 11 to the verification location by following the route guidance on the display screen, either by operating the vehicle themselves, by remote control from server 10, or by using the current model's automated driving function. Until the vehicle arrives at the verification location, the vehicle-side control unit acquires image data of the surrounding environment of vehicle 11 using imaging means 12 such as a camera. The image data may be still images or moving images. In addition to a camera, point cloud data from a laser sensor or the like can also be used as an imaging means.

[0090] Furthermore, the acquired image data of both the current model and the updated model undergoes recognition processing, and the results of this recognition processing are compared and verified. For example, the difference in the recognition processing results of the current model image data and the updated model image data under the same collection conditions is calculated, and data is derived from the presence or absence of this difference to determine whether or not verification and analysis of the updated recognition model is necessary. This verification process is carried out until the image data collection process is completed in step S12, but the control step in step S11 may be executed along the way.

[0091] ≪Step S11≫ In step S11, a decision is made as to whether or not to stop the image data collection operation. For example, the operator may decide to stop the collection operation for some reason. In this case, the operator selects the "Stop Collection Operation Button" displayed on the navigation system's display screen. If the Stop Collection Operation Button is selected (Yes decision), the system returns to step S08 and cancels the execution of the collection operation. As a result, the operator returns to the original route and resumes normal driving.

[0092] Furthermore, in step S09, the collection conditions list is updated. However, in this case, since the collection operation has been interrupted by the operator, the collection conditions list is updated to indicate that the current collection conditions have not been collected and that these collection conditions will not be presented again. This terminates the current control flow.

[0093] ≪Step S12≫ On the other hand, if the "Stop Collection" button is not selected in step S11 (No judgment), the collection process in step S10 continues, and once all collection work is completed, the process moves to step S13.

[0094] ≪Step S13≫ In step S13, the collection conditions list is updated. If sufficient data has been collected under the current collection conditions listed in the collection conditions list, the collection is considered complete, and the current collection conditions are removed from the next collection conditions list. This updated collection conditions list will repeat the above process whenever new verification locations, time periods, weather conditions, etc. that match the collection conditions appear.

[0095] Here, Figure 9 shows the screen displayed on the navigation device's display screen during image data collection. The navigation device's display screen 901 is provided with a map display area 902. The map display area 902 displays the vehicle's position 903 and the destination, the verification location 904, as well as the guidance route 905 from the vehicle's position 903 to the verification location 904.

[0096] Therefore, in manual operation, the operator steers the vehicle 11 along the displayed guidance path 905, and in automated operation, the vehicle 11 is automatically driven and guided by the current model of the vehicle-side control device. Image data is collected during the process of traveling along this guidance path 905.

[0097] Outside the map display area 902 of the display screen 901, there is a text data display unit 906 that displays a request for cooperation in collecting image data, a driving method display unit 907 that indicates whether manual or automatic driving is being performed, a collection permission button 908 that allows the collection of image data, a collection rejection button 909 that rejects the collection, a collection work stop button 910 that stops the collection work midway, and a time display unit 911 that displays the approximate time required for the collection work.

[0098] In this embodiment of the vehicle information collection system, when the collection conditions in the collection condition list are met, the vehicle-side control unit requests the driver to drive towards the verification location. Once the request is accepted, it provides route guidance to the verification location. During the drive to the verification location, it then operates to acquire and collect the necessary image data. As a result, it is possible to comprehensively and proactively collect the image data necessary for verification, enabling efficient verification of the recognition model. [Examples]

[0099] Next, a second embodiment of the present invention will be described. The second embodiment is characterized by collecting image data in cooperation with other vehicles. For example, it is characterized by communicating with other vehicles by an inter-vehicle information communication unit to obtain at least the driving speed and driving direction of other vehicles, and by controlling the driving state of the vehicle by an induction control unit to create the necessary scenery according to the driving state of other vehicles. The following will be explained with reference to Figures 10 to 12.

[0100] Figure 10 shows the overall control block that demonstrates coordination with other vehicles, based on the control block shown in Figure 2. Figure 11 shows the detailed control block of the induction control unit. Figure 12 shows the flowchart of the control flow in Figure 10. Since Figure 10 has almost the same configuration as the control block shown in Figure 2, except for the control block that coordinates with other vehicles, the explanation of the same control block will be omitted.

[0101] In Figures 10 and 11, a vehicle position acquisition unit 215 and an inter-vehicle information communication unit 216 are newly provided, and this information is input to the guidance control unit 217. The vehicle position acquisition unit 215 acquires the current position of the vehicle, which can be obtained by detecting the vehicle position and direction of travel used by the navigation system.

[0102] Furthermore, the inter-vehicle information communication unit 216 communicates with other vehicles to obtain their position and direction of travel, and exchanges control data with other vehicles. It can communicate directly with other vehicles or via the server 10.

[0103] As shown in Figure 11, the guidance control unit 217 is equipped with a vehicle-to-vehicle information decoding unit 606 that receives information from the vehicle position acquisition unit 215 and the vehicle-to-vehicle communication unit 216. The control data decoded and generated by the vehicle-to-vehicle information decoding unit 606 is input to the vehicle guidance operation unit 604.

[0104] The inter-vehicle information decoding unit 606 decodes the inter-vehicle information and the vehicle's own position information and inputs control data to the vehicle guidance control unit 604 that allows the driving state of the vehicle and other vehicles to create the necessary scene. Based on this control data, the vehicle guidance operation unit 604 drives the vehicle and, as in the first embodiment, acquires image data of the driving state using the camera 208.

[0105] For example, as mentioned above, when acquiring image data from one's own vehicle, it may be necessary to have image data where static targets are present along with other moving targets (e.g., other vehicles in motion). In this case, it is effective to drive in a way that creates the desired scene for other vehicles in order to produce the required scene.

[0106] Therefore, in this embodiment, the vehicle and the other vehicle exchange information regarding their respective positions, directions of travel, and speeds via communication, enabling each vehicle to create the aforementioned scenario. For example, in a specific example, to create a scenario where the vehicle and the other vehicle are traveling toward a merging lane, and the other vehicle has priority at the merging point, the inter-vehicle information decoding unit 606 decodes the respective positions, directions of travel, and speeds of the other vehicle, and manages the vehicle's driving state so that the camera 208 can capture the situation of the other vehicle passing the merging point. This makes it possible to create the necessary scenario and acquire image data.

[0107] Although the vehicle information collection system shown in Figure 10 is represented by functional blocks, its actual operation is performed by control software on a computer mounted on the vehicle-side control device of vehicle 11. The operations performed by this control software will be explained below using the flowchart shown in Figure 11.

[0108] ≪Step S21≫ In step S21, the server-side control unit of server 10 obtains collection conditions according to a predetermined collection condition list. The collection conditions in the collection condition list include, for example, "verification location," "verification time period," "verification weather conditions," and "verification vehicle driving status." Once the collection conditions are obtained, the process proceeds to step S22.

[0109] ≪Step S22≫ In step S22, current external environmental information corresponding to the collection conditions in the collection conditions list is acquired. For example, the vehicle position, verification time period, verification weather conditions, and verification vehicle driving status in the verification area corresponding to the verification location shown in step S21 are acquired. Once the environmental information is acquired, the process proceeds to step S23. ≪Step S23≫ In step S23, the collection conditions obtained in step S21 are compared with the environmental information obtained in step S22 to determine the matching status and extract the corresponding collection conditions. In other words, it is determined whether a matching state has occurred between the collection conditions obtained in step S21 and the environmental information obtained in step S22, and if a matching state is determined, it is determined that collection can be performed.

[0110] For example, if the following conditions are met, such as vehicle 11 is driving near a verification location which is a collection condition, the current time is within the verification time period which is a collection condition, the current weather is a rainy state which is a collection condition, and the current direction of travel is towards a predetermined landmark which is a collection condition, then it is determined that collection is possible and the conditions are extracted as collection conditions. Once the collection conditions for collecting image data are extracted, the process proceeds to step S24.

[0111] ≪Step S24≫ In step S24, the contents of the collection conditions extracted in step S03 are determined, and the vehicle driving method formula is determined. Here, vehicle 11 is (1) Pattern 1: The current model is unable to perform image data recognition processing, but the updated model is able to perform image data recognition processing. (2) Pattern 2, in which the current model can perform image data recognition processing and the updated model can similarly perform image data recognition processing, (3) Pattern 3, in which the current model can perform image data recognition processing, but the updated model cannot perform image data recognition processing, (4) There are vehicles in Pattern 4 where the current model is unable to perform image data recognition processing, and similarly, the updated model is also unable to perform image data recognition processing.

[0112] Then, the driving method is determined based on the recognition status of these models. The determination of these driving methods is the same as in the first embodiment. Once the contents of the collection conditions are determined and the vehicle driving method formula is determined, the process proceeds to step S25.

[0113] ≪Step S25≫ In step S25, the notification content (message) to be sent to the operator is selected. This selection involves selecting text data such as "Request for cooperation in data collection," and icon data such as a "Collection permission button" and a "Collection refusal button." In addition, to assist the operator in deciding whether to grant or refuse permission for image data collection, the operator selects (filters) information necessary for making that decision. Examples include the location of the verification site and the time required to travel to the verification site. Once the notification content (message) to the operator is selected, the process proceeds to step S26.

[0114] ≪Step S26≫ In step S26, communication with another vehicle is initiated via a communication means. In this case, the other vehicle is another vehicle located in the verification area, including the verification location, and the vehicle and the other vehicle will be coordinated through this communication. Once communication with the other vehicle is initiated, the process proceeds to step S27.

[0115] ≪Step S27≫ In step S27, the notification content selected in step S25 is communicated to all drivers of other vehicles. For example, the notification content is displayed on the display screen of the vehicle's navigation system. This display is achieved by providing the navigation system with a pop-up function, which displays the notification content. Furthermore, a request for cooperation in data collection can also be announced by voice. Once all drivers have been notified, the process proceeds to step S28.

[0116] ≪Step S28≫ In step S28, it is determined whether the operator selected the "Allow Data Collection" button or the "Deny Data Collection" button displayed on the navigation system's screen. The operator is any of the operators of other vehicles that were communicated with the system in step S26. At this point, the system can also decide whether or not to cooperate with the collection of image data based on the verification location and the estimated travel time to the verification location displayed on the screen.

[0117] Then, if it is determined that even one of the pilots has selected the button to refuse image data collection (No decision), the system proceeds to step S29. If it is determined that all pilots have selected the button to allow image data collection (Yes decision), the system proceeds to step S31.

[0118] ≪Step S29≫ In step S29, since the operator has refused to collect image data for the recognition model to be updated, the image data collection operation is canceled. As a result, the operator returns to the original route and resumes normal driving. The notification displayed on the navigation system's display screen is also cleared. Once the collection operation is canceled, the process proceeds to step S30.

[0119] ≪Step S30≫ In step S30, the collection conditions list is updated. However, since the collection operation has been canceled by the operator, the collection conditions list is updated to indicate that no image data was collected under the current collection conditions and that these collection conditions will not be presented again. This completes the current control flow.

[0120] ≪Step S31≫ In step S31, since it is determined that the data collection permission button was selected by all operators in step S28, the collection of image data for the current recognition model and the recognition model to be updated is performed. In order to collect image data for the recognition model, it is necessary to guide the vehicle to the verification location. In this case, the vehicle 11 is guided to the verification location using the driving method determined in step S24.

[0121] As mentioned above, since the vehicle itself and other vehicles are interconnected, their movement is managed to create the necessary scenery in order to acquire the required image data. In this embodiment, the vehicle itself and other vehicles exchange information such as their respective positions, directions of travel, and speeds via communication, enabling each vehicle to create the aforementioned scenery.

[0122] For example, in a specific scenario where the system simulates a scene where both the system and another vehicle are traveling toward a merging lane, and the other vehicle has priority at the merging point, the inter-vehicle information decoding unit 606 decodes the relative positions, directions of travel, and speeds of each vehicle, and manages the system's own vehicle's driving state so that the camera 208 can capture images of the other vehicle passing the merging point. This makes it possible to create the desired scene and acquire image data.

[0123] Furthermore, while the vehicle's driving status is being managed, the vehicle-side control unit of the vehicle 11 displays route guidance to the verification location on the navigation device's display screen. The driver drives the vehicle 11 to the verification location, either by driving it themselves, by remote control from the server 10, or by using the current model's automated driving function, following the route guidance on the display screen. Until the vehicle arrives at the verification location, the vehicle-side control unit acquires image data of the surrounding environment of the vehicle 11 using imaging means 12 such as a camera.

[0124] Furthermore, the acquired image data of both the current model and the updated model undergoes recognition processing, and the results of this recognition processing are compared to verify the recognition results. For example, the difference in the recognition processing results of the current model image data and the updated model image data under the same collection conditions is calculated, and data is derived from the presence or absence of this difference to determine whether or not verification and analysis of the updated recognition model is necessary. This verification process is carried out until the image data collection process is completed in step S35, but control steps S32 and S33 may be executed along the way.

[0125] ≪Step S32≫ In step S32, a decision is made as to whether or not to stop the image data collection operation. For example, the operator may decide to stop the collection operation for some reason. In this case, the operator selects the "Stop Collection Operation Button" displayed on the navigation system's display screen. If the Stop Collection Operation Button is selected (Yes decision), the system returns to step S29 and cancels the execution of the collection operation. As a result, the operator returns to the original route and resumes normal driving.

[0126] Furthermore, in step S30, the collection conditions list is updated. However, in this case, since the collection operation has been interrupted by the operator, the collection conditions list is updated to indicate that the current collection conditions have not been collected and that these collection conditions will not be presented again. This terminates the current control flow.

[0127] ≪Step S33≫ On the other hand, if the collection stop button is not selected in step S32 (No determination), the process proceeds to step S33, where it is determined whether the collection stop button has been selected by the operator of another vehicle (at least one person). If it is determined that the collection stop button has been selected (Yes determination), the process proceeds to step S34; if it is determined that the collection stop button has not been selected (No determination), the process proceeds to step S35.

[0128] ≪Step S34≫ On the other hand, if the "Stop Collection" button is selected in step S33, step S34 communicates with other vehicles to stop the collection of image data. Simultaneously, communication with other vehicles is terminated, and the process returns to step S29, canceling the execution of the collection operation. Steps S29 and S30 are then executed.

[0129] ≪Step S35≫ On the other hand, if the "Stop Collection" button is not selected in step S33 (No judgment), the collection work in step S31 continues, and as soon as all collection work is completed, communication with other vehicles is terminated and the system proceeds to step S36.

[0130] ≪Step S36≫ In step S36, the collection conditions list is updated. If sufficient data has been collected under the current collection conditions listed in the collection conditions list, the collection is considered complete, and the current collection conditions are removed from the next collection conditions list. This updated collection conditions list will repeat the above process whenever new verification locations, time periods, weather conditions, etc. that match the collection conditions appear.

[0131] Thus, according to this embodiment, in order to acquire the necessary image data, the vehicle itself and other vehicles are managed in cooperation with other vehicles to create the necessary scenes, thereby enabling more efficient collection of image data. [Examples]

[0132] Next, a third embodiment of the present invention will be described. The third embodiment is characterized by acquiring vehicle operation information linked to the image data along with the acquisition of image data. This will be explained below with reference to Figure 13.

[0133] Figure 13 shows the overall control block that acquires vehicle operation information based on the control block shown in Figure 2. Note that Figure 13 has almost the same configuration as the control block shown in Figure 2, except for the control block that acquires vehicle operation information; therefore, the explanation of the identical control block is omitted.

[0134] In Figure 13, the vehicle 11 is equipped with various on-board equipment 220, and detection signals from these on-board equipment 220 are input to the vehicle information acquisition unit 221. Examples of on-board equipment 220 include equipment installed in the power unit, equipment installed in the steering unit, equipment installed in the braking unit, and so on.

[0135] For example, in the powertrain, parameters such as accelerator pedal depression, engine speed, intake air volume, vehicle speed, and acceleration are detected. Similarly, in the steering system, parameters such as steering wheel angle and wheel angle are detected. In the braking system, parameters such as brake pedal depression, skid ratio, and vehicle deceleration are detected.

[0136] The output of the in-vehicle equipment 220 is input to the vehicle information acquisition unit 221 and converted into vehicle information detection data necessary for control. The vehicle information detection data obtained by the vehicle information acquisition unit 221 is input to the vehicle operation processing unit 222 and the communication unit 214.

[0137] The vehicle operation processing unit 222 obtains control processing data to specifically control the power unit, steering unit, braking unit, etc., based on the vehicle information detection data, and operates the power unit, steering unit, braking unit, etc., using this data. The control processing data from the vehicle operation processing unit 222 is input to the vehicle processing result collection unit 223 and input as vehicle processing result data to the communication unit 214 and the collection condition update unit 207.

[0138] Here, the vehicle processing result data is linked to the image data mentioned above, and the vehicle processing result data corresponds to changes in the image data, making it easier to clarify the causal relationship between them. This linked information can then be used for analysis work on server 10. [Examples]

[0139] Next, a fourth embodiment of the present invention will be described. The fourth embodiment relates to remote control of Pattern 1 (see Figure 8), which was described in the vehicle driving method of the first embodiment. As described in the first embodiment, automatic driving cannot be performed if the current recognition model cannot recognize image data. For this reason, a vehicle remote control unit has been added to enable automatic driving. The following explanation will be given with reference to Figure 14.

[0140] Figure 14 shows the overall control block for remotely controlling the vehicle, based on the control block shown in Figure 2. Note that Figure 14 has almost the same configuration as the control block shown in Figure 2, except for the control block for remote vehicle operation; therefore, the explanation of the identical control block is omitted.

[0141] In Figure 14, for example, the vehicle remote control unit 224, located on the server 10, receives information for remote control from the driving condition classification unit 401 (see Figure 4) of the verification location setting unit 203 via the communication unit 214. In addition, information that the collection permission button has been selected is received from the response receiving unit 205 via the communication unit 214. Therefore, the vehicle remote control unit 224 instructs the guidance control unit 206 via the communication unit 214 to guide the vehicle toward the verification location. Here, the vehicle remote control unit 224 is equipped with the current model and has the same functions as the current model installed on the vehicle 11.

[0142] In Pattern 1, the current recognition model installed in the vehicle 11 is not functioning, so the image data from the external information acquisition unit 210 is input to the vehicle remote control unit 224 via the communication unit 214. Therefore, the vehicle remote control unit 224 can perform the same operations as the current recognition model installed in the vehicle 11.

[0143] Therefore, the control processing data from the vehicle remote control unit 224 is provided to the guidance control unit 206 via the communication unit 214, enabling automatic driving. Of course, it goes without saying that the image data at this time is acquired in the same way as in the first embodiment. Thus, according to this embodiment, even if the current recognition model of the vehicle 11 is not functioning, the vehicle remote control unit 224 on the server 10 side can perform automatic driving.

[0144] As described above, the present invention relates to a vehicle information collection system for collecting update data for verifying and analyzing an updated model related to vehicle operation, and is characterized by comprising: a collection condition acquisition unit that acquires predetermined collection conditions for collecting update data for the updated model; an external environment acquisition unit that acquires external environmental information; a verification location setting unit that sets a target verification location based on the information from the external environment acquisition unit and the collection condition acquisition unit; and a guidance control unit that guides the vehicle to the set verification location.

[0145] According to this, when collecting update data for use in updated models, it becomes possible to proactively collect the necessary update data based on predetermined collection conditions, which is required to comprehensively verify and analyze various driving conditions to confirm driving safety. Furthermore, the present invention is not limited to the embodiments described above, and various modifications are included. The embodiments described above are explained in detail for the purpose of clearly illustrating the present invention, and are not necessarily limited to those having all the configurations described. In addition, it is possible to replace parts of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace other configurations for the configuration of each embodiment. [Explanation of Symbols]

[0146] 201...Collection condition acquisition unit, 202...External environment acquisition unit, 203...Verification location setting unit, 204...Notification unit, 205...Response reception unit, 206...Guidance control unit, 207...Collection condition update unit, 208...Camera, 210...External information acquisition unit, 211...Updated model side recognition processing unit, 212...Current model side recognition processing unit, 213...Recognition processing result verification unit.

Claims

1. In a vehicle information collection system equipped with update data collection means for collecting update data to verify and analyze update models related to vehicle operation, The update data collection means is A collection condition acquisition unit that acquires predetermined collection conditions for collecting the update data of the update model, An external environment acquisition unit that acquires external environmental information, Based on the information from the external environment acquisition unit and the collection condition acquisition unit, a verification location setting unit sets the target verification location. A guidance control unit that guides the vehicle to the designated verification location, A notification unit that asks the operator of the vehicle whether to permit or refuse the collection of the update data, The notification unit comprises a response receiving unit that receives a response from the operator indicating whether they permit or refuse the collection of the update data. When the response receiving unit receives a response granting permission to collect the update data, the guidance control unit guides the vehicle to the verification location. A vehicle information collection system characterized by the following features.

2. In the vehicle information collection system according to Claim 1, The aforementioned notification unit, The system includes a notification content selection unit that selects and notifies the operator of decision-making support information to assist in their decision to permit or refuse the collection of the update data. A vehicle information collection system characterized by the following features.

3. In the vehicle information collection system according to Claim 1, The update data collection means is When the response receiving unit receives a response refusing to collect the update data, the guidance control unit is equipped with a cancellation determination unit that cancels the control to guide the vehicle to the verification location. A vehicle information collection system characterized by the following features.

4. In the vehicle information collection system according to Claim 3, When the cancellation determination unit receives operator operation information from the operator operation reception unit, which detects the override state by the operator, it cancels the control by the guidance control unit that guides the vehicle to the verification location. A vehicle information collection system characterized by the following features.

5. In the vehicle information collection system according to Claim 1, The update data collection means is When the guidance control unit guides the vehicle to the verification location, the system includes an external information acquisition unit that acquires image data of the surrounding environment by imaging means that photographs the surrounding environment of the vehicle during the process of guiding it to the verification location. A vehicle information collection system characterized by the following features.

6. In the vehicle information collection system according to claim 5, The update data collection means is The update model side recognition processing unit recognizes the aforementioned image data, A recognition processing result verification unit compares the recognition result of the updated model side recognition processing unit with the recognition result of the image data by the current model side recognition processing unit of the current model used for controlling the vehicle, The system includes a communication unit that transmits the recognition result verification data from the recognition processing result verification unit to an external verification / analysis means. A vehicle information collection system characterized by the following features.

7. In the vehicle information collection system according to claim 6, The recognition processing result verification unit adds discrimination data for cases where there is a difference between the two image data and cases where there is no difference, and transmits the recognition result verification data from the communication unit to the verification / analysis means. A vehicle information collection system characterized by the following features.

8. In the vehicle information collection system according to claim 6, The updated model's recognition processing unit and the current model's recognition processing unit both recognize the image data captured by the external information acquisition unit. A vehicle information collection system characterized by the following features.

9. In the vehicle information collection system according to claim 6, The verification location setting unit includes a driving condition classification unit that classifies manual driving by the operator and automated driving by the vehicle based on the recognition results of the updated model side recognition processing unit and the current model recognition processing unit. The induction control unit, If classified as manual driving, the navigation system will guide the driver to the verification location. When classified as autonomous driving, the vehicle is guided to the verification location by at least the vehicle's power unit and steering unit. A vehicle information collection system characterized by the following features.

10. In the vehicle information collection system according to claim 6, The update data collection means is The collection condition update unit updates the collection conditions stored in the collection condition storage unit that constitutes the collection condition acquisition unit, based on information from the induction control unit and the recognition processing result verification unit. A vehicle information collection system characterized by the following features.

11. In the vehicle information collection system according to claim 10, If the verification by the recognition processing result verification unit fails, the collection condition update unit updates the collection conditions in the collection condition storage unit, including the fact that the verification failed. A vehicle information collection system characterized by the following features.

12. In the vehicle information collection system according to claim 7, The communication unit transmits at least the image data from the external information acquisition unit, the recognition processing data from the updated model side recognition processing unit, the processing result verification data from the recognition processing result verification unit, and the collected data from the verification location setting unit to the verification / analysis means. A vehicle information collection system characterized by the following features.

13. In the vehicle information collection system according to Claim 1, The aforementioned data collection condition acquisition unit stores a list of data collection conditions, each assigned a priority, which includes at least the "location to be verified," the "time of day to be verified," the "weather conditions to be verified," and the "vehicle driving conditions to be verified." A vehicle information collection system characterized by the following features.

14. In the vehicle information collection system according to Claim 1, The update data collection means is The vehicle-to-vehicle information communication unit communicates with other vehicles to obtain at least the driving speed and driving direction of the other vehicles and inputs this information to the guidance control unit. The induction control unit controls the vehicle's driving state to create the necessary scenery in accordance with the driving state of the other vehicle. A vehicle information collection system characterized by the following features.

15. In the vehicle information collection system according to claim 6, The update data collection means is The system includes a vehicle processing result collection unit that acquires processing result data of the vehicle, and the processing result data of the vehicle processing result collection unit is sent from the communication unit to the verification / analysis means with location information added. A vehicle information collection system characterized by the following features.

16. In the vehicle information collection system according to claim 6, The verification location setting unit includes a driving condition classification unit that classifies manual driving by the operator and remote driving by the vehicle remote control unit based on the recognition results of the updated model side recognition processing unit and the current model recognition processing unit. The induction control unit, If classified as manual driving, the navigation system will guide the driver to the verification location. When classified as remotely controlled operation, the vehicle is guided to the verification location by at least the vehicle's power unit and steering unit in response to a command from the vehicle remote control unit. A vehicle information collection system characterized by the following features.

Images