Vehicle information collection system
The vehicle information collection system actively collects image data by setting conditions and guiding vehicles to specific locations, addressing inefficiencies in existing systems by ensuring thorough verification and analysis of AI models for improved driving safety.
Patent Information
- Application Number
- PCT/JP2023/043108
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-01
- Filing Date
- 2023-12-01
- Publication Date
- 2024-11-07
AI Technical Summary
Existing vehicle information collection systems face inefficiencies in collecting comprehensive image data for AI model verification under various driving conditions, particularly due to passive data collection methods that rely on encountering specific external events, leading to inadequate verification and analysis of driving safety.
A vehicle information collection system that includes a collection condition acquisition unit, external environment acquisition unit, verification location setting unit, and guidance control unit to proactively collect image data by setting predetermined conditions, guiding vehicles to specific locations, and acquiring image data under controlled conditions for thorough verification and analysis.
Enables comprehensive and proactive collection of image data, enhancing the efficiency of verifying and analyzing AI models by ensuring data is collected under predetermined conditions, thereby improving driving safety verification.
Smart Images

Figure JP2023043108_07112024_PF_FP_ABST
Abstract
Description
Vehicle information collection system
[0001] The present invention relates to a vehicle information collection system that collects various types of vehicle information, and more particularly to a vehicle information collection system equipped with an AI (artificial intelligence) function.
[0002] Recently, there has been progress in the development of autonomous driving, which allows automobiles to drive autonomously, and to implement this autonomous driving, it is effective to use AI functions to control the vehicle's driving.When developing a new updated model for the current model that is actually installed and operating in the automobile and making the automobile drive autonomously, it is necessary to verify and analyze whether the updated model can operate as intended.
[0003] In verifying and analyzing this updated model, in order to confirm driving safety, it is necessary to spend a huge amount of time to comprehensively collect driving data under various driving conditions using actual vehicles. In particular, to enable autonomous driving of a vehicle, image data of the driving environment under a variety of driving conditions is necessary. For this reason, the vehicle's operating status is checked while actually driving, and image data is collected, verified, and analyzed.
[0004] For example, in order to enable a car to drive automatically, it is necessary to verify and analyze the degree of similarity between the recognition information inferred from actually captured image data and the actual correct data.In addition, the image data used for this AI learning and verification is collected by driving multiple cars and sharing the image data collection among each car, thereby shortening the time required to collect a large amount of image data.
[0005] However, in general, when collecting image data, a large amount of image data of scenes that are easy to photograph is collected, and image data of unusual scenes, such as different weather or different time periods, is not collected, and there is a problem that various image data is not collected comprehensively, making it impossible to sufficiently verify and analyze. To solve this situation, a technology for collecting comprehensive image data is known, for example, as described in Patent Document 1 (WO 2022 / 107595).
[0006] Patent Document 1 discloses that, for the purpose of efficiently relearning a recognition model, the system includes a collection timing control unit that controls the timing of collecting image data candidates that are candidates for image data to be used in relearning the recognition model, and an image data collection unit that selects image data from the collected image data candidates based on at least one of the characteristics of the image data candidates and the similarity with stored image data.
[0007] The recognition model is used to recognize a predetermined recognition target (object) around the vehicle, and the image data collection unit selects appropriate image data from image data candidates including image data obtained by photographing the surroundings of the vehicle with an imaging means such as a camera installed in the vehicle.
[0008] International Publication No. 2022 / 107595
[0009] Incidentally, the image data collection method described in Patent Document 1 determines whether or not to collect image data based on a certain external event (for example, an event such as the occurrence of construction work, the occurrence of an accident, rainfall conditions, or snowfall conditions), and classifies the image data collected based on this external event according to the output result (reliability) of the recognition model.
[0010] However, with this image data collection method, it is not possible to collect image data unless the relevant external event (such as construction work, an accident, rainfall, or snowfall) is encountered by chance. As such, with the passive image data collection method of Patent Document 1, the probability of encountering the above-mentioned external events during normal driving is low, so it is uncertain whether the actually necessary image data can be collected in sufficient quantities, resulting in poor verification and analysis efficiency. It is anticipated that similar issues will arise not only with image data for updated models, but also with other updated data.
[0011] An object of the present invention is to provide a vehicle information collection system that can actively collect, based on predetermined collection conditions, update data necessary for comprehensively verifying and analyzing various driving conditions to ensure driving safety when collecting update data to be used for an update model.
[0012] The present invention is characterized in that the vehicle information collection system collects update data for verifying and analyzing update models related to vehicle driving, and includes a collection condition acquisition unit that acquires predetermined collection conditions for collecting update data for the update model, an external environment acquisition unit that acquires external environmental information, a verification location setting unit that sets a target verification location based on the information from the external environment acquisition unit and the collection condition acquisition unit, and a guidance control unit that guides the vehicle to the set verification location.
[0013] According to the present invention, when collecting update data to be used for verifying and analyzing an update model, it is possible to actively collect update data necessary for comprehensively verifying and analyzing various driving conditions for confirming driving safety based on predetermined collection conditions.
[0014] 7 is a configuration diagram illustrating a configuration for explaining the concept of a vehicle information collection system according to the present invention. FIG. 10 is a control block diagram illustrating a specific configuration of a vehicle information collection system according to a first embodiment of the present invention. FIG. 11 is a control block diagram illustrating a specific configuration of a collection condition acquisition unit shown in FIG. 2. FIG. 12 is a control block diagram illustrating a specific configuration of a verification location setting unit shown in FIG. 2. FIG. 13 is a control block diagram illustrating a specific configuration of a notification unit shown in FIG. 2. FIG. 14 is a control block diagram illustrating a specific configuration of a guidance control unit shown in FIG. 2. FIG. 15 is a flowchart diagram illustrating a control flow for executing the functions of the control blocks shown in FIG. 7. FIG. 16 is an explanatory diagram illustrating an example of a table for determining a driving method in the control flow (step S04) shown in FIG. 7. FIG. 17 is an explanatory diagram illustrating map information displayed on the display screen of a navigation device of a vehicle. FIG. 18 is a control block diagram illustrating a specific configuration of a vehicle information collection system according to a second embodiment of the present invention. FIG. 19 is a control block diagram illustrating a specific configuration of a guidance control unit shown in FIG. 10. FIG. 19 is a flowchart diagram illustrating a control flow for executing the functions of the control blocks shown in FIG. 10. FIG. 19 is a control block diagram illustrating a specific configuration of a vehicle information collection system according to a third embodiment of the present invention. FIG. 19 is a control block diagram illustrating a specific configuration of a vehicle information collection system according to a fourth embodiment of the present invention.
[0015] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the following embodiment, and various modifications and application examples within the technical concept of the present invention are also included within its scope.
[0016] 1 shows a configuration for explaining the concept of a vehicle information collection system according to the present invention. The vehicle information collection system is roughly composed of a server 10 equipped with a server-side control unit having an AI function unit, a vehicle (here, an automobile) 11 equipped with a vehicle-side control unit also having an AI function unit, and communication means (not shown) provided in each of the server 10 and the vehicle 11 so that they can communicate wirelessly.
[0017] Here, the AI function unit of the server 10 is equipped with an updated model to be verified and, if necessary, a current model used to control the vehicle 11. The AI function unit of the vehicle 11 is equipped with a current model used for actual control and the updated model described above. The updated model may be installed only in the server 10 or the vehicle 11, or the server 10 and the vehicle 11 may constitute an updated model.
[0018] The actual driving control of the vehicle, such as autonomous driving, is performed using the current model, which has already been verified, and the updated model is not used. Therefore, the updated model of the AI function part installed in the vehicle control unit does not affect the driving of the vehicle, ensuring the safety of the vehicle.
[0019] Since the object of the present invention is to comprehensively collect update data for verifying and analyzing an update model, detailed explanations of the configuration of the update model, the verification of collected images, the analysis method, operation, etc. will be omitted, and the explanation will focus on the method of collecting update data. Furthermore, the update data here is image data used to recognize predetermined recognition objects (targets, etc.) around the vehicle 11, and is used for a recognition model as an example of an update model.
[0020] The recognition targets are objects such as other vehicles, pedestrians, and traffic signs that are required for autonomous driving. Of course, other targets can also be used. Furthermore, the recognition processing results of the image data of the current model and the updated model are compared with each other to obtain recognition result verification data. For example, if there is a difference between the current model and the updated model in this recognition result verification data, the image data is considered to be the target of verification and analysis.
[0021] In Fig. 1, a server-side control unit of a server 10 transmits a request to acquire image data (update data) to a vehicle-side control unit of a vehicle 11. The request to acquire image data is made in accordance with a predetermined collection condition list. Examples of collection conditions in the collection condition list include "location to be verified (hereinafter referred to as verification location)," "time period to be verified (hereinafter referred to as verification time period)," "weather conditions to be verified (hereinafter referred to as verification weather conditions)," and "vehicle driving conditions to be verified (hereinafter referred to as verification driving conditions)." Of course, other collection conditions can also be set.
[0022] The verification location may refer to a specific verification location (position), or may refer to a verification area having an area that includes the specific verification location. The following description will be given focusing on a verification area that includes the specific verification location. The specific verification location may be, for example, a merging road or other location where a large amount of image data is required to allow the vehicle 11 to travel safely. Of course, it may also be any other location.
[0023] Then, when the vehicle-side control unit of the vehicle 11 determines, according to the collection condition list, that the conditions are met, that the vehicle is traveling near a verification location (for example, a merging road) during a verification time period (for example, around 5 p.m. in the evening) under verification weather conditions (for example, rainfall), the vehicle 11 displays a request on the display screen of the vehicle 11 or issues an audible request to the driver of the vehicle 11 to travel through the verification location (here, the verification area). Note that various conditions other than those mentioned above can be set for the items in the collection condition list.
[0024] When the driver responds with an acceptance of the request to travel through the verification area, the vehicle-side control unit displays route guidance through the verification area to the verification location on the display screen of the navigation device. The driver drives the vehicle 11 to the verification location by himself or automatically using a current model, following the route guidance on the display screen. If necessary, remote control from the server 10 (the embodiment shown in FIG. 14 ) can also be performed.
[0025] During this traveling process, the vehicle-side control device acquires image data of the surrounding environment of the vehicle 11 using an imaging means 12 such as a camera. The image data may be still images or moving images (in the case of moving images, image data for each hour is used). Then, upon arrival at the verification location, the collection of image data is completed. Of course, it is also possible to collect image data of the surroundings of the verification location as needed. This can be decided as appropriate depending on the need for the image data to be collected.
[0026] The image data may also include vehicle position data, distance data from the imaging means 12 to characteristic points of targets such as landmarks and marks, time data, weather data, etc. The image data may also include driving condition data necessary for analysis, and may further include other data.
[0027] Examples of image data of the guided route are, for example, image data when driving on a straight road in the evening, or image data when driving on a curved road during rainfall, as shown in the figure. These image data are image data when they meet the above-mentioned collection conditions, such as the verification location, verification time period, verification weather, etc.
[0028] The image data obtained by the imaging means 12, such as a camera, is then compiled as a data set by an AI function unit of the vehicle-side control unit and sent to the server-side control unit of the server 10 for uploading. The data set includes information linked to the image data, such as vehicle driving situation data, location data, weather data, and recognition result verification data. As described above, the recognition result verification data indicates that if a difference occurs between the recognition processing result of the image data of the current model and the recognition processing result of the image data of the updated model, the difference is to be subject to verification and analysis. These data are sent to the server 10, which is the "verification / analysis means."
[0029] Then, the AI function unit of the server-side control unit of the server 10 uses the uploaded data set to perform detailed verification and analysis of the recognition model. It goes without saying that this detailed verification and analysis is performed by a developer or the like.
[0030] As described above, the recognition result verification data compares the recognition processing results based on image data of the current model and the recognition processing results based on image data of the updated model, assuming the same driving conditions, and verifies whether there are any differences between the two. In other words, if there are no differences in the recognition of vehicles or targets between the two sets of image data, the current model and the updated model are deemed to have the same capabilities or performance. However, if there are differences in the recognition of vehicles or targets between the two sets of image data, the current model and the updated model are deemed to have different capabilities or performance. Based on this verification result, the server-side control unit of the server 10 can perform a detailed analysis of the updated model. The server-side control unit of the server 10 integrates data sets uploaded from many vehicles 11, verifies and analyzes the recognition model, and reconstructs the recognition model to improve its completeness.
[0031] In this invention, when the collection conditions in the collection condition list described above are met, the vehicle-side control unit requests the driver to drive toward the verification location, and if the request is accepted, provides route guidance to the verification location. The vehicle-side control unit then operates to acquire image data required for the journey to the verification location and perform the collection work. This allows for comprehensive and proactive collection of image data that requires verification and analysis, enabling efficient verification and analysis of the recognition model in the updated model.
[0032] Next, a specific embodiment will be described. Fig. 2 shows a specific control block configuration of the update data collection means constituting the vehicle information collection system according to the first embodiment of the present invention. This vehicle information collection system is provided in the vehicle-mounted control unit of the vehicle 11, but may also be configured in cooperation with the server 10.
[0033] In FIG. 1 , the collection condition acquisition unit 201 acquires collection conditions from a collection condition list sent from the server 10. As described above, these collection conditions include, for example, "verification location," "verification time period," "verification weather conditions," and "verification driving conditions." It is sufficient that appropriate collection conditions are set, and it is not necessary for all collection conditions to be satisfied. Priorities may also be assigned to the collection conditions. For example, if there are multiple scenes that match the collection conditions while the vehicle 11 is traveling, the priority is set so that the scene with the least occurrence frequency for which image data is to be collected is the one to be collected. In other words, priority is given to scenes that occur rarely.
[0034] The external environment acquisition unit 202 acquires external data that corresponds to at least the above-mentioned collection conditions. For example, the current vehicle position, the current time, the current weather, the current driving direction and driving conditions, etc. The current vehicle position can be obtained from a GPS sensor, the server 10, etc. The current time can be obtained from the vehicle's timing function, a timestamp of the GPS signal, the server 10, etc. The current weather can be obtained from the server 10. Furthermore, the current driving direction can be obtained from a navigation device, and the driving conditions can be obtained from a power control device or a steering control device. Note that information that corresponds to the above-mentioned collection conditions can also be obtained from information other than these.
[0035] The collection condition data from the collection condition acquisition unit 201 and the external environment data from the external environment acquisition unit 202 are input to the verification location setting unit 203. The verification location setting unit 203 compares the external environment data, such as the current vehicle position, the current time, the current weather, the current driving direction and driving conditions, with the collection conditions, such as the "verification location," "verification time zone," "verification weather conditions," and "verification driving conditions," and determines whether all of these conditions are met.
[0036] Here, the current vehicle position and the verification location (location) to be verified do not have to be an exact match; the vehicle only needs to be present in the verification area described above. Even if the vehicle 11 is traveling near the verification location, the location condition is considered to be met. For example, if the vehicle 11 is present within a 5-km radius of the verification location, the location condition can be considered to be met. When the verification location setting unit 203 determines that the collection conditions and the external environment data match, the verification location setting unit 203 sends notification data to the notification unit 204 of the vehicle's display device and also sends location data of the verification location (destination) to the guidance control unit 206.
[0037] When the notification data is sent, the notification unit 204 displays a request for cooperation in verification using a pop-up function on the display screen of the navigation device. The request for cooperation in verification can also be read out aloud in conjunction with this display. Here, the pop-up function displays a collection permission button and a collection refusal button simultaneously with the request for cooperation in collection.
[0038] When the driver, who has received a request for cooperation in collection, selects either the collection permission button or the collection refusal button on the display screen of the navigation device, the response acceptance unit 205 accepts the selection. When the collection refusal button is selected, the device returns to its original state (default state). On the other hand, when the collection permission button is selected, guidance start data is sent to the guidance control unit 206 to guide the vehicle 11. In this state, the pop-up function is stopped, and the display screen of the navigation device is returned to its original state.
[0039] When the collection permission button is pressed, the guidance control unit 206 displays map data of a predetermined range including the position data and route data of the vehicle 11 based on the position data of the verification location sent from the verification location setting unit 203. Here, the vehicle position data is obtained from the GPS sensor 209. Then, the guidance control unit 206 obtains a guidance route from the links and nodes connecting the position data of the vehicle 11 and the position data of the verification location, and displays this guidance route superimposed on the map data.
[0040] When the vehicle 11 is being manually driven by the driver operating the steering wheel, the guidance control unit 206 guides the driver to the verification location, similar to a normal navigation device, and when the vehicle 11 is being automatically driven, the guidance control unit 206 autonomously guides the vehicle 11 to the verification location. As a result, the vehicle 11 is driven to the verification location and image data is acquired. When the vehicle 11 arrives at the verification location, image data is collected as necessary. Furthermore, the guidance control unit 206 notifies the collection condition update unit 207 that the vehicle is driving to the verification location and instructs it to update the collection conditions.
[0041] The image data is mainly image data captured by the camera 208, and as will be described later, this image data is linked to position data from the GPS sensor 209. The image data from the camera 208 is acquired by the external information acquisition unit 210 as still image data or as moving image data and stored in a predetermined storage area.
[0042] The image data acquired by the external information acquisition unit 210 is sent to a communication unit 214 that transmits data to an updated model side recognition processing unit 211. In the updated model side recognition processing unit 211, the acquired image data is subjected to recognition processing by an AI function unit, and the output of this recognition processing result is sent to a recognition processing result verification unit 213. Here, the updated model side recognition processing unit 211 performs recognition processing on the updated model, for example, recognizing people and targets from image data, and the recognition processing result of image data including people and targets is sent to the recognition processing result verification unit 213.
[0043] At this time, in parallel, recognition processing using the same image data is being performed in the current model side recognition processing unit 212 of the current model. Therefore, the recognition processing result of the image data including people and targets by the current model side recognition processing unit 212 of the current model is also sent to the recognition processing result verification unit 213.
[0044] The recognition processing result verification unit 213 verifies the recognition processing result of the acquired image data. For example, the capability or performance of the updated model is verified from the difference between the recognition processing result of the image data of the updated model and the recognition processing result of the image data of the current model under the same collection conditions.
[0045] For example, if there is no difference, verification and analysis are not necessary, and if there is a difference, verification and analysis are necessary. In this way, the recognition processing result verification unit 213 derives data on whether or not verification and analysis of the updated recognition model are necessary based on the presence or absence of a difference. Note that the recognition processing result verification unit 213 receives input of position data from the GPS sensor 209 and links it to the recognition processing verification result data.
[0046] If there is no substantial difference in the recognition processing results, it is assumed that there is no difference in the recognition processing between the updated model and the current model. In other words, the capabilities or performance of the current model and the updated model can be considered to be similar. In this case, the recognition processing result verification data includes discrimination data indicating that there is no difference.
[0047] On the other hand, if there is a difference in the recognition processing results, it is assumed that there is a difference in the recognition processing between the updated model and the current model. In other words, it can be assumed that the capabilities or performance of the current model and the updated model are different. In this case, the recognition processing result verification data includes discrimination data indicating the existence of a difference. Therefore, in this case, the developer uses the server 10 to verify and analyze the recognition model to be updated based on the image data at that time.
[0048] The image data from the external information acquisition unit 210, the recognition processing result data from the updated model side recognition processing unit 211, and the recognition processing result verification data from the recognition processing result verification unit 213 are sent to the communication unit 214 and transmitted as a data set to the server 10 by wireless communication means. In the server 10, the AI function unit of the server-side control unit performs verification and analysis of the recognition model based on the received image data, recognition processing result data, and recognition processing result verification data, and reconstructs the recognition model.
[0049] The image recognition processing of the updated model side recognition processing unit 211 and the specific methods of verification and analysis in the server 10 are well-known technologies and have little direct relevance to the present invention, so detailed explanations will be omitted.
[0050] Furthermore, the recognition processing result verification data of the recognition processing result verification unit 213 is sent to the collection condition update unit 207, and if the verification is sufficient under the above-mentioned current collection conditions in the collection condition list, the verification is deemed to be complete, and the collection condition list is updated so that the current collection conditions are removed from the next collection condition list. Here, if the recognition processing result verification unit 213 fails to verify the authentication result of the current recognition model and the recognition result of the updated recognition model, this failure result is also sent to the collection condition update unit, and updated and reflected as failure information.
[0051] Furthermore, this updated collection condition list is sent again to the collection condition acquisition unit 201, and when a new verification location, verification time period, verification weather conditions, etc. that match the collection conditions appears, the above-mentioned operation is repeated. In this way, scenes to be verified are actively searched for, thereby improving the efficiency of verification and analysis of the updated recognition model.
[0052] Next, the main control blocks among the above-mentioned control blocks will be briefly described. Fig. 3 shows the configuration of the collection condition acquisition unit 201, Fig. 4 shows the configuration of the verification location setting unit 203, Fig. 5 shows the configuration of the notification unit 204, and Fig. 6 shows the configuration of the guidance control unit 206.
[0053] 3, the collection condition acquisition unit 201 is composed of an external collection condition acquisition unit 301 and a collection condition storage unit 302. The external collection condition acquisition unit 301 has a function of receiving a collection condition list from the server 10, and receives the collection conditions, such as "verification location," "verification time period," "verification weather conditions," and "verification driving situation," as described above.
[0054] The received collection condition list is then stored in the collection condition storage unit 302. The collection condition storage unit 302 is composed of a writable / readable memory such as RAM or flash ROM. The collection condition storage unit 302 stores a list of collection conditions including at least "inspection location," "inspection time period," "inspection weather conditions," and "travel status of the inspection vehicle" with collection priorities assigned.
[0055] The collection condition storage unit 302 receives an updated collection condition list from the collection condition update unit 207, and in this case, the collection condition list in the memory is overwritten. The collection conditions stored in the collection condition storage unit 302 are sent to the next verification location setting unit 203.
[0056] 4, the verification location setting unit 203 is composed of a driving condition classification unit 401, a destination setting unit 402, and a collection condition setting unit 403. Collection conditions from the collection condition acquisition unit 201 are input to the driving condition classification unit 401, the destination setting unit 402, and the collection condition setting unit 403. In addition, external environment information is input to the collection condition setting unit 403 from the external environment acquisition unit 102. The environment information is, for example, the current time, weather conditions, vehicle driving direction, and driving conditions corresponding to the collection conditions. The current location can be acquired from a navigation device, etc.
[0057] The driving condition classification unit 401 classifies whether the vehicle will travel to the verification location under the control of the driver, by automatic driving, or by command from the server 10; the method of this classification will be described later.
[0058] Furthermore, when the vehicle 11 travels near a verification location to be verified and collects image data, the destination setting unit 402 sets the destination to guide the vehicle 11 to the verification location. For this purpose, the system monitors the relationship between the current vehicle position and the location of the verification location, and when the vehicle approaches the vicinity of the verification location, it provisionally sets the verification location as the destination. The route to guide the vehicle and the destination is similar to the well-known route search performed by a navigation device. Furthermore, when the collection consent button is selected, the provisional setting becomes final and is set as the destination.
[0059] The collection condition setting unit 403 also acquires environmental information corresponding to the collection conditions acquired from the collection condition acquisition unit 101, and when the collection conditions and the environmental information match, it notifies the notification unit 104 that the collection conditions and the environmental information match, and prompts the driver to decide whether to allow or reject the collection. Similarly, the unit 403 functions to send the driving conditions set by the driving condition classification unit 401 and the destination set by the destination setting unit 402 to the guidance control unit 106.
[0060] 5, the notification unit 204 is composed of a notification content selection unit 501 and a notification content display unit 502, and selects a message to be conveyed to the driver when the vehicle approaches the vicinity of the verification location from the verification location setting unit 203. This message selection is made by selecting text data such as "Request for cooperation in collection" and icon data such as a "collection consent button" and a "collection refusal button." When a message to the driver is selected, this message data is sent to the notification content display unit 502.
[0061] Furthermore, to assist the driver in deciding whether to permit or deny collection, decision-support information necessary for the driver to decide whether to permit or deny collection can be filtered and displayed. This information is displayed on the display screen as needed. One example is the location of the verification location, the time required to drive to the verification location, etc., and this selected decision-support information is sent to the notification content display unit 502. The location of the verification location and the time required to drive to the verification location are important information for deciding whether to permit or deny collection of image data. For example, if the required time is quite long, the driver can decide to deny collection of image data.
[0062] The notification content display unit 502 displays the above-mentioned messages and decision support information on the display screen of the navigation device, for example. This display is achieved by, for example, providing the navigation device with a pop-up function, and displaying the above-mentioned messages and decision support information using the pop-up function.
[0063] Here, since the display screen of the navigation device has function display areas for icon data such as a collection permission button and a collection refusal button set, the notification content display unit 502 functions to send the input status of the collection permission button or collection refusal button, indicating whether collection is permitted or not, to the response receiving unit 105. The information on whether collection is permitted or not received by the response receiving unit 205 is sent to the guidance control unit 206.
[0064] The guidance control unit 206 is made up of a driver operation reception unit 601 , a rejection operation reception unit 602 , a cancellation determination unit 603 , a vehicle guidance operation unit 604 , and a collection state switching unit 605 .
[0065] The driver operation acceptance unit 601 has a function to accept an input to stop collection by the driver when the driver wants to stop the image data collection work while the image data collection work is being performed by the driver. Also, the driver operation acceptance unit 601 has a function to determine that an override state has occurred and accept an input to stop the image data collection work when an operation signal for the accelerator pedal, brake pedal, etc. is input or when the steering wheel, etc. is operated during automated driving. Also, during manual driving, if the driver deviates from a previously presented guidance route, etc., the driver operation acceptance unit 601 has a function to determine that an override state has occurred and accept an input to stop the image data collection work.
[0066] In this embodiment, the operator operation accepting unit 601 is operated by an input from a collection operation stop button displayed on the display screen of the navigation device. Also, the refusal operation accepting unit 602 accepts an input of refusal to collect image data input to the notification content display unit 502 via the response accepting unit 105.
[0067] The input signal for stopping the collection operation and the input signal for refusing collection are sent to the stop determination unit 603, and when the stop determination unit 603 determines that either of these input signals has been received, it inputs a collection stop signal for not collecting image data to the collection state switching unit 605. The collection state switching unit 605 receives either the collection permission signal from the response receiving unit 105 or the collection stop signal from the stop determination unit 603, and in response thereto, executes a function for setting whether to automatically drive the vehicle to the verification location, to guide the driver, or to return to the original driving route. The setting status by this setting function is sent to the vehicle guidance operation unit 604 and the collection condition update unit 207.
[0068] When the vehicle guidance operation unit 604 is set to guide the vehicle to the inspection location, it executes various control operations related to guidance. For example, when the driver operates the vehicle to head to the inspection location, a map including the vehicle position and the inspection location is displayed on the display screen of the navigation device, and the guidance route is superimposed on this and route guidance is performed using voice or the like.
[0069] On the other hand, when heading to the inspection location by automatic driving, a map including the vehicle position and inspection location is displayed on the display screen of the navigation device, and a guided route is superimposed on this to provide route guidance, and the steering device and power unit (internal combustion engine or electric motor) are operated to automatically drive the vehicle to the inspection location.
[0070] In the vehicle information collection system according to this embodiment, when a collection condition in the collection condition list is met, the vehicle-side control unit requests the driver to drive toward the verification location, and if the request is accepted, the vehicle provides route guidance to the verification location. The system then operates to acquire and collect image data required until the driver arrives at the verification location. This allows for comprehensive and proactive collection of image data that requires verification and analysis, enabling efficient verification of the recognition model.
[0071] Although the vehicle information collection system shown in Fig. 2 is expressed by functional blocks, the actual operation is executed by control software of a computer installed in the vehicle-side control device of the vehicle 11. The operation executed by this control software will be explained below with reference to the flowchart shown in Fig. 7.
[0072] <Step S01> In step S01, collection conditions are acquired from a server-side control unit of the server 10 according to a predetermined collection condition list. The collection conditions in the collection condition list include, for example, "verification location," "verification time period," "verification weather conditions," "verification vehicle driving status," etc. Once the collection conditions are acquired, the process proceeds to step S02.
[0073] <Step S02> In step S02, current external environmental information corresponding to the collection conditions in the collection condition list is acquired. For example, the vehicle position, verification time period, verification weather conditions, and verification vehicle driving status in the verification area corresponding to the verification location indicated in step S01 are acquired. Once the environmental information is acquired, the process proceeds to step S03. <Step S03> In step S03, the collection conditions acquired in step S01 are compared with the environmental information acquired in step S02, a match is determined, and the corresponding collection conditions are extracted. In other words, it is determined whether a state in which the collection conditions acquired in step S01 and the environmental information acquired in step S02 match has occurred, and if a match is determined, it is determined that collection can be performed.
[0074] For example, if the vehicle 11 satisfies the following conditions, it is determined that image data can be collected and extracted as a collection condition: the vehicle 11 is traveling near a verification location, which is a collection condition; the current time is within a verification time period, which is a collection condition; the current weather is in a rainy state, which is a collection condition; and the current direction of travel is toward a predetermined landmark, which is a collection condition. When a collection condition for collecting image data is extracted, the process proceeds to step S04.
[0075] <Step S04> In step S04, the contents of the collection conditions extracted in step S03 are determined, and further, a driving method formula for the vehicle is determined. Here, the vehicles 11 are classified into the following patterns: (1) Pattern 1 in which the current model cannot perform image data recognition processing, but the updated model can; (2) Pattern 2 in which the current model can perform image data recognition processing, and similarly the updated model can also perform image data recognition processing; (3) Pattern 3 in which the current model can perform image data recognition processing, but the updated model cannot; and (4) Pattern 4 in which the current model cannot perform image data recognition processing, and similarly the updated model cannot perform image data recognition processing.
[0076] The driving method is then determined based on the recognition status of these models. Figure 8 shows a table for determining the driving method corresponding to the recognition status of this model.
[0077] 8, pattern 1 is a vehicle in which the current model cannot perform the recognition process and only the updated model can recognize the image data. In this case, the vehicle 11 is guided to the verification location, but because the current model cannot perform the recognition process, either a manual driving mode by the driver or a remotely controlled driving mode instructed by the server 10 is selected. Which driving mode is selected is determined according to a predetermined condition setting.
[0078] In this case, since it is not possible to compare and verify the recognition results of the current model and the updated model, analysis is performed based on the recognition results of the updated model. However, this method still provides the necessary image data and recognition results of the scene, so although it is not possible to compare and verify the recognition results, it is still advantageous in that image data and recognition results are actively collected.
[0079] Pattern 2 is a vehicle in which both the current model and the updated model can perform image data recognition processing. In this case, the vehicle 11 is guided to the verification location, and since the current model can perform image data recognition processing, either a manual driving mode by the driver or an automatic driving mode by the current model is selected. Which driving mode is selected is determined according to predetermined condition settings.
[0080] Pattern 3 is a vehicle in which the current model can perform recognition processing but the updated model cannot perform image data recognition. In this case, since the updated model cannot perform image data recognition processing, the vehicle is excluded from image data collection. Therefore, the vehicle 11 is not guided to the verification location.
[0081] Pattern 4 is a vehicle in which neither the current model nor the updated model can perform image data recognition processing. In this case, the updated model cannot perform image data recognition processing, so it is excluded from image data collection. Therefore, the vehicle 11 is not guided to the verification location.
[0082] Once the contents of the collection conditions have been determined and the vehicle driving method has been determined, the process proceeds to step S05.
[0083] <Step S05> In step S05, the notification content (message) to be sent to the driver is selected. This notification content is selected by selecting text data such as "Request for cooperation in collection" or icon data such as "Collection consent button" or "Collection refusal button." In addition, to assist the driver in deciding whether to consent to or refuse collection of image data, decision support information necessary for the driver to decide whether to collect image data is selected (filtered). Examples include the location of the verification location and the time required to drive to the verification location. Once the notification content (message) to be sent to the driver is selected, the process proceeds to step S06.
[0084] <Step S06> In step S06, the notification content selected in step S05 is notified to the driver. For example, the notification content is displayed on the display screen of the navigation device. This display is achieved, for example, by providing the navigation device with a pop-up function, and the notification content is displayed using the pop-up function. Furthermore, the request for cooperation in collection can also be notified by voice. Once the driver has been notified, the process proceeds to step S07.
[0085] <Step S07> In step S07, it is determined whether the driver selected the "collection permission button" or the "collection refusal button" displayed on the display screen of the navigation device in step S06. The driver can also decide whether or not to cooperate with the collection of image data based on the verification location and the driving time required to reach the verification location displayed on the display screen. If it is determined that the image data collection refusal button was selected (No determination), the process proceeds to step S08, and if it is determined that the image data collection permission button was selected (Yes determination), the process proceeds to step S10.
[0086] <Step S08> In step S08, since the driver has refused to collect image data for the recognition model to be updated, the execution of the image data collection operation is canceled. This allows the driver to drive normally. In addition, the notification content displayed on the display screen of the navigation device is erased. When the collection operation is canceled, the process proceeds to step S09.
[0087] <Step S09> In step S09, the collection condition list is updated. However, in this case, the collection operation has been canceled by the operator, so the collection condition list is updated to state that image data has not yet been collected under the current collection conditions and that the current collection conditions will not be presented again. This ends the current control flow.
[0088] <Step S10> In step S10, since it is determined that the collection permission button was selected in step S07, the image data collection work for the current recognition model and the recognition model to be updated is performed. To collect image data of the recognition model, it is necessary to guide the vehicle to the verification location. In this case, the vehicle 11 is guided to the verification location using the driving method determined in step S04.
[0089] The vehicle-side control unit of the vehicle 11 displays route guidance to the verification location to be verified on the display screen of the navigation device. The driver drives the vehicle 11 according to the route guidance on the display screen, either by driving the vehicle himself, by remote control from the server 10, or by automatic driving using a current model, to the verification location to be verified. During this driving, the vehicle-side control unit acquires image data of the environment surrounding the vehicle 11 using an imaging means 12 such as a camera. The image data may be a still image or a moving image. Note that, in addition to a camera, point cloud data obtained by a laser sensor or the like may also be used as the imaging means.
[0090] Furthermore, the acquired image data of the current model and the updated model are subjected to a recognition process, and the results of this recognition process are compared and verified. For example, the difference between the recognition process results of the image data of the current model and the image data of the updated model acquired under the same collection conditions is calculated, and the presence or absence of this difference is used to derive data on whether or not to verify and analyze the updated recognition model. This verification process is carried out until the image data collection process is completed in step S12, but the control step of step S11 may be executed during this process.
[0091] <Step S11> In step S11, a determination is made as to whether or not the image data collection work needs to be stopped. For example, the driver may want to stop the collection work for some reason. In this case, the driver selects the "collection work stop button" displayed on the display screen of the navigation device. If the collection work stop button is selected (determined as Yes), the process returns to step S08, and the execution of the collection work is canceled. This allows the driver to return to the original route and continue driving normally.
[0092] In addition, in step S09, the collection condition list is updated. However, in this case, the collection work has been interrupted by the operator, so the collection condition list is updated to state that the current collection conditions have not been collected and that the current collection conditions will not be presented again. This ends the current control flow.
[0093] <Step S12> On the other hand, if the collection operation stop button is not selected in step S11 (No determination), the collection operation in step S10 continues, and when all collection operations are completed, the process proceeds to step S13.
[0094] <Step S13> In step S13, the collection condition list is updated. If sufficient collection has been achieved under the current collection conditions in the collection condition list, collection is deemed complete, and the collection condition list is updated so that the current collection conditions are removed from the next collection condition list. Note that this updated collection condition list will repeat the above-mentioned operation if a new verification location, verification time period, verification weather conditions, etc. that match the collection conditions appears.
[0095] 9 shows a screen displayed on the display screen of a navigation device when collecting image data. A display screen 901 of the navigation device is provided with a map display area 902. The map display area 902 displays the vehicle position 903 of the vehicle 11 and a verification location 904, which is the destination, as well as a guide route 905 from the vehicle position 903 to the verification location 904.
[0096] Therefore, in the case of manual driving, the driver drives the vehicle 11 along the displayed guided route 905, and in the case of automatic driving, the vehicle 11 is automatically driven and guided by the current model of the vehicle-side control device. Then, image data is collected during the process of traveling along the guided route 905.
[0097] Outside the map display area 902 of the display screen 901, there are arranged a text data display section 906 that displays a request for cooperation in collecting image data, a driving method display section 907 that displays whether manual driving or automatic driving is being performed, a collection permission button 908 that permits the collection of image data, a collection refusal button 909 that refuses collection, a collection work stop button 910 that stops the collection work midway, and a required time display section 911 that displays the approximate time required for the collection work.
[0098] In the vehicle information collection system according to this embodiment, when a collection condition in the collection condition list is met, the vehicle-side control unit requests the driver to drive toward the verification location, and if the request is accepted, the vehicle-side control unit provides route guidance to the verification location. Then, during the driving process to arrive at the verification location, the system operates to acquire and collect the necessary image data. This makes it possible to comprehensively and proactively collect image data required for verification, enabling efficient verification of the recognition model.
[0099] Next, a second embodiment of the present invention will be described. The second embodiment is characterized by collecting image data in cooperation with other vehicles. For example, the second embodiment is characterized by communicating with other vehicles using an inter-vehicle information communication unit to acquire at least the traveling speed and traveling direction of the other vehicles, and controlling the traveling state of the vehicle by a guidance control unit to present a required scene in accordance with the traveling state of the other vehicles. The following description will be made with reference to FIGS. 10 to 12.
[0100] Fig. 10 shows an overall control block showing cooperation with other vehicles based on the control block shown in Fig. 2, Fig. 11 shows a detailed control block of the guidance control unit, and Fig. 12 shows a flowchart of the control flow of Fig. 10. Except for the control block for cooperation with other vehicles, Fig. 10 has almost the same configuration as the control block shown in Fig. 2, so a description of the same control block will be omitted.
[0101] 10 and 11, a vehicle position acquisition unit 215 and an inter-vehicle information communication unit 216 are newly provided, and information from these units is input to a guidance control unit 217. The vehicle position acquisition unit 215 acquires the current position of the vehicle, which can be acquired by detecting the vehicle position and traveling direction used by the navigation device.
[0102] In addition, the vehicle-to-vehicle information communication unit 216 communicates with other vehicles to obtain the position and direction of travel of the other vehicles, and exchanges control data with the other vehicles, and can communicate with the other vehicles directly or via the server 10.
[0103] 11, the guidance control unit 217 is provided with an inter-vehicle information decoding unit 606 to which information from the vehicle position acquisition unit 215 and the inter-vehicle information communication unit 216 is input. The control data decoded and generated by the inter-vehicle information decoding unit 606 is input to the vehicle guidance operation unit 604.
[0104] The vehicle-to-vehicle information decoding unit 606 decodes the vehicle-to-vehicle information and the vehicle position information, and inputs control data that enables the driving conditions of the vehicle and other vehicles to produce a required scene to the vehicle guidance operation unit 604. The vehicle guidance operation unit 604 drives the vehicle based on this control data, and acquires image data of the driving conditions by the camera 208, as in the first embodiment.
[0105] For example, when acquiring image data by the vehicle as described above, image data may be required in which a moving target (e.g., another vehicle) is present in addition to a static target. In this case, it is effective to drive the vehicle in a way that presents the required scene to the other vehicle in order to create the required scene.
[0106] Therefore, in this embodiment, the host vehicle and the other vehicle exchange information on their respective positions and traveling speeds through communication, so that each vehicle can create the above-mentioned scene. For example, in a specific example, when the host vehicle and the other vehicle are traveling toward a merging road and the other vehicle has priority at the merging point, the vehicle-to-vehicle information decoding unit 606 decodes the host vehicle's position information, traveling direction, traveling speed, etc., and manages the traveling state of the host vehicle so that the camera 208 can capture the other vehicle passing through the merging point. This allows the required scene to be created and image data to be acquired.
[0107] Although the vehicle information collection system shown in Fig. 10 is expressed by functional blocks, the actual operation is executed by control software of a computer installed in the vehicle-side control device of the vehicle 11. The operation executed by this control software will be explained below with reference to the flowchart shown in Fig. 11.
[0108] <Step S21> In step S21, collection conditions are acquired from the server-side control unit of the server 10 according to a predetermined collection condition list. The collection conditions in the collection condition list include, for example, "verification location," "verification time period," "verification weather conditions," "verification vehicle driving status," etc. Once the collection conditions are acquired, the process proceeds to step S22.
[0109] <Step S22> In step S22, current external environmental information corresponding to the collection conditions in the collection condition list is acquired. For example, the vehicle position, verification time period, verification weather conditions, and verification vehicle driving status in the verification area corresponding to the verification location indicated in step S21 are acquired. Once the environmental information is acquired, the process proceeds to step S23. <Step S23> In step S23, the collection conditions acquired in step S21 are compared with the environmental information acquired in step S22, a match is determined, and the corresponding collection conditions are extracted. In other words, it is determined whether a state in which the collection conditions acquired in step S21 and the environmental information acquired in step S22 match has occurred, and if a match is determined, it is determined that collection can be performed.
[0110] For example, if the vehicle 11 satisfies the following conditions, collection is possible and the conditions are extracted as collection conditions: the vehicle 11 is traveling near a verification location, which is a collection condition; the current time is within a verification time period, which is a collection condition; the current weather is in a rainy state, which is a collection condition; and the current direction of travel is toward a predetermined landmark, which is a collection condition. When the collection condition for collecting image data is extracted, the process proceeds to step S24.
[0111] <Step S24> In step S24, the contents of the collection conditions extracted in step S03 are determined, and further, a driving method formula for the vehicle is determined. Here, the vehicles 11 are classified into the following patterns: (1) pattern 1 in which the current model cannot perform image data recognition processing, but the updated model can; (2) pattern 2 in which the current model can perform image data recognition processing, and similarly the updated model can also perform image data recognition processing; (3) pattern 3 in which the current model can perform image data recognition processing, but the updated model cannot; and (4) pattern 4 in which the current model cannot perform image data recognition processing, and similarly the updated model cannot perform image data recognition processing.
[0112] Then, the driving method is determined based on the recognition status of these models. The determination of these driving methods is the same as in the first embodiment. After the contents of the collection conditions are determined and the vehicle driving method formula is also determined, the process proceeds to step S25.
[0113] <Step S25> In step S25, the notification content (message) to be sent to the driver is selected. This notification content is selected by selecting text data such as "Request for cooperation in collection" or icon data such as "Collection consent button" or "Collection refusal button." In addition, to assist the driver in deciding whether to consent to or refuse the collection of image data, decision support information necessary for the driver to decide whether to collect image data is selected (filtered). Examples include the location of the verification location and the time required to drive to the verification location. Once the notification content (message) to be sent to the driver is selected, the process proceeds to step S26.
[0114] <Step S26> In step S26, communication with another vehicle is initiated via the communication means. In this case, the other vehicle is another vehicle present in the verification area including the verification location, and the host vehicle and the other vehicle are linked through this communication. Once communication with the other vehicle is initiated, the process proceeds to step S27.
[0115] <Step S27> In step S27, the notification content selected in step S25 is notified to all drivers of other vehicles. For example, the notification content is displayed on the display screen of the vehicle's navigation device. This display is achieved by providing the navigation device with a pop-up function, and the notification content is displayed using the pop-up function. Furthermore, the request for collection cooperation can also be announced by voice. Once all drivers have been notified, the process proceeds to step S28.
[0116] <Step S28> In step S28, it is determined whether the driver has selected the "collection permission button" or the "collection refusal button" displayed on the display screen of the navigation device. The driver is the driver of all other vehicles linked by communication in step S26. Here, the decision as to whether to cooperate with the collection of image data can also be made based on the verification location and the travel time to the verification location displayed on the display screen.
[0117] If it is determined that at least one of the operators has selected the image data collection refusal button (No judgment), the process proceeds to step S29, and if it is determined that all operators have selected the image data collection consent button (Yes judgment), the process proceeds to step S31.
[0118] <Step S29> In step S29, since the driver has refused to collect image data for the recognition model to be updated, the execution of the image data collection work is canceled. As a result, the driver returns to the original route and continues driving normally. In addition, the notification content displayed on the display screen of the navigation device is erased. When the collection work is canceled, the process proceeds to step S30.
[0119] <Step S30> In step S30, the collection condition list is updated. However, in this case, the collection operation has been canceled by the operator, so the collection condition list is updated to state that image data has not yet been collected under the current collection conditions and that the current collection conditions will not be presented again. This ends the current control flow.
[0120] <<Step S31>> In step S31, since it is determined that the collection permission button was selected by all drivers in step S28, collection of image data of the current recognition model and the recognition model to be updated is executed. To collect image data of the recognition model, it is necessary to guide the vehicle to the verification location. In this case, the vehicle 11 is guided to the verification location using the driving method determined in step S24.
[0121] As described above, the subject vehicle and the other vehicles are linked to each other, and therefore, in order to acquire the necessary image data, the driving is managed so as to produce the necessary scenery. In this embodiment, the subject vehicle and the other vehicles exchange information such as their respective positions, driving directions, and driving speeds via communication, so that each vehicle can produce the above-mentioned scenery.
[0122] For example, in a specific example, when presenting a scene in which the subject vehicle and another vehicle are traveling toward a merging road and the other vehicle has priority at the merging point, the inter-vehicle information decoding unit 606 decodes each other's position information, traveling direction, traveling speed, etc., and manages the traveling state of the subject vehicle so that the other vehicle passing through the merging point can be photographed by the camera 208. This makes it possible to create the required scene and acquire image data.
[0123] Furthermore, while the driving state of the vehicle is managed, the vehicle-side control unit of the vehicle 11 displays route guidance to the verification location on the display screen of the navigation device. The driver drives the vehicle 11 according to the route guidance on the display screen, by driving it himself, by remote control from the server 10, or by automatic driving using a current model, to the verification location. During this driving, the vehicle-side control unit acquires image data of the environment around the vehicle 11 using the imaging means 12 such as a camera until the vehicle arrives at the verification location.
[0124] Furthermore, the acquired image data of the current model and the updated model are subjected to a recognition process, and the results of this recognition process are compared to verify the recognition results. For example, the difference between the recognition process results of the image data of the current model and the image data of the updated model acquired under the same collection conditions is calculated, and the presence or absence of this difference is used to derive data on whether or not to verify and analyze the updated recognition model. This verification process is carried out until the image data collection process is completed in step S35, but the control steps of steps S32 and S33 may be executed during this process.
[0125] <Step S32> In step S32, a determination is made as to whether or not the image data collection work needs to be stopped. For example, the driver may want to stop the collection work for some reason. In this case, the driver selects the "collection work stop button" displayed on the display screen of the navigation device. If the collection work stop button is selected (determined as Yes), the process returns to step S29, and the execution of the collection work is canceled. This allows the driver to return to the original route and continue driving normally.
[0126] In addition, in step S30, the collection condition list is updated. However, in this case, the collection work has been interrupted by the operator, so the collection condition list is updated to state that the current collection conditions have not been collected and that the current collection conditions will not be presented again. This ends the current control flow.
[0127] <Step S33> On the other hand, if the collection operation stop button is not selected in step S32 (determined as No), the process proceeds to step S33, where it is determined whether the collection operation stop button has been selected by the driver of the other vehicle (even if only one driver). If it is determined that the collection operation stop button has been selected (determined as Yes), the process proceeds to step S34, and if it is determined that the collection operation stop button has not been selected (determined as No), the process proceeds to step S35.
[0128] <Step S34> On the other hand, if the collection operation stop button is selected in step S33, communication is executed with the other vehicles to stop the collection of image data in step S34. At the same time, communication with the other vehicles is terminated, and the process returns to step S29, canceling the execution of the collection operation. Thereafter, steps S29 and S30 are executed.
[0129] <Step S35> On the other hand, if the collection operation stop button is not selected in step S33 (No judgment), the collection operation in step S31 continues, and as soon as all collection operations are completed, communication with other vehicles is terminated and the process proceeds to step S36.
[0130] <Step S36> In step S36, the collection condition list is updated. If sufficient data has been collected under the current collection conditions in the collection condition list, collection is deemed complete, and the collection condition list is updated so that the current collection conditions are removed from the next collection condition list. Note that this updated collection condition list will repeat the above-mentioned operation if a new verification location, time period, weather condition, etc. that matches the collection conditions appears.
[0131] Thus, according to this embodiment, in order to obtain the necessary image data, the vehicle itself and other vehicles are managed to cooperate with other vehicles to create the necessary scenes, thereby enabling more efficient collection of image data.
[0132] Next, a third embodiment of the present invention will be described. The third embodiment is characterized in that, together with acquiring image data, vehicle operation information linked to the image data is also acquired. The following description will be given with reference to FIG. 13 .
[0133] Fig. 13 shows the overall control block for acquiring vehicle operation information based on the control block shown in Fig. 2. Note that Fig. 13 has almost the same configuration as the control block shown in Fig. 2, except for the control block for acquiring vehicle operation information, and therefore a description of the same control block will be omitted.
[0134] 13, various on-board equipment 220 is provided on the vehicle 11, and detection signals from these on-board equipment 220 are input to a vehicle information acquisition unit 221. The on-board equipment 220 includes, for example, equipment provided in the power unit, equipment provided in the steering unit, equipment provided in the braking unit, etc.
[0135] For example, in a powertrain, parameters such as accelerator pedal depression, engine speed, intake air volume, vehicle speed, and acceleration are detected. Similarly, in a steering system, parameters such as steering wheel angle and wheel steering angle are detected. In addition, in a braking system, parameters such as brake pedal depression, skid rate, and vehicle deceleration are detected.
[0136] The output of the in-vehicle equipment 220 is input to a vehicle information acquisition unit 221 and converted into vehicle information detection data required for control. The vehicle information detection data obtained by the vehicle information acquisition unit 221 is input to a vehicle operation processing unit 222 and a communication unit 214.
[0137] The vehicle operation processing unit 222 obtains control processing data for specifically controlling the power unit, steering unit, braking unit, etc., based on the vehicle information detection data, and uses this control processing data to operate the power unit, steering unit, braking unit, etc. The control processing data from the vehicle operation processing unit 222 is input to the vehicle processing result collection unit 223, and then input to the communication unit 214 and the collection condition update unit 207 as vehicle processing result data.
[0138] Here, the vehicle processing result data is linked to the image data described above, and the vehicle processing result data corresponds to changes in the image data, so that the causal relationship between them can be easily clarified. Furthermore, this associated information can be used for analysis work in the server 10.
[0139] Next, a fourth embodiment of the present invention will be described. The fourth embodiment relates to the remote control of pattern 1 (see FIG. 8) described in the vehicle driving method of the first embodiment. As described in the first embodiment, if the current recognition model cannot recognize image data, autonomous driving is not possible. For this reason, a vehicle remote control unit is added to enable autonomous driving. The following description will be made with reference to FIG. 14.
[0140] Figure 14 shows the overall control block for remotely controlling the vehicle based on the control block shown in Figure 2. Note that Figure 14 has almost the same configuration as the control block shown in Figure 2, except for the control block for remotely controlling the vehicle, so a description of the same control block will be omitted.
[0141] 14 , for example, information for remote control is input to the vehicle remote control unit 224 provided in the server 10 from the driving condition classification unit 401 (see FIG. 4 ) of the verification location setting unit 203 via the communication unit 214. In addition, information indicating that the collection permission button has been selected is input from the response acceptance unit 205 via the communication unit 214. Therefore, the vehicle remote control unit 224 instructs the guidance control unit 206 via the communication unit 214 to perform guidance control to head toward the verification location. Here, the vehicle remote control unit 224 is a current model and has the same functions as the current model installed in the vehicle 11.
[0142] In pattern 1, the current recognition model installed in the vehicle 11 does not function, so the image data from the external information acquisition unit 210 is input to the vehicle remote control unit 224 via the communication unit 214. Therefore, the vehicle remote control unit 224 can perform the same operation as the current recognition model installed in the vehicle 11.
[0143] Therefore, the control processing data from the vehicle remote control unit 224 is sent to the guidance control unit 206 via the communication unit 214, enabling autonomous driving to be performed. Of course, it goes without saying that the image data at this time is acquired in the same way as in the first embodiment. As described above, according to this embodiment, even if the current recognition model of the vehicle 11 does not function, autonomous driving can be performed by the vehicle remote control unit 224 on the server 10 side.
[0144] As described above, the present invention is characterized in that, in a vehicle information collection system that collects update data for verifying and analyzing update models related to vehicle driving, it has a collection condition acquisition unit that acquires predetermined collection conditions for collecting update data for the update model, an external environment acquisition unit that acquires external environmental information, a verification location setting unit that sets a target verification location based on the information from the external environment acquisition unit and the collection condition acquisition unit, and a guidance control unit that guides the vehicle to the set verification location.
[0145] This makes it possible to actively collect update data necessary for comprehensively verifying and analyzing various driving conditions to verify driving safety when collecting update data to be used in an update model, based on predetermined collection conditions. Note that the present invention is not limited to the above-described embodiments and includes various modifications. The above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace other configurations with respect to the configuration of each embodiment.
[0146] 201...Collection condition acquisition unit, 202...External environment acquisition unit, 203...Verification location setting unit, 204...Notification unit, 205...Response reception unit, 206...Guidance control unit, 207...Collection condition update unit, 208...Camera, 210...External information acquisition unit, 211...Updated model side recognition processing unit, 212...Current model side recognition processing unit, 213...Recognition processing result verification unit.
Claims
1. A vehicle information collection system comprising an update data collection means for collecting update data for verifying and analyzing an update model related to vehicle driving, wherein the update data collection means comprises: a collection condition acquisition unit for acquiring predetermined collection conditions for collecting the update data for the update model; an external environment acquisition unit for acquiring external environmental information; a verification location setting unit for setting a target verification location based on information from the external environment acquisition unit and the collection condition acquisition unit; and a guidance control unit for guiding the vehicle to the set verification location.
2. A vehicle information collection system as described in claim 1, wherein the update data collection means comprises: a notification unit which inquires of the driver of the vehicle as to whether to permit or reject the collection of the update data; and a response receiving unit which receives a response from the driver of the vehicle from the notification unit indicating permission or rejection of the collection of the update data; and when the response receiving unit receives a response indicating permission to collect the update data, the vehicle is guided to the verification location by the guidance control unit.
3. A vehicle information collection system as described in claim 2, characterized in that the notification unit is provided with a notification content selection unit which selects and notifies decision support information which assists the driver in deciding whether to accept or reject the collection of the update data.
4. A vehicle information collection system as described in claim 2, characterized in that the update data collection means includes a stop judgment unit which stops control of guiding the vehicle to the verification location by the guidance control unit when the response receiving unit receives a response refusing to collect the update data.
5. A vehicle information collection system as described in claim 4, characterized in that the cancellation judgment unit cancels control of guiding the vehicle to the verification location by the guidance control unit when it receives driver operation information from a driver operation reception unit that detects an override state by the driver.
6. A vehicle information collection system as described in claim 2, characterized in that the update data collection means is provided with an external information acquisition unit that, when the vehicle is guided to the verification location by the guidance control unit, acquires image data of the surrounding environment by an imaging means that captures images of the surrounding environment of the vehicle during the process of guiding the vehicle to the verification location.
7. A vehicle information collection system as described in claim 6, characterized in that the update data collection means comprises: an update model side recognition processing unit that recognizes the image data; a recognition processing result verification unit that compares the recognition result of the update model side recognition processing unit with the recognition result of the image data by a current model side recognition processing unit of a current model used to control the vehicle; and a communication unit that transmits recognition result verification data of the recognition processing result verification unit to an external verification / analysis means.
8. A vehicle information collection system as claimed in claim 7, characterized in that the recognition processing result verification unit adds discrimination data for when there is a difference between the two image data and when there is no difference, and transmits the discrimination data from the communication unit to the verification / difference analysis means.
9. A vehicle information collection system as described in claim 7, characterized in that the update model side recognition processing unit of the update model and the current model side recognition processing unit of the current model commonly recognize image data captured by the external information acquisition unit.
10. A vehicle information collection system as described in claim 7, wherein the verification location setting unit includes a driving condition classification unit that classifies the driving conditions into manual driving by a driver and automatic driving by the vehicle based on the recognition results of the recognition processing unit on the updated model side and the current model, and the guidance control unit, when classified as manual driving, guides the vehicle to the verification location using a navigation device, and when classified as automatic driving, guides the vehicle to the verification location using at least the vehicle's power unit and steering unit.
11. A vehicle information collection system as described in claim 7, characterized in that the update data collection means includes a collection condition update unit that updates the collection conditions stored in the collection condition storage unit constituting the collection condition acquisition unit based on information from the guidance control unit and the recognition processing result verification unit.
12. A vehicle information collection system as described in claim 11, characterized in that, if the verification by the recognition processing result verification section fails, the collection condition update section updates the collection condition storage section to include the fact that the verification has failed.
13. A vehicle information collection system as described in claim 8, characterized in that the communication unit transmits at least the image data from the external information acquisition unit, the recognition processing data from the updated model side recognition processing unit, the processing result verification data from the recognition processing result verification unit, and the collected data from the verification location setting unit to the verification / analysis means.
14. A vehicle information collection system as described in claim 2, characterized in that the collection condition acquisition unit stores a list of collection conditions including at least the "location to be verified," "time period to be verified," "weather conditions to be verified," and "vehicle driving conditions to be verified" with collection priority.
15. A vehicle information collection system as described in claim 2, wherein the update data collection means comprises an inter-vehicle information communication unit which communicates with other vehicles to obtain at least the traveling speed and traveling direction of the other vehicles and inputs the information to the guidance control unit, and the guidance control unit controls the traveling state of the vehicle so as to present a required scene in accordance with the traveling state of the other vehicles.
16. A vehicle information collection system as described in claim 7, wherein the update data collection means includes a vehicle processing result collection unit that acquires processing result data of the vehicle, and the processing result data of the vehicle processing result collection unit is provided with location information and sent from the communication unit to the verification / analysis means.
17. A vehicle information collection system as described in claim 7, wherein the verification location setting unit includes a driving condition classification unit that classifies the driving conditions into manual driving by a driver and remotely controlled driving by a vehicle remote control unit based on the recognition results of the recognition processing performed by the updated model side recognition processing unit and the current model, and the guidance control unit, when classified as manual driving, guides the vehicle to the verification location using a navigation device, and when classified as remotely controlled driving, guides the vehicle to the verification location using at least the vehicle's power unit and steering unit in response to commands from the vehicle remote control unit.
Citation Information
Patent Citations
Information processing device, information processing method, and program
WO2022107595A1
Electronic device, method, and program
JP2020086960A
