Vehicle information collection system
The vehicle information collection system addresses inefficiencies in passive data collection by actively gathering image data based on predetermined conditions, ensuring comprehensive verification and analysis for autonomous vehicle development.
Patent Information
- Application Number
- JP2023075454
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-01
- Publication Date
- 2024-11-14
- Estimated Expiration
- 2043-05-01
AI Technical Summary
Existing image data collection methods for autonomous vehicle development are inefficient, as they rely on passive collection based on chance encounters with specific external events, leading to insufficient data for comprehensive verification and analysis under various driving conditions.
A vehicle information collection system that proactively collects image data by setting predetermined collection conditions, using a collection condition acquisition unit, external environment acquisition, verification location setting, and guidance control to ensure comprehensive data gathering under specified conditions.
Enables comprehensive and efficient collection of image data necessary for verifying and analyzing updated models, ensuring thorough verification and analysis of driving safety under various conditions.
Smart Images

Figure 2024160454000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a vehicle information collecting system for collecting various types of vehicle information, and more particularly to a vehicle information collecting system having an AI (artificial intelligence) function. [Background technology]
[0002] Recently, autonomous driving, which allows cars to drive autonomously, has been developed, and to implement autonomous driving, it is effective to use AI functions to control the driving of the car. When developing a new updated model for the current model that is actually installed and operating in the car, and making the car drive autonomously, it is necessary to verify and analyze whether the updated model can operate as intended.
[0003] In order to verify and analyze this updated model, it is necessary to spend a huge amount of time to comprehensively collect driving data under various driving conditions with the actual vehicle in order to confirm driving safety. In particular, image data of the driving environment under a variety of driving conditions is necessary to make the vehicle self-driving. For this reason, image data is collected by checking the operating status of the vehicle while actually driving the vehicle, and then verified and analyzed.
[0004] For example, in order to allow a car to drive autonomously, it is necessary to verify and analyze the degree to which the recognition information inferred from actually captured image data resembles the actual correct data.In addition, the image data used for learning and verifying this AI is collected by driving multiple cars and sharing the image data collection among each car, thereby shortening the time required to collect a large amount of image data.
[0005] However, in general, when collecting image data, a large amount of image data of scenes that are easy to photograph is collected, and image data of unusual scenes, such as different weather conditions or different time periods, is not collected, and various image data is not collected comprehensively, so sufficient verification and analysis cannot be performed. In order to resolve this situation, for example, a technology described in Patent Document 1 (International Publication No. 2022 / 107595) is known as a technology for comprehensively collecting image data.
[0006] Patent Document 1 discloses that, for the purpose of efficiently relearning a recognition model, the system includes a collection timing control unit that controls the timing of collecting image data candidates that are candidates for image data to be used in relearning the recognition model, and an image data collection unit that selects image data from the collected image data candidates based on at least one of the characteristics of the image data candidates and the similarity with stored image data.
[0007] In addition, the recognition model is used to recognize a specified recognition target (object) around the vehicle, and the image data collection unit selects appropriate image data from image data candidates including image data obtained by photographing the surroundings of the vehicle with an imaging means such as a camera installed in the vehicle. [Prior art documents] [Patent documents]
[0008] [Patent Document 1] International Publication No. 2022 / 107595 Summary of the Invention [Problem to be solved by the invention]
[0009] Incidentally, the method of collecting image data described in Patent Document 1 determines whether or not to collect image data based on a certain external event (e.g., an event such as the occurrence of construction work, the occurrence of an accident, rainfall conditions, or snowfall conditions), and classifies the image data collected based on this external event according to the output result (reliability) of the recognition model.
[0010] However, with this image data collection method, image data cannot be collected unless the relevant external event (construction, accident, rainfall, snowfall, etc.) is encountered by chance. Thus, with the passive image data collection method of Patent Document 1, the probability of encountering the above-mentioned external events during normal driving is low, so it is uncertain whether the actually required image data can be collected sufficiently, and there is a problem that verification and analysis efficiency is poor. It is conceivable that similar problems will arise not only with image data of the update model, but also with other update data.
[0011] An object of the present invention is to provide a vehicle information collection system that can actively collect update data required for comprehensively verifying and analyzing various driving conditions to confirm driving safety, based on predetermined collection conditions, when collecting update data to be used for an update model. [Means for solving the problem]
[0012] The present invention is characterized in that, in a vehicle information collection system that collects update data for verifying and analyzing an update model related to vehicle driving, it has a collection condition acquisition unit that acquires predetermined collection conditions for collecting update data for the update model, an external environment acquisition unit that acquires external environmental information, a verification location setting unit that sets a target verification location based on the information from the external environment acquisition unit and the collection condition acquisition unit, and a guidance control unit that guides a vehicle to the set verification location. Effect of the Invention
[0013] According to the present invention, in collecting update data used for verifying and analyzing an update model, it is possible to actively collect update data necessary for comprehensively verifying and analyzing various driving conditions for confirming driving safety based on predetermined collection conditions. [Brief description of the drawings]
[0014] [Figure 1] 1 is a configuration diagram showing a configuration for explaining the concept of a vehicle information collection system according to the present invention; [Diagram 2] 1 is a control block diagram showing a specific configuration of a vehicle information collection system according to a first embodiment of the present invention. [Diagram 3] 3 is a control block diagram showing a specific configuration of a collection condition acquisition unit shown in FIG. 2. [Figure 4] 3 is a control block diagram showing a specific configuration of a verification location setting unit shown in FIG. 2. [Diagram 5] 3 is a control block diagram showing a specific configuration of a notification unit shown in FIG. 2. [Figure 6] 3 is a control block diagram showing a specific configuration of a guidance control unit shown in FIG. 2. [Figure 7] 3 is a flow chart diagram showing a control flow for executing the functions of the control blocks shown in FIG. 2. [Figure 8] FIG. 8 is an explanatory diagram showing an example of a table for determining an operation mode in the control flow (step S04) shown in FIG. [Figure 9] FIG. 2 is an explanatory diagram illustrating map information displayed on a display screen of a vehicle navigation device. [Figure 10] FIG. 5 is a control block diagram showing a specific configuration of a vehicle information collection system according to a second embodiment of the present invention. [Figure 11] 11 is a control block diagram showing a specific configuration of a guidance control unit shown in FIG. 10. [Figure 12] 11 is a flow chart diagram showing a control flow for executing the functions of the control blocks shown in FIG. [Figure 13]FIG. 11 is a control block diagram showing a specific configuration of a vehicle information collection system according to a third embodiment of the present invention. [Figure 14] FIG. 11 is a control block diagram showing a specific configuration of a vehicle information collection system according to a fourth embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0015] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the following embodiment, and various modifications and application examples within the technical concept of the present invention are also included within its scope. EXAMPLES
[0016] 1 shows a configuration for explaining the concept of a vehicle information collection system according to the present invention. The vehicle information collection system is roughly composed of a server 10 equipped with a server-side control unit having an AI function unit, a vehicle (here, an automobile) 11 equipped with a vehicle-side control unit also having an AI function unit, and communication means (not shown) provided in each of the server 10 and the vehicle 11 so that the server 10 and the vehicle 11 can communicate wirelessly.
[0017] Here, the AI function unit of the server 10 is equipped with an update model to be verified, and, if necessary, a current model used for controlling the vehicle 11. Also, the AI function unit of the vehicle 11 is equipped with a current model used for actual control, and the above-mentioned update model. Note that the update model may be installed only in the server 10 or the vehicle 11, or the server 10 and the vehicle 11 may constitute an update model.
[0018] Actual autonomous driving and other driving control of the vehicle is performed using the current model, which has already been verified, and the updated model is not used. Therefore, the updated model of the AI function part installed in the vehicle control unit does not affect the driving of the vehicle, ensuring the safety of the vehicle.
[0019] Since the object of the present invention is to comprehensively collect update data for verifying and analyzing an update model, detailed explanations of the configuration of the update model, the verification of collected images, the analysis method, operation, etc. will be omitted, and the explanation will focus on the collection method of the update data. Also, the update data here is image data used to recognize a predetermined recognition target (target, etc.) around the vehicle 11, and is used for a recognition model as an example of an update model.
[0020] The recognition targets are objects such as other vehicles, pedestrians, and traffic signs that are required for autonomous driving. Of course, other objects can also be recognized. In addition, the recognition process results of the image data of the current model and the updated model are compared with each other to obtain recognition result verification data. For example, if there is a difference between the current model and the updated model in the recognition result verification data, the image data is considered to be the target of verification and analysis.
[0021] In Fig. 1, a server-side control unit of server 10 transmits a request to acquire image data (update data) to a vehicle-side control unit of vehicle 11. The request to acquire image data is made according to a predetermined collection condition list. The collection conditions in the collection condition list are, for example, "location to be verified (hereinafter referred to as verification location)", "time period to be verified (hereinafter referred to as verification time period)", "weather conditions to be verified (hereinafter referred to as verification weather conditions)", "vehicle driving conditions to be verified (hereinafter referred to as verification driving conditions)", etc. Of course, it is also possible to set collection conditions other than these.
[0022] The verification location may refer to a specific verification location (position), or may refer to a verification area having a size including the specific verification location. In the following, a description will be given of a verification area including the specific verification location. The specific verification location is, for example, a junction or other location where a large amount of image data is required to allow the vehicle 11 to travel safely. Needless to say, the specific verification location may be any other location.
[0023] Then, when the vehicle-side control unit of vehicle 11 determines according to the collection condition list that the conditions are met, that is, that the vehicle is traveling near a verification location (e.g., a junction) during a verification time period (e.g., around 5 p.m. in the evening), under verification weather conditions (e.g., rainfall), the vehicle displays a request on the display screen of vehicle 11 or issues an audio request to the driver of vehicle 11 to travel through the verification location (here, the verification area). Note that various conditions other than those mentioned above can be set for the items in the collection condition list.
[0024] When the driver responds by accepting the request to drive in the verification area, the vehicle-side control unit displays route guidance through the verification area to the verification location on the display screen of the navigation device. The driver drives the vehicle 11 by himself or automatically by the current model according to the route guidance on the display screen to drive to the verification location. If necessary, remote control from the server 10 (embodiment shown in FIG. 14) can also be performed.
[0025] During this travel process, the vehicle-side control device acquires image data of the surrounding environment of the vehicle 11 using imaging means 12 such as a camera. The image data may be still images or moving images (in the case of moving images, image data for each hour is used). Then, when the vehicle arrives at the verification location, collection of image data is completed. Of course, it is also possible to collect image data of the surroundings of the verification location as needed. This can be decided appropriately depending on the need for the image data to be collected.
[0026] The image data may also include vehicle position data, distance data from the imaging means 12 to characteristic points of targets such as landmarks and marks, time data, weather data, etc. The image data may also include driving condition data necessary for analysis, and may further include other data.
[0027] Examples of image data of the guided route are, for example, image data when driving on a straight road in the evening, image data when driving on a curved road during rainfall, etc. These image data are image data when they meet the above-mentioned collection conditions, such as the verification location, verification time period, verification weather, etc.
[0028] Then, the image data obtained by the imaging means 12 such as a camera is edited as a data set by the AI function unit of the vehicle-side control unit, and is transmitted to the server-side control unit of the server 10 and uploaded. The data set includes information such as vehicle driving situation data, position data, weather data, and recognition result verification data, which are linked to the image data. As described above, the recognition result verification data is data indicating that, when a difference occurs between the recognition processing result of the image data of the current model and the recognition processing result of the image data of the updated model, the difference is to be subject to verification and analysis. These data are transmitted to the server 10, which is the "verification / analysis means."
[0029] Then, the AI function unit of the server-side control unit of the server 10 uses the uploaded data set to perform detailed verification and analysis of the recognition model. It goes without saying that this detailed verification and analysis is performed by a developer or the like.
[0030] As described above, the recognition result verification data is obtained by comparing the recognition process result based on the image data of the current model and the recognition process result based on the image data of the updated model, which are based on the same driving conditions, and verifying whether there is a difference between the two. In other words, if there is no difference in the recognition of vehicles and targets between the two image data, the current model and the updated model are deemed to have the same capabilities or performance, and if there is a difference in the recognition of vehicles and targets between the two image data, the current model and the updated model are deemed to have different capabilities or performance. Based on this verification result, the server-side control unit of the server 10 can perform a detailed analysis of the updated model. Note that the server-side control unit of the server 10 integrates data sets uploaded from many vehicles 11, verifies and analyzes the recognition model, and reconstructs the recognition model to improve its completeness.
[0031] In the present invention, when the collection conditions in the collection condition list as described above are satisfied, the vehicle-side control unit requests the driver to drive toward the verification location, and when the request is accepted, provides route guidance to the verification location.Then, the vehicle-side control unit operates to acquire image data required for the journey to the verification location and perform the collection work.As a result, image data that needs to be verified and analyzed can be comprehensively and proactively collected, and the recognition model in the updated model can be verified and analyzed efficiently.
[0032] Next, a specific embodiment will be described. Fig. 2 shows a specific control block configuration of an update data collection means constituting a vehicle information collection system according to a first embodiment of the present invention. This vehicle information collection system is provided in an on-board control unit of a vehicle 11, but may be configured in cooperation with a server 10.
[0033] In FIG. 1, the collection condition acquisition unit 201 acquires collection conditions from the collection condition list sent from the server 10. As described above, the collection conditions are, for example, "verification location," "verification time period," "verification weather conditions," "verification driving conditions," and the like. It is sufficient that appropriate collection conditions are set, and it is not necessary that all collection conditions are satisfied. In addition, the collection conditions may be prioritized. For example, when there are multiple scenes that match the collection conditions while the vehicle 11 is traveling, the priority is determined so that the scene with the least occurrence frequency for which image data is to be collected is the collection target. In other words, the scene that occurs rarely is given priority.
[0034] The external environment acquisition unit 202 acquires external data corresponding to at least the above-mentioned collection conditions. For example, the current vehicle position, the current time, the current weather, the current driving direction and driving conditions, etc. The current vehicle position can be obtained from a GPS sensor or the server 10, etc., the current time can be obtained from the vehicle's clock function, the time stamp of the GPS signal, the server 10, etc., the current weather can be obtained from the server 10, the current driving direction can be obtained from a navigation device, and the driving conditions can be obtained from a power control device or a steering control device. Note that information corresponding to the above-mentioned collection conditions can also be obtained from information other than these.
[0035] The collection condition data from the collection condition acquisition unit 201 and the external environment data from the external environment acquisition unit 202 are input to a verification location setting unit 203. The verification location setting unit 203 compares the external environment data, such as the current vehicle position, the current time, the current weather, and the current driving direction and driving conditions, with the collection conditions, such as "verification location," "verification time zone," "verification weather conditions," and "verification driving conditions," and executes a judgment as to whether or not all of these conditions are met.
[0036] Here, the current vehicle position and the verification location (location) to be verified do not have to match perfectly, and it is sufficient that the vehicle is present in the above-mentioned verification area, and the location condition is considered to match even when the vehicle 11 is traveling near the verification location. For example, if the vehicle 11 is present within a 5 km radius of the verification location, the location condition can be considered to match. When the verification location setting unit 203 determines that the above-mentioned collection conditions and the external environment data match, the verification location setting unit 203 sends notification data to the notification unit 204 of the vehicle's display device, and also sends the location data of the verification location (destination) to the guidance control unit 206.
[0037] When the notification data is sent, the notification unit 204 displays a request for cooperation in verification by using a pop-up function on the display screen of the navigation device. In addition to this display, the request for cooperation in verification can also be read out aloud. Here, the pop-up function displays a collection permission button and a collection refusal button at the same time as the request for cooperation in collection.
[0038] When the driver who has received a request for cooperation in collection selects either the collection permission button or the collection refusal button on the display screen of the navigation device, the response receiving unit 205 accepts the selection. When the collection refusal button is selected, the original state (default state) is restored. On the other hand, when the collection permission button is selected, guidance start data is sent to the guidance control unit 206 to guide the vehicle 11. In this state, the pop-up function is stopped, and the state of the display screen of the navigation device is restored.
[0039] When the collection permission button is pressed, the guidance control unit 206 displays map data of a predetermined range including the position data and route data of the vehicle 11 based on the position data of the verification location sent from the verification location setting unit 203. Here, the vehicle position data is obtained from the GPS sensor 208. Then, the guidance control unit 206 obtains a guidance route from links and nodes connecting the position data of the vehicle 11 and the position data of the verification location, and displays this guidance route by superimposing it on the map data.
[0040] When the vehicle 11 is being manually driven by the driver operating the steering wheel, the guidance control unit 206 guides the driver to the verification location in the same manner as a normal navigation device, and when the vehicle 11 is being driven autonomously, the guidance control unit 206 guides the vehicle 11 to the verification location autonomously. In this way, the vehicle 11 is driven to the verification location and image data is acquired. When the vehicle 11 arrives at the verification location, image data is collected as necessary. Furthermore, the guidance control unit 206 notifies the collection condition update unit 207 that the vehicle is driving to the verification location and instructs it to update the collection conditions.
[0041] The image data is mainly image data captured by the camera 208, and this image data is linked to position data from the GPS sensor 209 as described below. The image data from the camera 208 is acquired as still image data or as moving image data by the external information acquisition unit 210 and stored in a predetermined storage area.
[0042] The image data acquired by the external information acquisition unit 210 is sent to the communication unit 214 which transmits data to the updated model side recognition processing unit 211. In the updated model side recognition processing unit 211, the acquired image data is subjected to recognition processing by the AI function unit, and the output of this recognition processing result is sent to the recognition processing result verification unit 213. Here, the updated model side recognition processing unit 211 is a recognition processing in the updated model, for example, recognizing people and targets from image data, and the recognition processing result of image data including people and targets is sent to the recognition processing result verification unit 212.
[0043] At this time, in parallel, the current model side recognition processing unit 211 of the current model is performing recognition processing using the same image data. Therefore, the recognition processing result of the image data including people and targets by the current model side recognition processing unit 211 of the current model is also sent to the recognition processing result verification unit 213.
[0044] The recognition processing result verification unit 213 verifies the recognition processing result of the acquired image data. For example, the capability or performance of the updated model is verified from the difference between the recognition processing result of the image data of the updated model and the recognition processing result of the image data of the current model under the same collection conditions.
[0045] For example, if there is no difference, verification and analysis are not necessary, and if there is a difference, verification and analysis are necessary. In this way, the recognition processing result verification unit 213 derives data on whether or not to verify and analyze the updated recognition model based on the presence or absence of a difference. Note that the recognition processing result verification unit 213 receives input of position data from the GPS sensor 209 and links it to the recognition processing verification result data.
[0046] If there is no substantial difference in the recognition process results, it is considered that there is no difference in the recognition process between the updated model and the current model. In other words, the capabilities or performance of the current model and the updated model can be considered to be similar. At this time, discrimination data indicating that there is no difference is added to the recognition process result verification data.
[0047] On the other hand, if there is a difference in the recognition processing results, it is assumed that there is a difference in the recognition processing between the updated model and the current model. In other words, it can be assumed that the capabilities or performance of the current model and the updated model are different. At this time, discrimination data indicating that there is a difference is added to the recognition processing result verification data. Therefore, in this case, the developer uses the server 10 to verify and analyze the recognition model to be updated based on the image data at that time.
[0048] The image data from the external information acquisition unit 210, the recognition processing result data from the updated model side recognition processing unit 211, and the recognition processing result verification data from the recognition processing result verification unit 213 are sent to the communication unit 214 and transmitted as a data set to the server 10 by wireless communication means. In the server 10, the AI function unit of the server-side control unit performs verification and analysis of the recognition model based on the received image data, recognition processing result data, and recognition processing result verification data, and reconstructs the recognition model.
[0049] Incidentally, the image recognition processing of the updated model side recognition processing unit 211 and the specific methods of verification and analysis in the server 10 are well-known technologies and have little direct relevance to the present invention, so detailed explanations are omitted.
[0050] Furthermore, the recognition process result verification data from the recognition process result verification unit 213 is sent to the collection condition update unit 207, and if the verification is sufficient under the above-mentioned current collection conditions in the collection condition list, the verification is deemed to be completed and the collection condition list is updated so that the current collection conditions are removed from the next collection condition list. Here, if the recognition process result verification unit 213 fails to verify the authentication result of the current recognition model and the recognition result of the updated recognition model, this failure result is also sent to the collection condition update unit and updated and reflected as failure information.
[0051] Furthermore, this updated collection condition list is sent again to the collection condition acquisition unit 201, and when a new verification location, verification time period, verification weather conditions, etc. that match the collection conditions appear, the above-mentioned operation is repeated. In this way, the scene to be verified is actively searched for, so that the efficiency of verification and analysis of the updated recognition model can be improved.
[0052] Next, the main control blocks among the above-mentioned control blocks will be briefly described. Fig. 3 shows the configuration of the collection condition acquisition unit 201, Fig. 4 shows the configuration of the verification location setting unit 203, Fig. 5 shows the configuration of the notification unit 204, and Fig. 6 shows the configuration of the guidance control unit 206.
[0053] 3, the collection condition acquisition unit 201 is composed of an external collection condition acquisition unit 301 and a collection condition storage unit 302. The external collection condition acquisition unit 301 has a function of receiving a collection condition list from the server 10, and receives the collection conditions such as "verification location", "verification time zone", "verification weather conditions", and "verification driving situation" as described above.
[0054] The received collection condition list is then stored in the collection condition storage unit 302. The collection condition storage unit 302 is composed of a writable / readable memory such as a RAM or a flash ROM. The collection condition storage unit 302 stores a list of collection conditions including at least "verification location," "verification time period," "verification weather conditions," and "traveling conditions of the verification vehicle" with collection priorities.
[0055] The collection condition storage unit 302 receives an updated collection condition list from the collection condition update unit 207, and in this case, the collection condition list in the memory is overwritten. The collection conditions stored in the collection condition storage unit 302 are sent to the next verification location setting unit 203.
[0056] 4, the verification location setting unit 203 is composed of a driving condition classification unit 401, a destination setting unit 402, and a collection condition setting unit 403, and the collection conditions from the collection condition acquisition unit 201 are input to the driving condition classification unit 401, the destination setting unit 402, and the collection condition setting unit 403. In addition, the collection condition setting unit 403 receives external environment information from the external environment acquisition unit 102. The environment information is, for example, the current time and weather conditions, and the vehicle's driving direction and driving conditions, which correspond to the collection conditions. The current position can be acquired from a navigation device, etc.
[0057] The driving condition classification unit 401 classifies whether the vehicle will travel to the verification location under the control of the driver, or under automatic driving, or under instructions from the server 10. The method of this classification will be described later.
[0058] Furthermore, the destination setting unit 402 sets the destination to guide the vehicle 11 to the verification location when the vehicle 11 travels near the verification location to collect image data. For this purpose, the mutual relationship between the current vehicle position and the location of the verification location is monitored, and when the vehicle approaches the vicinity of the verification location, the verification location is provisionally set as the destination. The route to guide the vehicle and the destination is the same as the well-known route search performed by a navigation device. Furthermore, when the collection consent button is selected, the provisional determination becomes final and is set as the destination.
[0059] The collection condition setting unit 403 also acquires environmental information corresponding to the collection conditions acquired from the collection condition acquisition unit 101, and when the collection conditions and the environmental information match, it notifies the notification unit 104 that the collection conditions and the environmental information match, and prompts the pilot to decide whether to permit or reject the collection. Similarly, it functions to send the driving conditions set by the driving condition classification unit 401 and the destination set by the destination setting unit 402 to the guidance control unit 106.
[0060] 5, the notification unit 204 is composed of a notification content selection unit 501 and a notification content display unit 502, and selects a message to be conveyed to the driver when the vehicle approaches the vicinity of the verification location from the verification location setting unit 203. This message is selected by selecting text data such as "Request for cooperation in collection" or icon data such as "Collection consent button" or "Collection refusal button." When a message to the driver is selected, this message data is sent to the notification content display unit 502.
[0061] Furthermore, to assist the operator in deciding whether to permit or reject collection, it is also possible to filter and display decision support information necessary for the operator to decide whether or not to allow collection. This is displayed on the display screen as necessary. One example is the location of the verification location, the time required to drive to the verification location, etc., and this selected decision support information is sent to the notification content display unit 502. The location of the verification location and the time required to drive to the verification location are important information for deciding whether to permit or reject collection of image data. For example, if the time required is quite long, the operator can decide to reject collection of image data.
[0062] The notification content display unit 502 displays the above-mentioned messages and decision support information on, for example, the display screen of the navigation device. For this display, for example, a pop-up function is provided in the navigation device, and the above-mentioned messages and decision support information are displayed by the pop-up function.
[0063] Here, since a functional display area for icon data such as a collection permission button and a collection refusal button is set on the display screen of the navigation device, the notification content display unit 502 functions to send the input state of the collection permission button or collection refusal button indicating whether collection is allowed or not to the response receiving unit 105. The information on whether collection is allowed or not received by the response receiving unit 205 is sent to the guidance control unit 206.
[0064] The guidance control unit 206 is made up of a driver's operation receiving unit 601 , a refusal operation receiving unit 602 , a stop determination unit 603 , a vehicle guidance operation unit 604 , and a collection state switching unit 605 .
[0065] The pilot operation reception unit 601 has a function of receiving an input to stop collection by the pilot when the pilot wants to stop the image data collection work in the middle of the image data collection work being performed by the pilot. Also, the pilot operation reception unit 601 has a function of judging that an override state has been entered when an operation signal of the accelerator pedal, brake pedal, etc. is entered during automatic driving, or when a steering wheel operation, etc. is performed, and receiving an input to stop the image data collection work. Also, during manual driving, if the pilot deviates from a previously presented guide route, etc., the pilot operation reception unit 601 has a function of judging that an override state has been entered, and receiving an input to stop the image data collection work.
[0066] In this embodiment, the operator operation reception unit 601 is operated by input from a collection operation stop button displayed on the display screen of the navigation device. Also, the refusal operation reception unit 602 receives an input of refusal to collect image data input in the notification content display unit 502 via the response reception unit 105.
[0067] The input signal to stop collection and the input signal to refuse collection are sent to the stop determination unit 603, and when the stop determination unit 603 determines that either of these input signals is present, it inputs a collection stop signal to not collect image data to the collection state switching unit 605. The collection state switching unit 605 receives either the collection permission signal from the response receiving unit 105 or the collection stop signal from the stop determination unit 603, and in response thereto, executes a function of setting whether to automatically drive the vehicle to the verification location, to guide the driver, or to return to the original driving route. The setting status by this setting function is sent to the vehicle guidance operation unit 604 and the collection condition update unit 207.
[0068] When the vehicle guidance operation unit 604 is set to guide the vehicle to the verification location, it executes various control operations related to guidance. For example, when the driver operates the vehicle to head to the verification location, a map including the vehicle position and the verification location is displayed on the display screen of the navigation device, and the guidance route is superimposed on the map, and route guidance is executed using voice or the like.
[0069] On the other hand, when heading to the inspection location by autonomous driving, a map including the vehicle position and the inspection location is displayed on the display screen of the navigation device, and the guided route is superimposed on this to provide route guidance, while the steering device and power unit (internal combustion engine or electric motor) are operated to automatically drive the vehicle to the inspection location.
[0070] In the vehicle information collection system according to this embodiment, when the collection conditions in the collection condition list are met, the vehicle-side control unit requests the driver to drive toward the verification location, and when the request is accepted, the vehicle-side control unit performs route guidance to the verification location.Then, the vehicle information collection system operates to acquire image data required until the vehicle arrives at the verification location and perform collection work.As a result, image data that requires verification and analysis can be comprehensively and proactively collected, enabling efficient verification of the recognition model.
[0071] Although the vehicle information collection system shown in Fig. 2 is expressed by functional blocks, the actual operation is executed by control software of a computer mounted on a vehicle-side control device of the vehicle 11. The operation executed by this control software will be explained below with reference to the flowchart shown in Fig. 7.
[0072] <Step S01> In step S01, the server 10 acquires collection conditions according to a predetermined collection condition list from the server-side control unit. The collection conditions in the collection condition list are, for example, "verification location," "verification time period," "verification weather conditions," "verification vehicle driving conditions," etc. After acquiring the collection conditions, the process proceeds to step S02.
[0073] <Step S02> In step S02, current external environmental information corresponding to the collection conditions in the collection condition list is acquired. For example, the vehicle position, the verification time period, the verification weather conditions, and the verification vehicle driving status in the verification area corresponding to the verification location shown in step S01 are acquired. After acquiring the environmental information, the process proceeds to step S03. <Step S03> In step S03, the collection conditions acquired in step S01 are compared with the environmental information acquired in step S02, and the matching state is determined to extract the corresponding collection conditions. In other words, it is determined whether a state in which the collection conditions acquired in step S01 and the environmental information acquired in step S02 match has occurred, and if it is determined that they match, it is determined that collection can be performed.
[0074] For example, if the vehicle 11 satisfies the following conditions: the vehicle 11 is traveling near a verification location, which is a collection condition; the current time is within a verification time period, which is a collection condition; the current weather is in a rainy state, which is a collection condition; and the current traveling direction is toward a specified landmark, which is a collection condition, the system determines that image data can be collected and extracts these as collection conditions. When the collection condition for collecting image data is extracted, the system proceeds to step S04.
[0075] <Step S04> In step S04, the contents of the collection conditions extracted in step S03 are determined, and further, the driving method of the vehicle is determined. (1) Pattern 1 in which the current model cannot perform image data recognition processing, and the updated model can perform image data recognition processing; (2) Pattern 2, in which the current model can perform image data recognition processing, and the updated model can also perform image data recognition processing. (3) Pattern 3 in which the current model can perform image data recognition processing and the updated model cannot perform image data recognition processing; (4) There are vehicles in pattern 4, in which the current model cannot perform image data recognition processing, and similarly the updated model cannot perform image data recognition processing either.
[0076] The driving method is then determined based on the recognition status of these models. Figure 8 shows a table for determining the driving method corresponding to the recognition status of this model.
[0077] 8, pattern 1 is a vehicle in which the current model cannot perform the recognition process and only the updated model can recognize the image data. In this case, the vehicle 11 is guided to the verification location, but since the current model cannot perform the recognition process, a manual driving mode by the driver or a remote control driving mode by instructions from the server 10 is selected. Which driving mode is selected is determined according to a predetermined condition setting.
[0078] In this case, since the recognition results of the current model and the updated model cannot be compared and verified, analysis is performed based on the recognition results of the updated model. However, this method still provides image data and recognition results of the required scene, so although it is not possible to compare and verify the recognition results, it is still advantageous in that image data and recognition results are actively collected.
[0079] Pattern 2 is a vehicle in which both the current model and the updated model can perform image data recognition processing. In this case, the vehicle 11 is guided to the verification location, and since the current model can perform image data recognition processing, a manual driving mode by the driver or an automatic driving mode by the current model is selected. Which driving mode is selected is determined according to a predetermined condition setting.
[0080] Pattern 3 is a vehicle in which the current model can perform recognition processing and the updated model cannot perform image data recognition. In this case, since the updated model cannot perform image data recognition processing, the vehicle is not subject to image data collection. Therefore, the vehicle 11 is not guided to the verification location.
[0081] Pattern 4 is a vehicle in which both the current model and the updated model cannot perform image data recognition processing. In this case, the updated model cannot perform image data recognition processing, so it is excluded from the collection of image data. Therefore, the vehicle 11 is not guided to the verification location.
[0082] Once the contents of the collection conditions have been determined and the vehicle driving method has been determined, the process proceeds to step S05.
[0083] <Step S05> In step S05, the notification content (message) to the driver is selected. This notification content is selected by selecting text data such as "Request for cooperation in collection" or icon data such as "Collection permission button" or "Collection refusal button." In addition, to assist the driver in deciding whether to permit or refuse the collection of image data, decision support information required for the driver to decide whether or not to collect image data is selected (filtered). One example is the location of the verification location, the time required to drive to the verification location, etc. Once the notification content (message) to the driver is selected, the process proceeds to step S06.
[0084] <Step S06> In step S06, the notification content selected in step S05 is notified to the driver. For example, the notification content is displayed on the display screen of the navigation device. For example, a pop-up function is provided to the navigation device, and the notification content is displayed by the pop-up function. Furthermore, the request for cooperation in collection can be notified by voice. Once the driver has been notified, the process proceeds to step S07.
[0085] <Step S07> In step S07, it is determined whether the driver selected the "collection permission button" or the "collection refusal button" displayed on the display screen of the navigation device in step S06. The driver can also decide whether or not to cooperate with the collection of image data based on the verification location and the driving time required to reach the verification location displayed on the display screen. If it is determined that the image data collection refusal button was selected (No determination), the process proceeds to step S08, and if it is determined that the image data collection permission button was selected (Yes determination), the process proceeds to step S10.
[0086] <Step S08> In step S08, since the driver has refused to collect image data of the recognition model to be updated, the execution of the image data collection operation is canceled. This allows the driver to drive normally. In addition, the display of the notification content displayed on the display screen of the navigation device is erased. When the collection operation is canceled, the process proceeds to step S09.
[0087] <Step S09> In step S09, the collection condition list is updated. However, in this case, the collection operation has been canceled by the operator, so the collection condition list is updated to state that image data has not been collected under the current collection conditions and that the current collection conditions will not be presented again. This ends the current control flow.
[0088] <Step S10> In step S10, since it is determined that the collection permission button was selected in step S07, the collection of image data of the current recognition model and the recognition model to be updated is performed. To collect image data of the recognition model, it is necessary to guide the vehicle to the verification location. In this case, the vehicle 11 is guided to the verification location using the driving method determined in step S04.
[0089] The vehicle-side control unit of the vehicle 11 displays route guidance to the verification location to be verified on the display screen of the navigation device. The driver drives the vehicle 11 according to the route guidance on the display screen, by driving the vehicle 11 by himself, by remote control from the server 10, or by automatic driving using a current model, to the verification location to be verified. During this driving, the vehicle-side control unit acquires image data of the surrounding environment of the vehicle 11 using the imaging means 12 such as a camera. The image data may be a still image or may be a moving image. Note that, in addition to a camera, point cloud data from a laser sensor or the like may also be used as the imaging means.
[0090] Furthermore, the acquired image data of the current model and the updated model are subjected to a recognition process, and the results of the recognition process are compared and verified. For example, a difference between the results of the recognition process of the image data of the current model and the image data of the updated model under the same collection conditions is calculated, and the presence or absence of this difference leads to data on whether or not to verify and analyze the updated recognition model. This verification process is performed until the collection of image data is completed in step S12, but the control step of step S11 may be performed during the process.
[0091] <Step S11> In step S11, a determination is made as to whether or not the image data collection work needs to be stopped. For example, the driver may want to stop the collection work for some reason. In this case, the driver selects a "collection work stop button" displayed on the display screen of the navigation device. When the collection work stop button is selected (Yes determination), the process returns to step S08, and the execution of the collection work is canceled. This allows the driver to return to the original route and carry out normal driving.
[0092] Also, in step S09, the collection condition list is updated. However, in this case, since the collection operation has been interrupted by the operator, the collection condition list is updated to state that the current collection conditions have not been collected and that the current collection conditions will not be presented again. This ends the current control flow.
[0093] <Step S12> On the other hand, if the collection operation stop button is not selected in step S11 (No determination), the collection operation in step S10 continues, and when all collection operations are completed, the process proceeds to step S13.
[0094] <Step S13> In step S13, the collection condition list is updated. If sufficient collection has been performed under the current collection conditions in the collection condition list, the collection is deemed to be completed, and the collection condition list is updated so that the current collection conditions are removed from the next collection condition list. Note that this updated collection condition list will repeat the above-mentioned operation when a new verification location, verification time period, verification weather conditions, etc. that match the collection conditions appear.
[0095] 9 shows a screen displayed on the display screen of the navigation device when collecting image data. A map display area 902 is provided on a display screen 901 of the navigation device. The map display area 902 displays the vehicle position 903 of the vehicle 11 and a verification location 904 that is the destination, and further displays a guide route 905 from the vehicle position 903 to the verification location 904.
[0096] Therefore, in the case of manual driving, the driver drives the vehicle 11 along the displayed guided route 905, and in the case of automatic driving, the vehicle 11 is automatically driven and guided by the current model of the vehicle-side control device. Then, image data is collected during the course of traveling along the guided route 905.
[0097] Outside the map display area 902 of the display screen 901, there are arranged a text data display section 906 that displays a request for cooperation in collecting image data, a driving method display section 907 that displays whether manual driving or automatic driving is being performed, a collection permission button 908 that permits the collection of image data, a collection refusal button 909 that rejects the collection, a collection work cancel button 910 that cancels the collection work midway, and a required time display section 911 that displays the approximate time required for the collection work.
[0098] In the vehicle information collection system according to this embodiment, when the collection conditions in the collection condition list are met, the vehicle-side control unit requests the driver to drive toward the verification location, and when the request is accepted, the vehicle-side control unit performs route guidance to the verification location. Then, during the driving process to arrive at the verification location, the vehicle information collection system operates to acquire the required image data and perform the collection work. Therefore, image data required for verification can be comprehensively and proactively collected, enabling efficient verification of the recognition model. EXAMPLES
[0099] Next, a second embodiment of the present invention will be described. The second embodiment is characterized in that image data is collected in cooperation with other vehicles. For example, the vehicle-to-vehicle information communication unit communicates with other vehicles to obtain at least the traveling speed and traveling direction of the other vehicles, and the guidance control unit controls the traveling state of the vehicle so as to present a required scene in accordance with the traveling state of the other vehicles. The following description will be given with reference to Figs. 10 to 12.
[0100] Fig. 10 shows an overall control block showing cooperation with other vehicles based on the control block shown in Fig. 2, Fig. 11 shows a detailed control block of the guidance control unit, and Fig. 12 shows a flowchart of the control flow of Fig. 10. Except for the control block for cooperation with other vehicles, Fig. 10 has almost the same configuration as the control block shown in Fig. 2, so a description of the same control block will be omitted.
[0101] 10 and 11, a vehicle position acquisition unit 215 and an inter-vehicle information communication unit 216 are newly provided, and information therefrom is input to a guidance control unit 217. The vehicle position acquisition unit 215 acquires the current position of the vehicle, which can be acquired by detecting the vehicle position and traveling direction used by a navigation device.
[0102] In addition, the vehicle-to-vehicle information communication unit 216 communicates with other vehicles to obtain the positions and traveling directions of the other vehicles, and exchanges control data with the other vehicles, and can communicate with the other vehicles directly or via the server 10.
[0103] 11, the guidance control unit 217 is provided with an inter-vehicle information decoding unit 606 to which information is input from the vehicle position acquisition unit 215 and the inter-vehicle communication unit 216. The control data decoded and generated by the inter-vehicle information decoding unit 606 is input to the vehicle guidance operation unit 604.
[0104] The vehicle-to-vehicle information decoding unit 606 decodes the vehicle-to-vehicle information and the vehicle position information, and inputs control data to the vehicle guidance control unit 604 so that the driving conditions of the vehicle and other vehicles can produce a required scene. The vehicle guidance operation unit 604 drives the vehicle based on this control data, and obtains image data of the driving conditions by the camera 208, as in the first embodiment.
[0105] For example, when acquiring image data by the vehicle as described above, image data may be required in which a moving target (e.g., another vehicle traveling) is present in addition to a stationary target. In this case, it is effective to drive the vehicle in such a way as to present the necessary scene to the other vehicles in order to create the necessary scene.
[0106] Therefore, in this embodiment, the vehicle and the other vehicle exchange information on their respective positions and the traveling speed in the traveling direction through communication, so that each vehicle can produce the above-mentioned scene. For example, as a specific example, when producing a scene in which the vehicle and the other vehicle are traveling toward a merging road and the other vehicle has priority at the merging point, the vehicle-to-vehicle information decoding unit 606 decodes each other's position information, traveling direction, traveling speed, etc., and manages the traveling state of the vehicle so that the camera 208 can capture the situation in which the other vehicle passes through the merging point. This makes it possible to produce a required scene and obtain image data.
[0107] Although the vehicle information collection system shown in Fig. 10 is expressed by functional blocks, the actual operation is executed by control software of a computer mounted on a vehicle-side control device of the vehicle 11. The operation executed by this control software will be explained below with reference to the flowchart shown in Fig. 11.
[0108] <Step S21> In step S21, the server side control unit of the server 10 acquires collection conditions according to a predetermined collection condition list. The collection conditions in the collection condition list are, for example, "verification location," "verification time period," "verification weather conditions," "verification vehicle driving conditions," etc. After acquiring the collection conditions, the process proceeds to step S22.
[0109] <Step S22> In step S22, current external environmental information corresponding to the collection conditions in the collection condition list is acquired. For example, the vehicle position, the verification time period, the verification weather conditions, and the verification vehicle driving status in the verification area corresponding to the verification location shown in step S21 are acquired. After acquiring the environmental information, the process proceeds to step S23. <Step S23> In step S23, the collection conditions acquired in step S21 are compared with the environmental information acquired in step S22, and the matching state is determined to extract the corresponding collection conditions. In other words, it is determined whether a state in which the collection conditions acquired in step S21 and the environmental information acquired in step S22 match has occurred, and if it is determined that they match, it is determined that collection can be performed.
[0110] For example, if the vehicle 11 satisfies the following conditions, it is determined that collection is possible and is extracted as a collection condition: the vehicle 11 is traveling near a verification location, which is a collection condition; the current time is within a verification time period, which is a collection condition; the current weather is in a rainy state, which is a collection condition; and the current traveling direction is toward a specified landmark, which is a collection condition. When the collection condition for collecting image data is extracted, the process proceeds to step S24.
[0111] <Step S24> In step S24, the contents of the collection conditions extracted in step S03 are determined, and further, the driving method of the vehicle is determined. (1) Pattern 1 in which the current model cannot perform image data recognition processing, and the updated model can perform image data recognition processing; (2) Pattern 2, in which the current model can perform image data recognition processing, and the updated model can also perform image data recognition processing. (3) Pattern 3 in which the current model can perform image data recognition processing and the updated model cannot perform image data recognition processing; (4) There are vehicles in pattern 4, in which the current model cannot perform image data recognition processing, and similarly the updated model cannot perform image data recognition processing either.
[0112] Then, the driving method is determined based on the recognition status of these models. The determination of these driving methods is the same as in the first embodiment. After the contents of the collection conditions are determined and the vehicle driving method is determined, the process proceeds to step S25.
[0113] <Step S25> In step S25, the notification content (message) to the driver is selected. This notification content is selected by selecting text data such as "Request for cooperation in collection" or icon data such as "Collection permission button" or "Collection refusal button." In addition, to assist the driver in deciding whether to permit or refuse the collection of image data, decision support information necessary for the driver to decide whether or not to collect image data is selected (filtered). One example is the location of the verification location, the time required to drive to the verification location, etc. Once the notification content (message) to the driver has been selected, the process proceeds to step S26.
[0114] <Step S26> In step S26, communication with other vehicles is started via the communication means. In this case, the other vehicles are other vehicles that exist in the verification area including the verification location, and the vehicle and the other vehicles are linked by this communication. When communication with the other vehicles is started, the process proceeds to step S27.
[0115] <Step S27> In step S27, the notification content selected in step S25 is notified to all drivers of other vehicles. For example, the notification content is displayed on the display screen of the vehicle's navigation device. This display is achieved by providing the navigation device with a pop-up function, and the notification content is displayed by the pop-up function. Furthermore, the request for cooperation in collection can be notified by voice. Once all drivers have been notified, the process proceeds to step S28.
[0116] <Step S28> In step S28, it is determined whether the driver has selected the "collection permission button" or the "collection refusal button" displayed on the display screen of the navigation device. The driver is any driver of other vehicles that were linked by communication in step S26. Here, the driver can also determine whether or not to cooperate with the collection of image data based on the verification location and the travel time required to reach the verification location displayed on the display screen.
[0117] If it is determined that even one of the operators has selected the image data collection refusal button (No judgment), the process proceeds to step S29, and if it is determined that all the operators have selected the image data collection permission button (Yes judgment), the process proceeds to step S31.
[0118] <Step S29> In step S29, since the driver has refused to collect image data of the recognition model to be updated, the execution of the image data collection operation is canceled. As a result, the driver returns to the original route and drives normally. In addition, the display of the notification content displayed on the display screen of the navigation device is erased. When the collection operation is canceled, the process proceeds to step S30.
[0119] <Step S30> In step S30, the collection condition list is updated. However, in this case, the collection operation has been canceled by the operator, so the collection condition list is updated to state that image data has not been collected under the current collection conditions and that the current collection conditions will not be presented again. This ends the current control flow.
[0120] <Step S31> In step S31, since it is determined that the collection permission button was selected by all drivers in step S28, collection of image data of the current recognition model and the recognition model to be updated is executed. To collect image data of the recognition model, it is necessary to guide the vehicle to the verification location. In this case, the vehicle 11 is guided to the verification location using the driving method determined in step S24.
[0121] Here, as described above, the vehicle and the other vehicles are linked to each other, so that the necessary image data is acquired, and the traveling is managed so as to produce the necessary scenery. In this embodiment, the vehicle and the other vehicles exchange information such as their respective position information, traveling direction, traveling speed, etc., by communication, so that each vehicle can produce the above-mentioned scenery.
[0122] For example, in a specific example, when presenting a scene in which the vehicle and another vehicle are traveling toward a merging road and the other vehicle has priority at the merging point, the vehicle-to-vehicle information decoding unit 606 decodes each other's position information, traveling direction, traveling speed, etc., and manages the traveling state of the vehicle so that the situation in which the other vehicle passes through the merging point can be photographed by the camera 208. This makes it possible to produce the necessary scene and acquire image data.
[0123] In addition, while the driving state of the vehicle is managed, the vehicle-side control unit of the vehicle 11 displays route guidance to the verification location to be verified on the display screen of the navigation device. The driver drives the vehicle 11 according to the route guidance on the display screen, by driving the vehicle 11 by himself, by remote control from the server 10, or by automatic driving using a current model, to the verification location to be verified. During this driving, the vehicle-side control unit acquires image data of the surrounding environment of the vehicle 11 by the imaging means 12 such as a camera until the vehicle arrives at the verification location.
[0124] Furthermore, the acquired image data of the current model and the updated model are subjected to a recognition process, and the results of the recognition process are compared to verify the recognition results. For example, a difference between the results of the recognition process of the image data of the current model and the image data of the updated model under the same collection conditions is calculated, and the presence or absence of this difference leads to data on whether or not to verify and analyze the updated recognition model. This verification process is performed until the collection of image data is completed in step S35, but the control steps of steps S32 and S33 may be performed during the process.
[0125] <Step S32> In step S32, a determination is made as to whether or not the image data collection work needs to be stopped. For example, the driver may want to stop the collection work for some reason. In this case, the driver selects the "collection work stop button" displayed on the display screen of the navigation device. When the collection work stop button is selected (Yes determination), the process returns to step S29 and the execution of the collection work is canceled. This allows the driver to return to the original route and carry out normal driving.
[0126] Also, in step S30, the collection condition list is updated. However, in this case, since the collection operation has been interrupted by the operator, the collection condition list is updated to state that the current collection conditions have not been collected and that the current collection conditions will not be presented again. This ends the current control flow.
[0127] <Step S33> On the other hand, if the collection operation stop button is not selected in step S32 (No judgment), the process proceeds to step S33, where it is judged whether the collection operation stop button has been selected by the driver of the other vehicle (even if only one person). If it is judged that the collection operation stop button has been selected (Yes judgment), the process proceeds to step S34, and if it is judged that the collection operation stop button has not been selected (No judgment), the process proceeds to step S35.
[0128] <Step S34> On the other hand, if the collection operation stop button is selected in step S33, communication is executed to the other vehicles to stop the collection of image data in step S34. At the same time, communication with the other vehicles is terminated, and the process returns to step S29 to cancel the execution of the collection operation. Thereafter, steps S29 and S30 are executed.
[0129] <Step S35> On the other hand, if the collection operation stop button is not selected in step S33 (No determination), the collection operation in step S31 continues, and as soon as all collection operations are completed, communication with other vehicles is terminated and the process proceeds to step S36.
[0130] <Step S36> In step S36, the collection condition list is updated. If sufficient collection has been performed under the current collection conditions in the collection condition list, the collection is deemed to be completed, and the collection condition list is updated so that the current collection conditions are removed from the next collection condition list. Note that this updated collection condition list will repeat the above-mentioned operation when a new verification location, time period, weather conditions, etc. that match the collection conditions appear.
[0131] Thus, according to this embodiment, in order to obtain the necessary image data, the vehicle itself and other vehicles are managed to cooperate with other vehicles to present the necessary scene, thereby enabling more efficient collection of image data. EXAMPLES
[0132] Next, a third embodiment of the present invention will be described. The third embodiment is characterized in that, together with acquiring image data, vehicle operation information linked to the image data is also acquired. Hereinafter, the description will be given with reference to FIG. 13.
[0133] Fig. 13 shows the overall control block for acquiring vehicle operation information based on the control block shown in Fig. 2. Note that Fig. 13 has almost the same configuration as the control block shown in Fig. 2, except for the control block for acquiring vehicle operation information, and therefore a description of the same control block will be omitted.
[0134] 13, various on-vehicle equipment 220 is provided on the vehicle 11, and detection signals from these on-vehicle equipment 220 are input to a vehicle information acquisition unit 221. The on-vehicle equipment 220 includes, for example, equipment provided in a power unit, equipment provided in a steering unit, equipment provided in a braking unit, and the like.
[0135] For example, in a power plant, parameters such as accelerator pedal depression amount, engine speed, intake air volume, vehicle speed, and acceleration are detected. Similarly, in a steering system, parameters such as the steering angle of the steering wheel and the steering angle of the wheels are detected. In addition, in a braking system, parameters such as brake pedal depression amount, skid rate, and vehicle deceleration are detected.
[0136] The output of the in-vehicle equipment 220 is input to a vehicle information acquisition unit 221 and converted into vehicle information detection data required for control. The vehicle information detection data obtained by the vehicle information acquisition unit 221 is input to a vehicle operation processing unit 222 and a communication unit 214.
[0137] The vehicle operation processing unit 222 obtains control processing data for specifically controlling the above-mentioned power unit, steering device, braking device, etc., based on the vehicle information detection data, and operates the power unit, steering device, braking device, etc. The control processing data of the vehicle operation processing unit 222 is input to the vehicle processing result collection unit 223, and input to the communication unit 214 and the collection condition update unit 207 as vehicle processing result data.
[0138] Here, the vehicle processing result data is linked to the image data described above, and the vehicle processing result data corresponds to the change in the image data, so that the causal relationship between them can be easily clarified. Furthermore, this associated information can be used for analysis work in the server 10. EXAMPLES
[0139] Next, a fourth embodiment of the present invention will be described. The fourth embodiment relates to the remote control of pattern 1 (see FIG. 8) described in the vehicle driving method of the first embodiment. As described in the first embodiment, if the current recognition model cannot recognize image data, automatic driving cannot be performed. For this reason, a vehicle remote control unit is added to enable automatic driving. The following description will be made with reference to FIG. 14.
[0140] Figure 14 shows the overall control block for remotely controlling the vehicle based on the control block shown in Figure 2. Note that Figure 14 has almost the same configuration as the control block shown in Figure 2, except for the control block for remotely controlling the vehicle, so a description of the same control block will be omitted.
[0141] 14, for example, information for remote control is input from the driving condition classification unit 401 (see FIG. 4) of the verification location setting unit 203 to the vehicle remote control unit 224 provided in the server 10 via the communication unit 214. Also, information that the collection permission button has been selected is input from the response receiving unit 205 via the communication unit 214. Therefore, the vehicle remote control unit 224 instructs the guidance control unit 206 to perform guidance control to head toward the verification location via the communication unit 214. Here, the vehicle remote control unit 224 is provided with a current model and has the same functions as the current model installed in the vehicle 11.
[0142] In pattern 1, since the current recognition model installed in the vehicle 11 does not function, the image data from the external information acquisition unit 210 is input to the vehicle remote control unit 224 via the communication unit 214. Therefore, the vehicle remote control unit 224 can perform the same operation as the current recognition model installed in the vehicle 11.
[0143] Therefore, the control processing data from the vehicle remote control unit 224 is provided to the guidance control unit 206 via the communication unit 214, and autonomous driving can be performed. Of course, it goes without saying that the image data at this time is acquired in the same manner as in the first embodiment. Thus, according to this embodiment, even if the current recognition model of the vehicle 11 does not function, autonomous driving can be performed by the vehicle remote control unit 224 on the server 10 side.
[0144] As described above, the present invention is characterized in that, in a vehicle information collection system that collects update data for verifying and analyzing an update model related to vehicle driving, it has a collection condition acquisition unit that acquires predetermined collection conditions for collecting update data for the update model, an external environment acquisition unit that acquires external environmental information, a verification location setting unit that sets a target verification location based on the information from the external environment acquisition unit and the collection condition acquisition unit, and a guidance control unit that guides a vehicle to the set verification location.
[0145] According to this, in collecting update data to be used for an update model, it becomes possible to proactively collect update data required for comprehensively verifying and analyzing various driving conditions to confirm driving safety based on predetermined collection conditions. The present invention is not limited to the above-described embodiments, but includes various modified examples. The above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those having all of the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace other configurations with respect to the configuration of each embodiment. [Explanation of symbols]
[0146] 201... collection condition acquisition unit, 202... external environment acquisition unit, 203... verification location setting unit, 204... notification unit, 205... response reception unit, 206... guidance control unit, 207... collection condition update unit, 208... camera, 210... external information acquisition unit, 211... updated model side recognition processing unit, 212... current model side recognition processing unit, 213... recognition processing result verification unit.
Claims
1. A vehicle information collection system including an update data collection means for collecting update data for verifying and analyzing an update model related to vehicle travel, The update data collection means includes: a collection condition acquisition unit that acquires predetermined collection conditions for collecting the update data of the update model; an external environment acquisition unit that acquires external environmental information; a verification location setting unit that sets a target verification location based on information from the external environment acquisition unit and the collection condition acquisition unit; a guidance control unit that guides the vehicle to the set verification location; A vehicle information collection system comprising:
2. In the vehicle information collection system according to claim 1, The update data collection means includes: a notification unit that inquires of a driver of the vehicle as to whether to permit or deny collection of the update data; a response receiving unit that receives a response of permission or refusal to collect the update data from the notification unit by the operator, When the response receiving unit receives a response of permission to collect the update data, the vehicle is guided to the verification location by the guidance control unit. A vehicle information collection system comprising:
3. 3. The vehicle information collection system according to claim 2, The notification unit is A notification content selection unit is provided for selecting and notifying decision assistance information that assists the operator in deciding whether to permit or reject the collection of the update data. A vehicle information collection system comprising:
4. 3. The vehicle information collection system according to claim 2, The update data collection means includes: and a stop determining unit that stops the control of guiding the vehicle to the verification location by the guidance control unit when the response receiving unit receives a response of refusal to collect the update data. A vehicle information collection system comprising:
5. The vehicle information collection system according to claim 4, The stop determination unit stops control of guiding the vehicle to the verification location by the guidance control unit when receiving driver operation information from a driver operation reception unit that detects an override state by the driver. A vehicle information collection system comprising:
6. 3. The vehicle information collection system according to claim 2, The update data collection means includes: When the vehicle is guided to the verification location by the guidance control unit, an external information acquisition unit is provided that acquires image data of the surrounding environment by an imaging unit that captures an image of the surrounding environment of the vehicle during the process of guiding the vehicle to the verification location. A vehicle information collection system comprising:
7. 7. The vehicle information collection system according to claim 6, The update data collection means includes: An updated model side recognition processing unit that recognizes the image data; a recognition processing result verification unit that compares the recognition result of the updated model side recognition processing unit with the recognition result of the image data by a current model side recognition processing unit of a current model used for controlling the vehicle; a communication unit that transmits the recognition result verification data of the recognition processing result verification unit to an external verification / analysis means; A vehicle information collection system comprising:
8. The vehicle information collection system according to claim 7, The recognition processing result verification unit adds discrimination data for when there is a difference between the two image data and when there is no difference, and transmits the discrimination data from the communication unit to the verification / difference analysis means. A vehicle information collection system comprising:
9. The vehicle information collection system according to claim 7, The update model side recognition processing unit of the update model and the current model side recognition processing unit of the current model commonly recognize image data captured by the external information acquisition unit. A vehicle information collection system comprising:
10. The vehicle information collection system according to claim 7, The verification location setting unit includes a driving condition classification unit that classifies the driving conditions into manual driving by a pilot and automatic driving by a vehicle based on the recognition results of the updated model side recognition processing unit and the current model recognition processing unit, The induction control unit is If the driving is classified as manual driving, the navigation device guides the vehicle to the verification location; When classified as an automatic driving, the vehicle is guided to the verification location by at least the power unit and steering unit of the vehicle. A vehicle information collection system comprising:
11. The vehicle information collection system according to claim 7, The update data collection means includes: and a collection condition update unit that updates the collection conditions stored in a collection condition storage unit that constitutes the collection condition acquisition unit based on information from the guidance control unit and the recognition processing result verification unit. A vehicle information collection system comprising:
12. The vehicle information collection system according to claim 11, The collection condition update unit updates the collection condition storage unit to include the fact that the verification by the recognition processing result verification unit has failed, if the verification by the recognition processing result verification unit has failed. A vehicle information collection system comprising:
13. The vehicle information collection system according to claim 8, The communication unit transmits at least the image data from the external information acquisition unit, the recognition processing data from the updated model side recognition processing unit, the processing result verification data from the recognition processing result verification unit, and the collected data from the verification location setting unit to the verification / analysis means. A vehicle information collection system comprising:
14. 3. The vehicle information collection system according to claim 2, The collection condition acquisition unit stores a list of collection conditions including at least "location to be verified," "time period to be verified," "weather conditions to be verified," and "vehicle driving conditions to be verified" with collection priorities. A vehicle information collection system comprising:
15. 3. The vehicle information collection system according to claim 2, The update data collection means includes: a vehicle-to-vehicle information communication unit that communicates with other vehicles to acquire at least the traveling speed and traveling direction of the other vehicles and inputs the acquired information to the guidance control unit; The guidance control unit controls the running state of the vehicle so as to present a necessary scene in accordance with the running state of the other vehicle. A vehicle information collection system comprising:
16. The vehicle information collection system according to claim 7, The update data collection means includes: A vehicle processing result collection unit is provided to acquire processing result data of the vehicle, and the processing result data of the vehicle processing result collection unit is provided with location information and is sent from the communication unit to the verification / analysis means. A vehicle information collection system comprising:
17. The vehicle information collection system according to claim 7, The verification location setting unit includes a driving condition classification unit that classifies the driving conditions into manual driving by a driver and remote driving by a vehicle remote control unit based on the recognition results of the updated model side recognition processing unit and the current model recognition processing unit, The induction control unit is If the driving is classified as manual driving, the navigation device guides the vehicle to the verification location; When the vehicle is classified as a remote-controlled driving, the vehicle is guided to the verification location by at least the power unit and steering unit of the vehicle in response to a command from the vehicle remote control unit. A vehicle information collection system comprising:
Citation Information
Patent Citations
Electronic device, method, and program
JP2020086960A
Information processing device, information processing method, and program
WO2022107595A1
