Image recognition model management device and image recognition model management system

Abstract

The purpose of the present invention is to provide an image recognition model management device and image recognition model management system making it possible to select an image recognition model from among a plurality of image recognition models in accordance with an external environment, which is the surrounding environment around a vehicle, and transfer the foregoing to an external world recognition device even if communication with a management server is not possible. In order to achieve the foregoing, the present invention is, for example, an image recognition model management device for transferring, to an external world recognition device, an image recognition model that is transmitted from a management server. The image recognition model management device comprises: a storage unit storing a first image recognition model that is preset and a second image recognition model that has been transmitted at or above a predetermined frequency from the management server; and a model processing unit for selecting the first image recognition model or the second image recognition model and transferring the foregoing to the external world recognition device. On the basis of external world recognition data representing a result of image recognition by the external world recognition device, the model processing unit switches the image recognition model to be transferred to the external world recognition device.

Description

Technical Field

[0001] The present invention relates to an image recognition model management device and an image recognition model management system.

Background Art

[0002] Technological development has been underway to realize a driving support system or an autonomous driving system for automobiles. An external environment recognition device mounted on a moving object such as a vehicle recognizes stationary objects, moving objects, road signs, white lines, etc. existing around the vehicle, but it is known that the recognition accuracy decreases depending on the external environment.

[0003] Regarding this problem, for example, there is the technology described in Patent Document 1. In Patent Document 1, it is described in paragraph 0011 that "the server 200 is a cloud server for providing a cloud service to a terminal. In this embodiment, a recognition model suitable for recognizing the surrounding environment of the moving object Ve is selected from a plurality of recognition models stored in the cloud 201 and supplied to the moving object Ve. The moving object Ve performs object recognition of the surrounding environment using the recognition model supplied from the server 200 and performs ADAS or AD."

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the driving support system described in Patent Document 1, after the recognition model is supplied from the server 200, if the moving object Ve becomes unable to communicate with the server 200, even if the surrounding environment of the moving object Ve changes, a new recognition model is not supplied from the server 200. Therefore, in this case, there is a problem that the moving object Ve uses a recognition model that is not suitable for the surrounding environment.

[0006] Therefore, in order to solve the above problems, the present invention aims to provide an image recognition model management device and an image recognition model management system that can select an image recognition model from a plurality of image recognition models according to the external environment surrounding the vehicle, even when communication with a management server is not possible, and transfer it to an external environment recognition device. [Means for solving the problem]

[0007] To achieve the above objective, the present invention provides, for example, an image recognition model management device that transfers an image recognition model transmitted from a management server to an external recognition device, comprising: a storage unit that stores a pre-set first image recognition model and a second image recognition model transmitted from the management server at a predetermined frequency or more; and a model processing unit that selects the first image recognition model or the second image recognition model and transfers it to the external recognition device, wherein the model processing unit switches the image recognition model to be transferred to the external recognition device based on external recognition data representing the result of image recognition by the external recognition device.

[0008] Furthermore, the image recognition model management system of the present invention is an image recognition model management system comprising, for example, a management server that transmits an image recognition model corresponding to the external environment of a vehicle, and an image recognition model management device that transfers the image recognition model transmitted from the management server to an external environment recognition device, wherein the management server includes a model database in which an image recognition model for each piece of external environment information of the vehicle is stored, an external environment analysis unit that generates external environment information of the vehicle based on the vehicle's current location information, and transfers the image recognition model corresponding to the external environment information generated from the current location information to the model database. The system comprises a model extraction unit that extracts from the image, and a transmission unit that transmits the extracted image recognition model to the image recognition model management device, the image recognition model management device comprising a storage unit that stores a pre-configured first image recognition model and a second image recognition model transmitted from the management server at a predetermined frequency or more, and a model processing unit that selects the first image recognition model or the second image recognition model and transfers it to the external recognition device, the model processing unit switches the image recognition model to be transferred to the external recognition device based on external recognition data representing the result of image recognition by the external recognition device. [Effects of the Invention]

[0009] According to the present invention, even when communication with a management server is not possible, an image recognition model management device and an image recognition model management system can be provided that can select an image recognition model from a plurality of image recognition models according to the external environment of the vehicle and transfer it to an external environment recognition device. Problems, configurations, and effects other than those described above will be clarified by the following description of embodiments. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an example of the overall configuration of the image recognition model management system in Example 1. [Figure 2] This shows an example of a flowchart of the processes performed by the image recognition model management system. [Figure 3] This figure shows an example flowchart for the image recognition model selection process. [Figure 4] This diagram illustrates the extraction of frequently occurring models. [Figure 5] This figure shows an example of a flowchart for the evaluation process of the image recognition model being used. [Figure 6] This figure shows examples of external environmental situations with good recognition rates. [Figure 7] This figure shows examples of external environmental situations with poor recognition rates. [Figure 8A] This is a table showing an example of weather classification. [Figure 8B] This table shows an example of how to classify travel time periods. [Figure 8C] This table shows an example of road shape classification. [Figure 9] Examples of external environment situation patterns [Figure 10] This figure shows an example of the overall configuration of the image recognition model management system in Example 2. [Figure 11] This table shows an example of evaluation data. [Modes for carrying out the invention]

[0011] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. [Examples]

[0012] Figure 1 shows an example of the overall configuration of the image recognition model management system of Embodiment 1. The image recognition model management system comprises a vehicle 1 and a management server 100.

[0013] <Management Server 100> The management server 100 determines whether it is necessary to switch the image recognition model used by the external environment recognition device 10 that performs image recognition, based on the current position information of the vehicle 1. When it is determined that it is necessary to switch to the image recognition model held by the management server 100, it is a device that transmits the image recognition model to the vehicle. That is, the management server 100 is a device that transmits an image recognition model according to the external environment of the vehicle to the image recognition model management device. Specifically, the management server 100 includes an external environment analysis unit 110, a model extraction unit 120, a model database 121, a management server side reception unit 130, and a management server side transmission unit 131.

[0014] <<Management server side reception unit 130>> The management server side reception unit 130 acquires the current position information of the vehicle that transmits the image recognition model. The current position information includes the current time in addition to the current position of the vehicle 1.

[0015] <<External environment analysis unit 110>> The external environment analysis unit 110 collects weather information using a public communication network such as the Internet, for example, and generates weather information 111 (sunny, cloudy, rainy, snowy, thunderstorm, etc.) of the current position of the vehicle 1. The external environment analysis unit 110 may include other communication networks of a wireless communication network such as a WAN (Wide Area Network) and a telephone communication network as means for collecting the weather information 111.

[0016] The external environment analysis unit 110 generates the driving time zone information 112 as the driving time zone that includes the current time among a plurality of preset driving time zones.

[0017] The external environment analysis unit 110 generates road shape information 113 around the current position of the vehicle 1 based on map information and the current position notified from the vehicle. The map information may be held in advance by the management server 100 in a non-volatile memory area, or may be acquired from a public communication network such as the Internet.

[0018] The external environment analysis unit 110 then transmits the generated weather information 111, driving time information 112, and road shape information 113 to the model extraction unit 120. Note that the weather information 111, driving time information 112, and road shape information 113 are examples of external environment information for the vehicle 1, and are not limited to these.

[0019] <<Model extraction unit 120>> The model extraction unit 120 extracts an image recognition model from the model database 121 that is suitable for the external environment of the vehicle 1's current location, based on the weather information 111, driving time information 112, and road shape information 113 transmitted by the external environment analysis unit 110.

[0020] <<Model Database 121>> The model database 121 stores multiple image recognition models for recognizing the surrounding environment of the current location. The stored image recognition models are a group of models that have been trained according to weather conditions, time of day, and road shape.

[0021] <<Management Server Side Transmission Unit 131>> The management server-side transmission unit 131 transmits the image recognition model extracted by the model extraction unit 120 to the vehicle 1. At this time, the weather information 111, driving time information 112, and road shape information 113 used to extract the image recognition model are transmitted along with the image recognition model.

[0022] The management server 100 transmits an image recognition model corresponding to external environmental information such as weather information 111, driving time information 112, and road shape information 113, enabling the external environment recognition device 10 to perform image recognition using an image recognition model suitable for the weather, day and night environment around the vehicle, driving environment, etc.

[0023] <Vehicle 1> Vehicle 1 comprises an external environment recognition device 10, a GNSS (Global Navigation Satellite System) receiver 20, a vehicle position receiving unit 21, a vehicle-side transmitting unit 22, a vehicle-side receiving unit 23, and an image recognition model management device 30.

[0024] <<External perception device 10>> The external perception device 10 performs image recognition on the observation results of the external world by a camera sensor using an image pickup device mounted on the vehicle 1, using an image recognition model, and generates external perception data such as moving objects (pedestrians, bicycles, motorcycles, automobiles, buses, trucks, etc.), road structures (pylons, construction signs, road signs, road markings, traffic lights, etc.), and road conditions (road width, road surface condition, etc.) existing around the vehicle 1. Then, the external perception device 10 transmits the external perception data to the model processing unit 32 of the image recognition model management device 30.

[0025] <<GNSS receiver 20>> The GNSS receiver 20 receives the navigation signals transmitted from the positioning satellites constituting GNSS. The GNSS receiver 20 generates observation data based on the navigation signals and the carrier waves of the signals. The GNSS receiver 20 sequentially outputs the observation data to the own vehicle position receiving unit 21.

[0026] <<Own vehicle position receiving unit 21, vehicle-side transmission unit 22>> The own vehicle position receiving unit 21 generates current position information including the current position and the current time based on the observation data output from the GNSS receiver 20, and sequentially outputs the current position information of the vehicle 1 to the management server 100 via the vehicle-side transmission unit 22. The output of the current position information of the vehicle 1 to the management server 100 is related to the transmission timing of the image recognition model, so it is possible to shorten the output period of the current position information to increase the update frequency of the image recognition model, or to lengthen the period to reduce the update frequency of the image recognition model.

[0027] <<Image recognition model management device 30>> The image recognition model management device 30 includes a microcontroller. The microcontroller is a processor (e.g., a CPU) that executes programs stored in a memory device. By executing a predetermined program, the microcontroller operates as a functional unit that provides various functions. The memory device includes a non-volatile storage area and a volatile storage area. The non-volatile storage area includes a program area that stores programs executed by the microcontroller and a data area that temporarily stores data used by the microcontroller when executing the program. The volatile storage area stores data used by the microcontroller when executing the program. The communication interface connects to other electronic control devices via a network such as CAN or Ethernet. In this embodiment, the image recognition model management device 30 includes a storage unit 31 (e.g., the memory device described above) and a model processing unit 32 (e.g., the processor described above).

[0028] <<Storage section 31>> The memory unit 31 stores the default model m1 (first image recognition model), the frequently occurring model m2 (second image recognition model), and the receiving model m3 (third image recognition model).

[0029] The default model m1 is an image recognition model pre-configured in the memory unit 31 and is stored in the non-volatile memory area as the basic image recognition model used by the external environment recognition device 10. This image recognition model is used immediately after the ignition switch is turned on.

[0030] The receiving model m3 is an image recognition model received by the vehicle-side receiving unit 23 from the management server 100. The vehicle-side receiving unit 23 stores the received image recognition model as receiving model m3 in the storage unit 31. At this time, weather information 111, driving time information 112, and road shape information 113 associated with receiving model m3 are also stored in the storage unit 31. Since transmission from the management server 100 is performed when wireless communication is established, in situations where wireless communication cannot be established, it is considered that the receiving model m3 may be an image recognition model that does not conform to the current external environment. Therefore, the receiving model m3, as well as the weather information 111, driving time information 112, and road shape information 113 associated with receiving model m3, are stored in a volatile memory area.

[0031] The frequently occurring model m2 is the received model m3 for driving time periods or road shapes where image recognition models are frequently transmitted from the management server 100, and is stored in the non-volatile memory area. Details on how frequency is determined will be described later. The frequently occurring model m2 stored in the non-volatile memory area can be deleted by the user. Alternatively, it may be saved as preset memory and selected by the user.

[0032] In this embodiment, as shown in Figure 1, an example is described in which one default model m1, one frequently occurring model m2, and one receiving model m3 are stored in the storage unit 31, but this is not limited to this. For example, multiple copies of any one of the default model m1, frequently occurring model m2, or receiving model m3 may be stored. Also, if no image recognition model is transmitted from the management server 100, the storage unit 31 will not store the receiving model m3. <<Model Processing Unit 32>> The model processing unit 32 selects an image recognition model from the image recognition models stored in the storage unit 31 to be transferred to the external environment recognition device 10, and transfers it to the external environment recognition device 10. It also stores the received model m3 as the frequently occurring model m2 in the storage unit 31, corresponding to the driving time periods or road shapes in which image recognition models are frequently transmitted from the management server 100. Furthermore, it evaluates the image recognition models used by the external environment recognition device 10 based on the external environment recognition data from the external environment recognition device 10. The specific processing performed by the model processing unit 32 will be described later with reference to Figures 3 to 5.

[0033] <Flowchart> Figure 2 shows an example flowchart of the process performed by the image recognition model management system. When the ignition switch is turned on, the process proceeds to step S100.

[0034] In step S100, the vehicle position receiving unit 21 of vehicle 1 generates current position information based on the observation data from the GNSS receiver 20.

[0035] In step S101, vehicle 1 determines whether it has established wireless communication with the management server 100. This determination is made, for example, by sending a communication request from vehicle 1 to the management server 100 and checking the response from the management server 100. If wireless communication is established, the process proceeds to step S102. If wireless communication is not established, the process proceeds to step S108.

[0036] In step S102, vehicle 1 transmits the current location information generated by the vehicle position receiving unit 21 to the management server 100.

[0037] In step S103, the external environment analysis unit 110 of the management server 100 generates weather information 111, driving time information 112, and road shape information 113 based on the current location information received from the vehicle 1.

[0038] In step S104, the model extraction unit 120 of the management server 100 determines whether or not it is necessary to switch the image recognition model. If it is determined that it is necessary to switch the image recognition model, the process proceeds to step S105. If it is determined that it is not necessary to switch the image recognition model, the process proceeds to step S108.

[0039] In step S105, the model extraction unit 120 of the management server 100 extracts an image recognition model to be transmitted to the vehicle 1 from the model database 121 based on weather information 111, driving time information 112, and road shape information 113.

[0040] In step S106, the management server side transmission unit 131 of the management server 100 transmits the image recognition model extracted in step S105 to the vehicle 1.

[0041] In step S107, vehicle 1 receives the image recognition model transmitted from management server 100.

[0042] In step S108, the model processing unit 32 of vehicle 1 selects an image recognition model to transfer to the external environment recognition device 10 from the image recognition models stored in the storage unit 31. The model processing unit 32 also stores the received model m3 as the frequently occurring model m2 in the storage unit 31 for driving time periods or road shapes in which image recognition models are frequently transmitted from the management server 100. Furthermore, the model processing unit 32 evaluates the image recognition model used by the external environment recognition device 10 based on the external environment recognition data from the external environment recognition device 10.

[0043] In step S109, the model processing unit 32 of vehicle 1 transfers the image recognition model extracted in step S108 to the external environment recognition device 10.

[0044] In step S110, the external environment recognition device 10 of the vehicle 1 applies the image recognition model transferred in step S109.

[0045] <<Selection of image recognition model to transfer>> Here, the selection of the image recognition model in step S108 will be explained using Figure 3. Figure 3 is a diagram showing an example of a flowchart for the image recognition model selection process.

[0046] In step S200, vehicle 1 determines whether it has established wireless communication with the management server 100. The determination of whether wireless communication has been established is the same as in step S101 in Figure 2. If wireless communication has been established, the process proceeds to step S201; otherwise, the process proceeds to step S203.

[0047] In step S201, the model processing unit 32 checks whether the image recognition model has been transmitted from the management server 100 to the vehicle 1. Specifically, the model processing unit 32 checks whether the received model m3 is stored in the storage unit 31. If the received model m3 is stored in the storage unit 31, it means that an image recognition model suitable for the external environment is stored, so the process proceeds to step S202. If the received model m3 is not stored in the storage unit 31, it means that an image recognition model suitable for the external environment is not stored, so the process proceeds to step S203.

[0048] In step S202, the model processing unit 32 selects the received model m3 as the image recognition model to be transferred to the external recognition device 10. If the received model m3 has already been selected, it continues to select the received model m3. In this case, no switching of the image recognition model occurs.

[0049] In step S203, the model processing unit 32 checks whether the frequently occurring model m2 is stored in the memory unit 31. If the frequently occurring model m2 is stored, the process proceeds to step S204; otherwise, the process proceeds to step S205.

[0050] In step S204, the model processing unit 32 selects the frequently occurring model m2 as the image recognition model to be transferred to the external environment recognition device 10. If the frequently occurring model m2 has already been selected, it continues to select the frequently occurring model m2. In this case, no switching of the image recognition model occurs.

[0051] In step S205, the model processing unit 32 selects the default model m1 as the image recognition model to be transferred to the external environment recognition device 10. If the default model m1 has already been selected, it continues to select the default model m1. In this case, no switching of the image recognition model occurs.

[0052] <<Extraction of frequently occurring models>> The extraction of frequently occurring models performed by the model processing unit 32 will be explained using Figure 4. Figure 4 shows an example of extracting frequently occurring models m2 by focusing on the travel time period. In Figure 4, the horizontal axis represents the travel time period (from 0:00 to 23:59), and the vertical axis represents the number of transmissions of the travel time period that are sent from the management server 100 in association with the image recognition model. In Figure 4, the width of the travel time period is set to 1 hour. In the example in Figure 4, the number of transmissions for the travel time period from 20:00 to 21:00 is high, exceeding the transmission threshold (e.g., 10 times). That is, the number of transmissions of the image recognition model associated with the travel time period from 20:00 to 21:00 is high, exceeding the transmission threshold (e.g., 10 times). Therefore, the received model m3 associated with the travel time period from 20:00 to 21:00 is stored in the storage unit 31 as frequently occurring model m2. If there are multiple travel time periods that exceed the threshold, and there are also multiple image recognition models, multiple image recognition models may be stored in the storage unit 31 as frequently occurring model m2. Alternatively, only the receiving model m3 associated with the driving time period with the most transmissions may be stored in the memory unit 31 as the frequently occurring model m2. The same applies to road shapes instead of driving time periods. Also, although the example in Figure 4 focuses on the number of transmissions, the transmission ratio (the number of transmissions relative to the total number of transmissions; in this case, the threshold is, for example, 50%) may also be considered.

[0053] <<Evaluation of the image recognition model used by the external environment recognition device 10>> The model processing unit 32 evaluates the image recognition model used by the external recognition device 10 based on the external recognition data from the external recognition device 10. Figure 5 shows an example of a flowchart for the evaluation process of the image recognition model used by the external recognition device 10. The image recognition model used by the external recognition device 10 is the one that the model processing unit 32 has transferred to the external recognition device 10. The flowchart shown in Figure 5 is explained as being performed in step S108 of Figure 2, but is not limited to this. For example, a step separate from step S108 may be included to evaluate the image recognition model being used and to instruct a switch of the image recognition model based on the evaluation result.

[0054] In step S300, the model processing unit 32 acquires external recognition data from the external recognition device 10. External recognition data is the result of image recognition by the external recognition device 10 and includes information on whether an object was detected or not, object type information for detected objects, object recognition rate, etc. The model processing unit 32 is supposed to acquire the external recognition data in real time, but it may also acquire external recognition data for a predetermined period at certain intervals. If, at the time of executing the flowchart shown in Figure 5, external recognition data cannot be acquired from the external recognition device 10 for reasons such as the external recognition device 10 not performing external recognition, the flowchart shown in Figure 5 is not performed.

[0055] In step S301, if the image recognition model used by the external environment recognition device 10 is the default model m1, the model processing unit 32 terminates the flowchart in Figure 5 without evaluating the image recognition model. If the image recognition model used by the external environment recognition device 10 is the frequently occurring model m2 or the receiving model m3, the process proceeds to step S302.

[0056] In step S302, the model processing unit 32 evaluates the image recognition model used by the external recognition device 10 based on the external recognition data, and based on the evaluation result, decides whether or not to switch the image recognition model used by the external recognition device 10 to the default model m1. Specifically, based on the external recognition data, if the number of repetitions of detection and non-detection of an object is greater than or equal to a predetermined value (e.g., 5 times), or if the number of changes in the recognition result of the object type for the same object, such as an object recognized as a regular car changing to a motorcycle, bus, truck, etc., is greater than or equal to a predetermined value (e.g., 5 times), the process proceeds to step S303; otherwise, the process proceeds to step S304.

[0057] In step S303, it is determined that the accuracy of either the frequently occurring model m2 or the receiving model m3 is poor, so the image recognition model being used is switched to the default model m1.

[0058] In step S304, it is determined that there are no problems with the accuracy of the frequently occurring model m2 or the receiving model m3, so the image recognition model being used will continue to be used.

[0059] <Specific Situations> Figure 6 shows an example of a situation with good recognition accuracy. The weather is sunny, the time of day is daytime (8am to 4pm), and the road shape is a general road (city street) with few obstacles. This shows a situation where a pedestrian attempting to cross a crosswalk is detected in this external environment. The situation shown in the example is not an external environment where recognition is difficult, so the basic recognition model can perform recognition with high accuracy, and there is no need to transmit the image recognition model from the management server 100.

[0060] Figure 7 shows an example of a situation with poor recognition accuracy. The weather is rainy, the time of travel is at night (7 PM to 8 AM), the road is a public road (residential road) with puddles, and there is a pedestrian attempting to cross a crosswalk in this external environment. The road is narrow and it is raining, and there is a concern that the recognition accuracy of the basic recognition model will decrease when it comes to pedestrians holding umbrellas. Furthermore, there is a concern that adding this situation to the basic recognition model's training may affect situations with good recognition accuracy. Therefore, by applying the image recognition model transmitted from the management server 100, the deterioration of recognition accuracy can be reduced. In addition, supporting all image recognition models for vehicle 1 would increase memory capacity, cost, and processing load, so the effect of reducing the deterioration of recognition accuracy can be obtained by using a minimum number of image recognition models. However, in the case of sudden events such as sudden thunderstorms or heavy rain, fallen trees or objects, or accidents occurring in front of the vehicle, the transmission of the image recognition model from the management server 100 may not be fast enough. Therefore, by evaluating the image recognition model being used, the deterioration of object recognition accuracy is minimized as much as possible. The effect of this invention, in which the management server 100 analyzes the external environment of vehicle 1 and transmits an appropriate image recognition model, is that there is no variation in the effect of the invention, and the same effect can be brought about for all vehicles. When vehicle 1 analyzes the external environment, the evaluation may differ from vehicle to vehicle, and recognition accuracy may improve in some vehicles and deteriorate in others. In this respect, by having the management server analyze the external environment, a consistent effect can be obtained for all vehicles, and furthermore, it becomes easy to change the method of external environment analysis.

[0061] Figure 8A is a table showing an example of weather classification. Figure 8B is a table showing an example of driving time classification. Figure 8C is a table showing an example of road shape classification. The model extraction unit 120 of the management server 100 determines the image recognition model to be transmitted to vehicle 1, mainly based on combinations of weather, driving time, and road shape. Figures 8A to 8C are examples, and the number of elements in the combination may be increased, or the weather and driving time may be set in more detail. These classifications will change depending on the granularity of the image recognition model to be transmitted.

[0062] Figure 9 illustrates the determination of whether or not to switch the image recognition model to be transmitted. In Figure 9, each row shows the combination of weather information 111, driving time information 112, and road shape information 113 output by the external environment analysis unit 110 of the management server 100. The fourth column from the left indicates whether or not to switch the image recognition model to be transmitted to vehicle 1. Weather and driving time have a significant impact on the image recognition model, so these two factors are often used to determine the image recognition model. Changes in the external environment due to puddles after rain or snow accumulation, as well as mountain roads with many steep slopes, also affect the determination of the image recognition model.

[0063] For example, the model extraction unit 120 of the management server 100 determines that a switch in the image recognition model is necessary if the driving time is between 7 PM and 8 AM. Furthermore, if the driving time is between 4 PM and 7 PM, the model extraction unit 120 determines that a switch in the transmitted image recognition model is not necessary if the weather is sunny, but a switch is necessary if the weather is cloudy. The model extraction unit 120 also determines that a switch in the transmitted image recognition model is necessary if the weather is rainy or snowy. Additionally, the model extraction unit 120 determines that a switch in the transmitted image recognition model is necessary if the road surface is covered in snow. However, the method for determining whether to switch the transmitted image recognition model is not limited to these examples. For instance, the method of determination may vary depending on the season or the region being driven in.

[0064] According to the present invention, even when communication with a management server is not possible, an image recognition model management device and an image recognition model management system can be provided that can select an image recognition model from a plurality of image recognition models according to the external environment of the vehicle and transfer it to an external environment recognition device.

[0065] Furthermore, by providing an analysis and recognition model of the external environment on the management server side, it becomes possible to reduce the variability that occurs in the evaluation of each vehicle. [Examples]

[0066] Figure 10 shows an example of the overall configuration of the image recognition model management system in Example 2. It differs from Example 1 in that the image recognition model management device 30 includes a feedback unit 33. The following will mainly describe the differences from Example 1.

[0067] Embodiment 1 assumed the application and evaluation of an image recognition model transmitted from the management server 100. In contrast, this embodiment assumes that the evaluation results of the image recognition model transmitted from the management server 100 to the vehicle 1 are fed back to the management server 100. Therefore, this embodiment assumes that wireless communication between the management server 100 and the vehicle 1 has been established.

[0068] The feedback unit 33 generates evaluation data 114 based on the evaluation results of the model processing unit 32 so that it can be analyzed by the external environment analysis unit 110 of the management server 100.

[0069] Figure 11 is a table showing an example of evaluation data. The vehicle ID is unique to each vehicle and does not overlap. Latitude, longitude, and direction are stored from information obtained from the GNSS receiver 20 of vehicle 1. Weather, time of day, and road shape are based on information transmitted from the management server 100. Object type and object recognition rate are stored as evaluation results from the model processing unit 32. Vehicle 1 transmits the information from the feedback unit 33 to the management server 100 via the vehicle-side transmission unit 22.

[0070] The management server-side receiving unit 130 of the management server 100 receives evaluation data 114 transmitted from the vehicle. The external environment analysis unit 110 transmits the evaluation data 114 received by the management server-side receiving unit 130 to the model extraction unit 120. When communication between the management server 100 and the vehicle 1 is established and the management server 100 is transmitting an image recognition model, the model processing unit 32 selects the image recognition model (received model m3) transmitted by the management server 100, as shown in the flowchart of Figure 3, and the external environment recognition device 10 will use the received model m3. Therefore, when communication between the management server 100 and the vehicle 1 is established and the management server 100 is transmitting an image recognition model, the evaluation data 114 represents the evaluation of the image recognition model transmitted by the management server 100.

[0071] The model extraction unit 120 switches the image recognition model to be transmitted to another image recognition model and transmits it, for example, if the object recognition rate included in the evaluation data 114 is below a predetermined value. The model extraction unit 120 also stores the evaluation data 114 in the model database 121, linking it to the corresponding image recognition model. For example, if both image recognition model A and image recognition model B stored in the model database 121 are suitable for similar weather information 111, driving time information 112, and road shape information 113, the evaluation data 114 can be used to select the one with the best object recognition rate and transmit it to the vehicle 1.

[0072] Therefore, according to this embodiment, the management server 100 can transmit an image recognition model that is more suitable for the external environment of the vehicle. This improves the accuracy of image recognition by the external environment recognition device 10.

[0073] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are included. For example, the embodiments described above are described in detail to make the present invention easier to understand, and are not necessarily limited to those having all the configurations described. Furthermore, some or all of the above configurations, functions, processing units, processing means, etc., may be implemented in hardware, for example, by designing them as integrated circuits. Furthermore, the above configurations, functions, etc., may be implemented in software by having a processor interpret and execute a program that realizes each function. Information such as programs, tables, files, etc., that realize each function can be stored in memory, a recording medium such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD. [Explanation of Symbols]

[0074] 1...Vehicle, 10...External Environment Recognition Device, 20...GNSS Receiver, 21...Self-Position Receiving Unit, 22...Vehicle-Side Transmitter, 23...Vehicle-Side Receiving Unit, 30...Image Recognition Model Management Device, 31...Storage Unit, 32...Model Processing Unit, 33...Feedback Unit, m1...Default Model, m2...Frequently Occurring Model, m3...Received Model, 100...Management Server, 110...External Environment Analysis Unit, 111...Weather Information, 112...Driving Time Information, 113...Road Shape Information, 114...Evaluation Data, 120...Model Extraction Unit, 121...Model Database, 130...Management Server-Side Receiving Unit, 131...Management Server-Side Transmitter

Claims

1. An image recognition model management device mounted on the same vehicle, which transfers an image recognition model wirelessly transmitted from a management server to the vehicle to an external environment recognition device of the vehicle, A storage unit that stores a pre-configured first image recognition model and a second image recognition model transmitted from the management server at a predetermined frequency or more, The system comprises a model processing unit that selects the first image recognition model or the second image recognition model and transfers it to the external environment recognition device, If communication with the management server cannot be established, the model processing unit will: The system acquires external recognition data representing the image recognition result from the external recognition device using the second image recognition model, evaluates the second image recognition model based on the external recognition data, and switches the image recognition model to be transmitted to the external recognition device to the first image recognition model based on the evaluation result. Based on the external environment recognition data, if it is determined that the number of times an object is detected and not detected exceeds a predetermined value, or if it is determined that the number of times the recognition result for the same object's type is changed exceeds a predetermined value, the image recognition model to be transmitted to the external environment recognition device is switched to the first image recognition model. An image recognition model management device characterized by the following features.

2. An image recognition model management device according to claim 1, The system includes a vehicle-side receiving unit that receives road shape information at the vehicle's current location, information on the vehicle's travel time, and an image recognition model suitable for the road shape information and the travel time information from the management server. The model processing unit measures the number of times each piece of road shape information or each piece of driving time information is transmitted from the management server, and stores the image recognition model suitable for the road shape information or driving time information with a high transmission frequency as the second image recognition model in the storage unit. An image recognition model management device characterized by the following features.

3. An image recognition model management device according to claim 1, The storage unit stores the image recognition model transmitted from the management server as a third image recognition model. If the third image recognition model is stored in the storage unit, the model processing unit selects the third image recognition model and transfers it to the external environment recognition device. An image recognition model management device characterized by the following features.

4. An image recognition model management device according to claim 3, The model processing unit acquires external recognition data from the external recognition device using the second image recognition model or the third image recognition model, evaluates the image recognition model used by the external recognition device based on the external recognition data, and switches the image recognition model to be transferred to the external recognition device to the first image recognition model based on the evaluation result. An image recognition model management device characterized by the following features.

Images