Model update necessity determination system and model update necessity determination method for the model update necessity determination system

Abstract

To appropriately determine an update necessity of a machine learning model according to a situation change within an area that the machine learning model is responsible for.SOLUTION: A model update necessity determining system that determines an update necessity of a machine learning model which predicts a behavior change of a target vehicle based on target vehicle data obtained from the target vehicle within a predefined area where learning is performed by using data from a vehicle under predetermined learning condition, and comprises: a deviation number calculation unit that calculates the number of deviations by which the target vehicle data deviates from the learning conditions based on the target vehicle data of the target vehicle within the area; and an update necessity determining unit that determines an update necessity for the machine learning model based on the deviation number.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a model update necessity determination system and a method for determining the necessity of updating a model update necessity determination system.

Background Art

[0002] Conventionally, as a technical document related to a model update necessity determination system, Japanese Patent Application Laid-Open No. 2020-052607 is known. This publication discloses a model update necessity determination system that determines whether an unstable behavior is caused by a driver when a target vehicle for information collection exhibits an unstable behavior.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in the model update necessity determination system as described above, it is considered to use a machine learning model to process information acquired from a large number of target vehicles. An appropriate model is used for each preset area in the machine learning model. However, since the situation within the area changes over time, it is required to appropriately determine the necessity of updating the machine learning model.

Means for Solving the Problems

[0005] One aspect of the present disclosure is a model update necessity determination system that determines the necessity of updating a machine learning model that is trained using data of vehicles under predetermined learning conditions and predicts changes in the behavior of target vehicles based on target vehicle data acquired from target vehicles within a preset area, The target vehicle data includes at least one of location information, vehicle speed information, and outside temperature information, and the learning conditions include at least one of location range, vehicle speed range, and outside temperature range corresponding to the information contained in the target vehicle data. based on the target vehicle data of the target vehicles within the area and the learning conditions, If the values ​​of the information included in the target vehicle data fall outside the range of the corresponding learning conditions,The system includes a deviation calculation unit that calculates the number of deviations in the target vehicle data from the learning conditions, and an update necessity determination unit that determines whether or not the machine learning model needs to be updated based on the deviation count.

[0006] According to a model update necessity determination system in one aspect of this disclosure, by taking into account the learning conditions of the machine learning model and considering the number of deviations in the target vehicle data within the area, it is possible to appropriately determine whether or not to update the machine learning model in response to changes in the situation within the area covered by the machine learning model.

[0007] Another aspect of this disclosure is a method for determining whether a model needs to be updated, which determines whether a machine learning model needs to be updated, which is trained using data from vehicles under predetermined training conditions and predicts changes in the behavior of a target vehicle based on target vehicle data acquired from target vehicles within a predetermined area. The target vehicle data includes at least one of location information, vehicle speed information, and outside temperature information, and the learning conditions include at least one of location range, vehicle speed range, and outside temperature range corresponding to the information contained in the target vehicle data. Based on the target vehicle data of the target vehicles within the area, If the values ​​of the information included in the target vehicle data fall outside the range of the corresponding learning conditions, The system calculates the number of deviations in the target vehicle data from the learning conditions, and determines whether or not to update the machine learning model based on the number of deviations.

[0008] According to the method for determining whether a model needs to be updated in a model update determination system relating to other aspects of this disclosure, by taking into account the learning conditions of the machine learning model and considering the number of deviations in the target vehicle data within the area, it is possible to appropriately determine whether a machine learning model needs to be updated in response to changes in the situation within the area covered by the machine learning model. [Effects of the Invention]

[0009] According to one aspect and other aspects of this disclosure, it is possible to appropriately determine whether or not to update the machine learning model in response to changes in the situation within the area covered by the machine learning model. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows a model update necessity determination system according to one embodiment. [Figure 2]This is a diagram illustrating the areas of a machine learning model. [Figure 3] This is an example of an update detection server. [Figure 4] This flowchart shows an example of the update determination process of the update determination server. [Figure 5] This flowchart shows another example of the update determination process of the update determination server. [Modes for carrying out the invention]

[0011] The embodiments of this disclosure will be described below with reference to the drawings.

[0012] Figure 1 shows a model update necessity determination system 100 according to one embodiment. The model update necessity determination system 100 determines whether or not to update a machine learning model that predicts changes in the behavior of target vehicles based on target vehicle data acquired from target vehicles within a pre-set area. A machine learning model update is necessary, for example, when the machine learning model's suitability to the situation within the area is low. The machine learning model is a trained model that has been trained using vehicle data under predetermined training conditions. Details about the machine learning model, training conditions, and area will be described later.

[0013] As shown in Figure 1, in the model update necessity determination system 100, the target vehicles 2 (target vehicles 2A to 2Z) are connected to the update determination server 10 via network N in a communication-enabled manner. Network N is a wireless communication network. Any well-known wireless communication network can be used for network N.

[0014] Target vehicle 2 is a vehicle that is subject to information collection by the model update necessity determination system 100. Target vehicle 2 is assigned an ID [identification] (vehicle identification number) for vehicle identification. There may be only one target vehicle. Target vehicle 2 does not need to be a vehicle with the same configuration, and may be of a different type, etc. Target vehicle may be an autonomous vehicle, or it may be a vehicle that does not have autonomous driving capabilities. In this embodiment, target vehicle 2 is a vehicle that receives support such as information provision from a machine learning model. Note that the vehicle that receives support may or may not be included in target vehicle 2 for information collection.

[0015] The target vehicle data collected by the model update necessity determination system 100 includes location information of target vehicle 2. Location information is generated in association with time. The target vehicle data may include an ID. The target vehicle data may include vehicle speed information of target vehicle 2, acceleration information, steering angle information, and yaw rate information. The target vehicle data may include object detection information regarding objects around target vehicle 2 detected by the sensors of target vehicle 2, and information on the driving position relative to the lane. The target vehicle data may include information on the driver's driving operations. The object detection information may include location information and object type information of objects as seen from target vehicle 2. The object type information includes information on the type of object, such as pedestrians, bicycles, motorcycles, cars, walls, and other stationary objects.

[0016] Furthermore, the target vehicle data may include wiper operation information of target vehicle 2. The target vehicle data may also include road surface friction information calculated by the onboard equipment of target vehicle 2, and outside temperature information detected by the temperature sensor of target vehicle 2. The target vehicle data may also include images captured by the external camera of target vehicle 2. All of the above-mentioned information is associated with the location information and time of target vehicle 2.

[0017] The machine learning model is a neural network such as a convolutional neural network (CNN). The neural network can include multiple layers including a plurality of convolutional layers and pooling layers. As the neural network, a deep learning network based on deep learning may be used.

[0018] The machine learning model takes the target vehicle data collected from the target vehicle 2 as input and outputs a prediction result of the behavior change of the target vehicle 2. The behavior change of the target vehicle 2 is, for example, the occurrence of unstable behavior. Unstable behavior is a sudden behavior change that makes the driving of the target vehicle 2 unstable. Unstable behavior includes slip. Unstable behavior may include sudden deceleration or sudden steering angle change. Unstable behavior may include lane departure of the target vehicle 2 and may also include excessive approach to an object by the target vehicle 2 (such as a collision warning).

[0019] The machine learning model outputs, for example, a prediction result (reproducibility determination result of unstable behavior) of unstable behavior in the target vehicle 2 based on the target vehicle data. The machine learning model makes a reproducibility determination by extracting a feature quantity from the target vehicle data of the target vehicle 2 in which unstable behavior has been detected. As the feature quantity, any parameter can be used from the vehicle speed, acceleration, tire air pressure drop, weight, vehicle size (vehicle height, vehicle width), vehicle type, driving tendency of the driver, etc. of the target vehicle 2. The tire air pressure drop is, for example, a state where the air pressure of the tire is less than a predetermined threshold value. The driving tendency of the driver can be obtained from the past driving operation information of the driver. For the determination of the driving tendency of the driver, the number of years of driving experience, age, gender, and other information of the driver may be used. As additional feature quantities, the time when unstable behavior occurs in the target vehicle 2, the outside air temperature, the road condition, etc. may be used. The machine learning model predicts unstable behavior by comparing the feature quantity extracted from the target vehicle data of the target vehicle 2 in which unstable behavior has been detected with the feature quantity extracted from the target vehicle data of the target vehicle 2 to be predicted. Regarding the determination of the reproducibility of unstable behavior, it is described in, for example, Japanese Patent Application No. 2022-187683 and Japanese Patent Application No. 2022-145185. The behavior change may include changes in behavior other than unstable behavior.

[0020] For the machine learning model, a model adapted to a preset area is used. Here, FIG. 2 is a diagram for explaining the area of the machine learning model. As shown in FIG. 2, the area is set by being divided into a mesh shape for each certain range, for example, on a map. The area may be set as a 10 km mesh (10 km square), may be set as a 5 km mesh, or may be set as a 3 km mesh. It may also be set with a mesh of 10 km or more. The areas do not necessarily have to be of the same shape and may be set as areas where the environment is likely to be uniform according to the terrain. The area may be set as a mesh code. The mesh code is defined from latitude and longitude information, and a unique code is assigned to each mesh.

[0021] For the machine learning model, an appropriate model is adopted according to, for example, the presence or absence of major roads in the area, the ratio of residential areas and commercial areas in the area, etc. The machine learning model has been learned using data of vehicles under predetermined learning conditions (for example, past target vehicle data). The vehicle data used for learning is not limited to the target vehicle data of target vehicle 2, and data of probe vehicles not connected to the update determination server 10 may also be used.

[0022] The learning conditions include at least one of, for example, the range of outside air temperature, the range of position, the range of vehicle speed, the range of time, etc. Specifically, the range of outside air temperature as a learning condition means, for example, learning based on the target vehicle data when the outside air temperature is within the range of 19°C to 25°C. In this case, the target vehicle data with an outside air temperature of less than 19°C becomes data that deviates from the learning conditions (data that the machine learning model cannot handle). The range of position as a learning condition means, for example, that although there are six areas (Area A, Area B, Area C, Area D, Area E, Area F) within the area, learning was performed based on the target vehicle data of target vehicle 2 traveling in Area A, Area B, Area C, and Area D during learning. In this case, the target vehicle data of target vehicle 2 traveling in Area E or Area F becomes data that deviates from the learning conditions.

[0023] Similarly, the vehicle speed range as a learning condition means, for example, that the model was trained based on the target vehicle data of target vehicle 2 whose vehicle speed is within the range of 60 km / h or less. In this case, the target vehicle data of target vehicle 2 with a vehicle speed exceeding 60 km / h will be considered data that deviates from the learning condition. The time range as a learning condition means, for example, that the model was trained based on the target vehicle data from 6 AM to 12 PM. In this case, the target vehicle data from after 12 PM to 6 AM will be considered data that deviates from the learning condition. Learning conditions may be set by narrowing down the target vehicle data to improve prediction accuracy in an area, or they may arise due to a lack of target vehicle data at the time of training.

[0024] [Configuration of the model update necessity determination system] The configuration of the model update necessity determination system 100 according to this embodiment will be described below. The model update necessity determination system 100 shown in Figure 1 is configured to include an update determination server 10. The model update necessity determination system 100 may also be configured to include at least a portion of the in-vehicle computing devices of the target vehicles 2A to 2Z.

[0025] The update determination server 10 is installed in a facility such as an information management center and is configured to communicate with the target vehicles 2A to 2Z. Figure 3 is a block diagram showing an example of the configuration of the update determination server 10. The update determination server 10 shown in Figure 3 is configured as a general computer equipped with a processor 11, a storage unit 12, a communication unit 13, and a user interface 14.

[0026] The processor 11 controls the update determination server 10 by running an operating system, for example. The processor 11 is a arithmetic unit such as a CPU (Central Processing Unit) that includes a control unit, arithmetic unit, registers, etc. The processor 11 manages the storage unit 12, the communication unit 13, and the user interface 14. The storage unit 12 is composed of at least one of memory and storage. Memory is a recording medium such as ROM (Read Only Memory) or RAM (Random Access Memory). Storage is a recording medium such as HDD (Hard Disk Drive).

[0027] The communication unit 13 is a communication device for performing communication via the network N. The communication unit 13 can use network devices, network controllers, network cards, etc. The user interface 14 is a device that includes output devices such as displays and speakers, and input devices such as touch panels. The update determination server 10 does not necessarily have to be located in a facility, and may be mounted on a mobile device such as a vehicle or ship. The update determination server 10 may be composed of multiple servers.

[0028] The update determination server 10 is connected to a target vehicle database 15 that stores past target vehicle data for target vehicle 2. The target vehicle database 15 has a storage device such as an HDD and can be configured in the same way as a well-known database.

[0029] The target vehicle database 15 may store target vehicle data in association with a number of pre-defined areas. Areas are defined, for example, by dividing them into certain ranges on a map. Areas may be defined using, for example, a 10km mesh (10km square), a 5km mesh, or a 3km mesh. They may also be defined using meshes of 10km or more. Areas do not need to be the same shape and may be defined as regions where the environment tends to be uniform depending on the terrain. Areas may also be defined as mesh codes. Mesh codes are defined from latitude and longitude information, and each mesh is assigned a unique code.

[0030] The location information of target vehicle 2 can be obtained as latitude and longitude information from the GPS (Global Positioning System) or GNSS (Global Navigation Satellite System) installed on target vehicle 2. This also contributes to cost reduction by eliminating the need for maps. The target vehicle database 15 may be configured together with the update determination server 10, or it may be located in a facility separate from the update determination server 10.

[0031] Next, the functional configuration of the processor 11 will be described. As shown in Figure 3, the processor 11 includes a target vehicle information recognition unit 11a, an unstable behavior recognition unit 11b, a storage processing unit 11c, a deviation count calculation unit 11d, and an update necessity determination unit 11e.

[0032] The target vehicle information recognition unit 11a recognizes the target vehicle data transmitted from the target vehicle 2. The target vehicle data is as described above. The target vehicle information recognition unit 11a acquires the target vehicle data, including location information and time, through communication with the target vehicle 2. The target vehicle information recognition unit 11a aggregates the target vehicle data within the target period of the machine learning model used to determine whether an update is necessary. The target period is not particularly limited and may be one week, one month, or two months or more.

[0033] The unstable behavior recognition unit 11b detects unstable behavior of the target vehicle 2 based on the target vehicle data acquired by the target vehicle information recognition unit 11a. The unstable behavior recognition unit 11b may use the activation start conditions of a well-known antilock brake system [ABS] as the detection of slippage. For example, in an antilock brake system, as one example, it activates when a wheel that is thought to be locked is identified by comparing the wheel speed of each wheel with the estimated vehicle speed. The estimated vehicle speed may be determined from the wheel speed of each wheel up to the point of slippage, or from the change in acceleration up to the point of slippage.

[0034] Furthermore, the unstable behavior recognition unit 11b may use the activation start conditions of a well-known vehicle stability control system [VSC] or the activation start conditions of a well-known traction control system [TRC] to detect slippage. Traction control can also be activated when a wheel that is slipping is identified by comparing the wheel speed of each wheel with the estimated vehicle speed. The unstable behavior recognition unit 11b may also detect slippage of the target vehicle 2 by other well-known methods.

[0035] The unstable behavior recognition unit 11b may detect sudden deceleration as unstable behavior based on the deceleration detected by the acceleration sensor. In this case, the unstable behavior recognition unit 11b detects sudden deceleration of the target vehicle 2 when, for example, the absolute value of the deceleration degree becomes equal to or greater than the sudden deceleration threshold. The sudden deceleration threshold is a preset threshold value. Hereafter, the threshold used in the description refers to a preset threshold value.

[0036] The unstable behavior recognition unit 11b may detect a sudden change in steering angle as unstable behavior based on the yaw rate detected by the yaw rate sensor. In this case, the unstable behavior recognition unit 11b detects a sudden change in steering angle of the target vehicle 2, for example, when the yaw rate exceeds a steering angle change threshold. Note that tire steering angle or steering angle may be used instead of yaw rate.

[0037] The memory processing unit 11c stores the target vehicle data and unstable behavior data in the target vehicle database 15. The memory processing unit 11c may also store the unstable behavior data as part of the target vehicle data, associated with the identification ID of the target vehicle 2.

[0038] The deviation calculation unit 11d calculates the number of deviations in the target vehicle data from the machine learning model's learning conditions, based on the target vehicle data of the target vehicle 2 within the area and the learning conditions of the machine learning model that have been stored in advance. The deviation calculation unit 11d may calculate the number of deviations for each area, or it may calculate the number of deviations for multiple areas handled by a single machine learning model by summing them up.

[0039] The deviation calculation unit 11d adds a deviation when it detects target vehicle data where the ambient temperature is below 19°C, for example, when the ambient temperature range as a learning condition is between 19°C and 25°C. The deviation calculation unit 11d also adds a deviation when it detects target vehicle data where the vehicle speed exceeds 60 km / h, for example, when the vehicle speed range as a learning condition is 60 km / h or less.

[0040] The update necessity determination unit 11e determines whether or not the machine learning model needs to be updated (whether the machine learning model's fitness is high or low). The update necessity determination unit 11e determines, for example, whether or not a predetermined period has elapsed since the last update of the machine learning model. The predetermined period is a pre-set period. The predetermined period is not particularly limited. The predetermined period may be six months or one year. If the predetermined period has not elapsed since the last update of the machine learning model, the update necessity determination unit 11e determines that an update of the machine learning model is unnecessary.

[0041] The update necessity determination unit 11e determines whether a machine learning model needs to be updated based on the number of deviations if it determines that a predetermined period has elapsed since the last update of the machine learning model. The update necessity determination unit 11e determines whether an update is necessary for each machine learning model. For example, the update necessity determination unit 11e determines that an update of the machine learning model is necessary if the duration for which the number of deviations is equal to or greater than the update determination threshold exceeds the continuation determination threshold. The duration for which the number of deviations is equal to or greater than the update determination threshold is the duration for which the state in which the number of deviations is equal to or greater than the update determination threshold (a state in which there is a tendency to deviate) continues. The update determination threshold and the continuation determination threshold are threshold values ​​that have been set in advance. The update determination threshold and the continuation determination threshold can be set to any value from the perspective of updating the machine learning model. If the repeatedly calculated number of deviations falls below the update determination threshold even once, the update necessity determination unit 11e resets the duration count and starts counting from zero.

[0042] The deviation calculation unit 11d may also calculate the number of unstable deviations as the number of deviations, which is the number of times the target vehicle data deviates from the learning conditions when unstable behavior occurs in the target vehicle 2. The deviation calculation unit 11d can calculate the number of unstable deviations based on the target vehicle data of the target vehicle 2 in the area, the learning conditions of the machine learning model, and the unstable behavior recognized by the unstable behavior recognition unit 11b.

[0043] In this case, the deviation calculation unit 11d narrows down the target vehicle data to data where target vehicle 2 exhibits unstable behavior, and then calculates the number of deviations in the target vehicle data that deviate from the learning conditions of the machine learning model. The update necessity determination unit 11e may determine whether or not to update the machine learning model based on the number of unstable deviations. The update necessity determination unit 11e may determine that an update of the machine learning model is necessary if the duration for which the number of unstable deviations is equal to or greater than the update determination threshold is equal to or greater than the continuation determination threshold.

[0044] Furthermore, the deviation calculation unit 11d may calculate, in addition to the unstable deviation count, the normal deviation count, which is the number of target vehicle data that deviates from the learning conditions when no unstable behavior occurs. The normal deviation count can be considered a leading indicator of the unstable deviation count. The update necessity determination unit 11e may set the continuation determination threshold to a smaller value when the normal deviation count is above the early determination threshold compared to when the normal deviation count is below the early determination threshold. The early determination threshold is a threshold value that has been set in advance. The update necessity determination unit 11e determines that the machine learning model needs to be updated when the duration for which the unstable deviation count is above the update determination threshold exceeds the continuation determination threshold. As a result, the model update necessity determination system 100 can grasp changes in the area situation early and determine that the machine learning model needs to be updated at an appropriate time, taking into account the impact on the prediction accuracy of the unstable deviation count.

[0045] [Model Update Necessity Determination System's Update Necessity Determination Method] Next, the model update necessity determination method of the model update necessity determination system 100 according to this embodiment will be explained with reference to the drawings. Figure 4 is a flowchart showing an example of the update necessity determination process (update necessity determination method) of the update determination server 10. The model update necessity determination system 100 executes the update necessity determination process, for example, at regular intervals.

[0046] As shown in Figure 4, the update determination server 10 of the model update necessity determination system 100, in S10, aggregates target vehicle data within the target period of the machine learning model area for which the update necessity is determined by the target vehicle information recognition unit 11a. After that, the update determination server 10 proceeds to S11.

[0047] In S11, the update determination server 10 calculates the number of deviations using the deviation calculation unit 11d. The deviation calculation unit 11d calculates the number of deviations in which the target vehicle data deviates from the learning conditions of the machine learning model, based on the target vehicle data of the target vehicle 2 in the area and the learning conditions of the machine learning model. The deviation calculation unit 11d may also calculate the number of unstable deviations as the number of deviations in which the target vehicle data deviates from the learning conditions when unstable behavior occurs in the target vehicle 2. After that, the update determination server 10 proceeds to S12.

[0048] In S12, the update determination server 10 determines whether a predetermined period has elapsed since the machine learning model was updated using the update necessity determination unit 11e. If the update determination server 10 determines that a predetermined period has elapsed since the machine learning model was updated (S12: YES), it proceeds to S13. If the update determination server 10 determines that a predetermined period has not elapsed since the machine learning model was updated (S12: NO), it proceeds to S15.

[0049] In S13, the update determination server 10 determines, using the update necessity determination unit 11e, whether the duration for which the number of deviations is equal to or greater than the update determination threshold has exceeded the continuation determination threshold. If the update determination server 10 determines that the duration for which the number of deviations is equal to or greater than the continuation determination threshold has exceeded the continuation determination threshold (S13: YES), it proceeds to S14. If the update determination server 10 does not determine that the duration for which the number of deviations is equal to or greater than the update determination threshold has exceeded the continuation determination threshold (S13: NO), it proceeds to S15.

[0050] In S14, the update determination server 10 determines, using the update necessity determination unit 11e, that an update to the machine learning model is necessary. The update determination server 10 may also notify a predetermined recipient that an update to the machine learning model is necessary. After that, the update determination server 10 terminates the update necessity determination process.

[0051] In S15, the update determination server 10 determines, using the update necessity determination unit 11e, that updating the machine learning model is unnecessary. Subsequently, the update determination server 10 terminates the update necessity determination process.

[0052] Figure 5 is a flowchart showing another example of the update necessity determination process of the update determination server. The process in Figure 5 is assumed to be executed after the aggregation in S11 of Figure 4.

[0053] As shown in Figure 5, the update determination server 10 calculates the number of deviations using the deviation calculation unit 11d in S20. The deviation calculation unit 11d calculates the number of unstable deviations and the number of normal deviations based on the target vehicle data of the target vehicle 2 in the area, the learning conditions of the machine learning model, and the unstable behavior recognized by the unstable behavior recognition unit 11b. After that, the update determination server 10 proceeds to S21.

[0054] In S21, the update determination server 10 determines whether a predetermined period has elapsed since the machine learning model was updated using the update necessity determination unit 11e. If the update determination server 10 determines that a predetermined period has elapsed since the machine learning model was updated (S21: YES), it proceeds to S22. If the update determination server 10 determines that a predetermined period has not elapsed since the machine learning model was updated (S21: NO), it proceeds to S26.

[0055] In S22, the update determination server 10 determines whether the number of normal deviations is less than the early detection threshold using the update necessity determination unit 11e. If the update determination server 10 determines that the number of normal deviations is less than the early detection threshold (S22: YES), it proceeds to S24. If the update determination server 10 does not determine that the number of normal deviations is less than the early detection threshold (S22: NO), it proceeds to S23.

[0056] In S23, the update determination server 10 changes the continuation determination threshold to a smaller value using the update necessity determination unit 11e. The update necessity determination unit 11e changes the continuation determination threshold to a second value that is smaller than the initial value, for example. After that, the update determination server 10 proceeds to S24.

[0057] In S24, the update determination server 10 determines, using the update necessity determination unit 11e, whether the duration for which the number of unstable deviations is equal to or greater than the update determination threshold has exceeded the continuation determination threshold. If the update determination server 10 determines that the duration for which the number of unstable deviations is equal to or greater than the continuation determination threshold has exceeded the continuation determination threshold (S24: YES), it proceeds to S25. If the update determination server 10 does not determine that the duration for which the number of unstable deviations is equal to or greater than the update determination threshold has exceeded the continuation determination threshold (S24: NO), it proceeds to S26.

[0058] In S25, the update determination server 10 determines, using the update necessity determination unit 11e, that an update to the machine learning model is necessary. The update determination server 10 may also notify a predetermined recipient that an update to the machine learning model has been determined to be necessary. After that, the update determination server 10 terminates the update necessity determination process.

[0059] In S26, the update determination server 10 determines, using the update necessity determination unit 11e, that updating the machine learning model is unnecessary. Subsequently, the update determination server 10 terminates the update necessity determination process.

[0060] According to the model update necessity determination system 100 (and the model update necessity determination method for the model update necessity determination system 100) described above, by taking into account the number of deviations in the target vehicle data based on the learning conditions of the machine learning model, it is possible to appropriately determine whether or not to update the machine learning model in response to changes in the situation within the area.

[0061] Furthermore, the model update necessity determination system 100 may use the number of unstable deviations as the number of deviations. In this case, the model update necessity determination system 100 can determine whether or not to update the machine learning model, taking into account the impact on the prediction results (reproducibility determination results) of the unstable behavior of the target vehicle 2 by the machine learning model.

[0062] Furthermore, in the model update necessity determination system 100, if the number of normal deviations is equal to or greater than the early determination threshold, the continuation determination threshold may be set to a smaller value compared to when the number of normal deviations is less than the early determination threshold. In this case, the model update necessity determination system 100 can grasp the changes in the area's situation early and determine that an update of the machine learning model is necessary at an appropriate time, taking into account the impact on the prediction accuracy of the number of unstable deviations.

[0063] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above. The present invention can be implemented in various forms, starting with the embodiments described above, by making various changes and improvements based on the knowledge of those skilled in the art.

[0064] The update determination server 10 of the model update necessity determination system 100 does not necessarily need to be directly connected to the target vehicle 2 by communication. The update determination server 10 may acquire target vehicle data via another server. In this case, the update determination server 10 does not need to have a target vehicle information recognition unit 11a.

[0065] Similarly, the update determination server 10 may acquire unstable behavior data from another server. In this case, the update determination server 10 does not need to have an unstable behavior recognition unit 11b.

[0066] The model update necessity determination system 100 may use the number of consecutive occurrences instead of the duration for which the number of deviations is equal to or greater than the update determination threshold. If the number of deviations is calculated at regular intervals, the number of consecutive occurrences for which the number of deviations has been determined to be equal to or greater than the update determination threshold can be used instead of the duration for which they are equal to or greater than the update determination threshold.

[0067] The update necessity determination unit 11e does not necessarily need to determine whether a predetermined period has elapsed since the update of the machine learning model. Furthermore, the update necessity determination unit 11e does not necessarily need to use the duration during which the number of deviations exceeds the update determination threshold in its determination. If the number of deviations exceeds the mandatory update threshold, the update necessity determination unit 11e may determine that an update of the machine learning model is necessary without counting the duration. The mandatory update threshold is a pre-set threshold value. The mandatory update threshold can be set to a value greater than the update determination threshold if the duration during which the number of deviations exceeds the update determination threshold is also used in the determination. [Explanation of symbols]

[0068] 2...Target vehicle, 10...Update determination server, 11d...Deviation count calculation unit, 11e...Update necessity determination unit, 100...Model update necessity determination system.

Claims

1. A model update necessity determination system that determines whether or not a machine learning model needs to be updated, which is trained using data from vehicles that meet predetermined training conditions and predicts changes in the behavior of a target vehicle based on target vehicle data acquired from target vehicles within a pre-set area, The aforementioned target vehicle data includes at least one of the following: location information, vehicle speed information, and outside temperature information. The learning conditions include at least one of the following: a range of location, a range of vehicle speed, and a range of outside temperature, which correspond to the information contained in the target vehicle data. A deviation calculation unit calculates the number of deviations in the target vehicle data that deviates from the learning conditions when the value of the information contained in the target vehicle data falls outside the range of the corresponding learning conditions, based on the target vehicle data of the target vehicle in the area and the learning conditions. A model update necessity determination system comprising: an update necessity determination unit that determines whether or not the machine learning model needs to be updated based on the number of deviations.

2. The model update necessity determination system according to claim 1, wherein the update necessity determination unit determines that the machine learning model needs to be updated when the duration for which the number of deviations is equal to or greater than the update determination threshold exceeds the duration for which the deviations are equal to or greater than the continuation determination threshold.

3. The machine learning model predicts the occurrence of unstable behavior, which is a sudden change in the behavior of the target vehicle. The model update necessity determination system according to claim 1 or 2, wherein the deviation number calculation unit calculates the number of unstable deviations in the target vehicle data when the unstable behavior of the target vehicle occurs, where the data deviates from the learning conditions, as the deviation number.

4. The machine learning model predicts the occurrence of unstable behavior, which is a sudden change in the behavior of the target vehicle. The deviation calculation unit calculates the deviation number as the number of unstable deviations, which is the number of deviations in which the target vehicle data deviates from the learning conditions when the unstable behavior of the target vehicle occurs, and the number of normal deviations, which is the number of deviations in which the target vehicle data deviates from the learning conditions when the unstable behavior does not occur. The update necessity determination unit determines that an update of the machine learning model is necessary when the duration for which the number of unstable deviations is equal to or greater than the update determination threshold exceeds the continuation determination threshold. The model update necessity determination system according to claim 1, wherein when the number of deviations from normal conditions is equal to or greater than the early determination threshold, the continuation determination threshold is set to a smaller value compared to when the number of deviations from normal conditions is less than the early determination threshold.

5. A method for determining whether a machine learning model needs to be updated, which is trained using data from vehicles that meet predetermined training conditions and determines whether a machine learning model needs to be updated to predict changes in the behavior of a target vehicle based on target vehicle data acquired from target vehicles within a pre-set area, wherein the model is trained using data from vehicles that meet predetermined training conditions and the model determines whether a machine learning model needs to be updated to predict changes in the behavior of a target vehicle based on target vehicle data acquired from target vehicles within a pre-set area, The aforementioned target vehicle data includes at least one of the following: location information, vehicle speed information, and outside temperature information. The learning conditions include at least one of the following: a range of location, a range of vehicle speed, and a range of outside temperature, which correspond to the information contained in the target vehicle data. Based on the target vehicle data of the target vehicle within the area, if the value of the information included in the target vehicle data falls outside the range of the corresponding learning condition, the number of deviations in the target vehicle data from the learning condition is calculated. A method for determining whether a model needs to be updated, which determines whether the machine learning model needs to be updated based on the number of deviations.

Images