Data selection device and data selection program

Abstract

To select various data having different properties as a learning object.SOLUTION: A data selection device 1 that selects data used for training of a machine learning model from a data set including a plurality of pieces of target data includes a feature amount calculation unit 33 that calculates a target data feature amount of the target data from one piece of target data in the data set using a feature amount calculation model, and a selection unit 35 that determines whether or not to select the target data as data used for training of the machine learning model based on a similarity between an individual feature amount of the target data including the target data feature amount and an individual feature amount of another piece of target data in the data set or an individual feature amount of data already selected for training of the machine learning model.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to a data selection device and a data selection program. [Background technology]

[0002] Various machine learning models have been known for some time (Patent Documents 1 to 3). In particular, Patent Document 1 discloses that data used for learning a machine learning model is selected based on the results when each piece of data is input into the machine learning model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2022-550094 [Patent Document 2] Japanese Patent Publication No. 2023-085353 [Patent Document 3] Japanese Patent Publication No. 2022-056611 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, when training a machine learning model, using data with different properties allows for more appropriate training. This is because training using a wide range of data with different properties allows for the generation of a machine learning model that can adapt to a variety of different situations. On the other hand, using data with similar properties not only reduces the learning effect, unnecessarily lengthens the learning processing time, and unnecessarily increases the annotation workload. However, the method of Patent Document 1 does not necessarily allow for the selection of various data with different properties as data to be used for training the machine learning model.

[0005] In view of the above problems, an object of the present disclosure is to enable the selection of various data with different properties as learning targets. [Means for solving the problem]

[0006] The gist of the present disclosure is as follows.

[0007] (1) A data selection device that selects data to be used for training a machine learning model from a dataset including a plurality of target data, a feature calculation unit that calculates target data feature values ​​of one target data item in the data set using a feature calculation model; a selection unit that determines whether to select the target data as data to be used for training the machine learning model based on the similarity between individual features of the target data including the target data features and individual features of other target data in the dataset or individual features of data already selected for training the machine learning model. (2) a situation data acquisition unit that acquires situation data relating to the situation when each target data was generated; The data selection device described in (1) above further comprises an individual feature calculation unit that calculates individual features by combining the target data features calculated by the feature calculation unit for each target data with the features of the situation data acquired by the situation data acquisition unit. (3) The data selection device described in (1) or (2) above, wherein the selection unit selects the target data as data to be used for training the machine learning model when the similarity between the individual features of data already selected for training the machine learning model and the individual features of the target data is equal to or less than a predetermined value. (4) A data selection device described in any one of (1) to (3) above, wherein the selection unit selects the target data as data to be used for training the machine learning model when the similarity between the individual features of data already used for training the machine learning model and the individual features of the target data is equal to or less than a predetermined value. (5) A data selection program that selects data to be used for training a machine learning model from a dataset containing multiple target data, calculating a target data feature of one target data from the data set using a feature calculation model; determining whether to select the target data as data to be used for training the machine learning model based on the similarity between individual features of the target data including the target data features and individual features of other target data in the dataset or individual features of data already selected for training the machine learning model; A data selection program that causes a computer to execute the above. [Effects of the Invention]

[0008] According to the present disclosure, it becomes possible to select various data with different properties as learning targets. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a vehicle that transmits image data for constructing a data set. [Figure 2] FIG. 2 is a diagram illustrating a configuration of a data selection device according to an embodiment. [Figure 3] FIG. 3 is a diagram showing how the target data feature vector and the situation data feature vector are combined. [Figure 4] FIG. 4 is a diagram showing a schematic diagram of a vector space of individual feature vectors. [Figure 5] FIG. 5 is a flowchart showing the flow of a data selection process executed in a data selection device according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, the embodiments will be described in detail with reference to the drawings. In the following description, like components are designated by like reference numerals.

[0011] First, a data selection device 1 according to one embodiment will be described with reference to Figures 1 and 2. The data selection device 1 selects data to be used for training a machine learning model from a data set including multiple pieces of target data.

[0012] Before describing the data selection device 1, we will first describe the machine learning model and the dataset. In this embodiment, when image data is input, the machine learning model outputs information related to the image data. For example, when image data is input, the machine learning model outputs a prediction result regarding an object included in an image represented by the image data.

[0013] In particular, in this embodiment, image data of the area ahead of the vehicle captured by an external camera attached to the vehicle is input to the machine learning model. The machine learning model then outputs prediction results such as the positions and types of objects (e.g., surrounding vehicles, pedestrians, roads, lane markings, signs, obstacles on the road, etc.) included in the image represented by the input image data. Note that the machine learning model may be any model that, when input with any data (not limited to image data), outputs information related to the data.

[0014] The data set includes image data transmitted from a plurality of vehicles 100. Figure 1 is a diagram showing a schematic configuration of a vehicle 100 that transmits each image data constituting the data set.

[0015] 1, vehicle 100 has an exterior camera 111, a driving environment sensor 112, a driving condition sensor 113, an exterior communication module 114, and an ECU 115. These are connected to each other so as to be able to communicate with each other via an in-vehicle network 116 that complies with standards such as CAN (Controller Area Network).

[0016] The exterior camera 111 is a camera that captures an image of the area ahead of the vehicle 100. The exterior camera 111 captures an image of the area ahead of the vehicle 100 at predetermined imaging intervals and generates image data.

[0017] The driving environment sensor 112 is a sensor that detects the driving environment of the vehicle 100. The driving environment sensor 112 detects the driving environment, such as the driving position of the vehicle 100, the weather during driving, and the driving time. Specifically, the driving environment sensor 112 includes, for example, a sensor (for example, a GNSS receiver) that measures the self-position of the vehicle 100, a rain sensor that determines whether it is raining, and a time sensor that detects the current time.

[0018] The running condition sensor 113 is a sensor that detects the running condition of the vehicle 100. The running condition sensor 113 detects running conditions such as the speed and acceleration of the vehicle 100, and the rate of change of the yaw angle (yaw rate) when the vehicle 100 turns. Specifically, the running condition sensor 113 includes, for example, a speed sensor that detects the speed of the vehicle 100, an acceleration sensor that detects the acceleration of the vehicle 100, and a yaw sensor that detects the yaw rate of the vehicle 100.

[0019] When certain image data is generated by the outside-vehicle camera 111, the driving environment detected by the driving environment sensor 112 and the driving state detected by the driving state sensor 113 are data relating to the situation when this image data was generated. Therefore, in this specification, such data is referred to as situation data when the image data was generated by the outside-vehicle camera 111.

[0020] The exterior communication module 114 communicates with devices outside the vehicle. The exterior communication module 114 is a device that performs wireless communication with a wireless base station in accordance with a predetermined mobile communication standard. The exterior communication module 114 transmits, to the data selection device 1, image data generated by the exterior camera 111 and situation data when each image data item was generated.

[0021] The ECU 115 controls the transmission of data from the exterior communication module 114. The ECU 115 stores data (image data and situation data) detected by the exterior camera 111, the driving environment sensor 112, and the driving condition sensor 113. In addition, the ECU 115 transmits the stored data to the data selection device 1 via the exterior communication module 114 at any timing.

[0022] Next, the configuration of the data selection device 1 will be described with reference to Fig. 2. Fig. 2 is a diagram showing the configuration of the data selection device 1 according to one embodiment.

[0023] 2, the data selection device 1 includes a communication interface 10, a storage unit 20, and a processor 30. The communication interface 10, the storage unit 20, and the processor 30 may be separate circuits, or may be configured as a single integrated circuit.

[0024] The communication interface 10 is an interface circuit for connecting the data selection device 1 to a device external to the data selection device 1. The data selection device 1 transmits and receives data to and from the external device via the communication interface 10. The external device includes, for example, an exterior communication module 114 of any vehicle 100. The external device also includes a learning device that trains a machine learning model. In addition, the external device may include a user input device (e.g., a keyboard, a mouse, etc.) and an output device to the user (e.g., a display, a speaker, etc.). In this embodiment, the communication interface 10 receives data (image data and situation data) stored in the ECU 115 of the vehicle 100 via the exterior communication module 114 from the ECU 115 and stores the data in the storage unit 20. The communication interface 10 also transmits data of learning images selected by the data selection device 1 and subsequently annotated to the learning device.

[0025] The storage unit 20 is a non-transitory storage medium that stores data. The storage unit 20 includes, for example, at least one of a volatile semiconductor memory, a non-volatile semiconductor memory, a hard disk drive (HDD), and a solid-state drive (SSD). The storage unit 20 stores a computer program executed by the processor 30, particularly a data selection program for executing a data selection process. In addition, the storage unit 20 stores data of learning images that have been selected and annotated by the processor 30.

[0026] The storage unit 20 also stores data used in the computer program executed by the processor 30, such as data received from the vehicle 100 via the communication interface 10. In this embodiment, the storage unit 20 stores multiple pieces of data (image data and situation data) received from multiple vehicles 100 as one data set. Therefore, one data set includes multiple pieces of image data and situation data.

[0027] The processor 30 has one or more central processing units (CPUs) and their peripheral circuits. The processor 30 may further have other arithmetic circuits such as a logic unit or a numerical operation unit. The processor 30 executes a computer program stored in the storage unit 20. In particular, in this embodiment, the processor 30 executes a data selection program stored in the storage unit 20.

[0028] 2, the processor 30 includes a target data acquisition unit 31, a situation data acquisition unit 32, a feature calculation unit 33, an individual feature calculation unit 34, a selection unit 35, and an annotation unit 36. Each of these units included in the processor 30 is, for example, a functional module realized by a computer program running on the processor 30. Alternatively, each of the units included in the processor 30 may be implemented in the data selection device 1 as an independent integrated circuit, microprocessor, or firmware.

[0029] The target data acquisition unit 31 acquires target data to be selected as data to be used in training the machine learning model from the storage unit 20. In this embodiment, since image data is input to the machine learning model, the target data acquisition unit 31 acquires image data to be selected from the data set stored in the storage unit 20.

[0030] The situation data acquisition unit 32 acquires situation data relating to the situation when the target data acquired by the target data acquisition unit 31 was generated from the storage unit 20. The situation data acquisition unit 32 acquires situation data corresponding to the target data acquired by the target data acquisition unit 31 from one data set stored in the storage unit 20. In this embodiment, as described above, the situation data includes data relating to the driving environment and driving state of the vehicle 100 when the image data was generated by the exterior camera 111.

[0031] The feature calculation unit 33 calculates the target data feature of one piece of target data in the data set using the feature calculation model. In this embodiment, the feature calculation unit 33 calculates the target data feature of the image data by inputting one piece of image data to the feature calculation model. The target data feature is, for example, a feature vector (target data feature vector) that represents the feature of the target data.

[0032] The feature calculation model for target data is, for example, an encoder that receives input of one piece of image data and outputs a target data feature vector. The feature calculation model for target data is, for example, a convolutional neural network (CNN) that is pre-trained to receive input of one piece of image data and output a target data feature vector. The feature calculation model for target data is configured to output similar feature vectors as the images represented by the image data become more similar. Therefore, for example, the feature calculation model for target data outputs similar feature vectors (vectors whose feature vectors are close to each other) for image data representing an image of an urban intersection and image data representing an image of another urban intersection. On the other hand, the feature calculation model for target data outputs different feature vectors (vectors whose feature vectors are far from each other) for image data representing an image of an urban intersection and image data representing an image of a rural road.

[0033] Additionally, the feature calculation unit 33 may calculate a situation data feature of one piece of situation data in the data set using a feature calculation model. In this embodiment, the feature calculation unit 33 inputs one piece of situation data (data representing the driving environment and driving state when one piece of image data was generated) into a feature calculation model for situation data, thereby calculating a feature for the situation data (situation data feature). The situation data feature is, for example, a feature vector (situation data feature vector) representing the feature of the situation data.

[0034] The feature quantity calculation model for situation data is, for example, an encoder that receives one piece of situation data and outputs a situation data feature vector. The feature quantity calculation model for situation data is, for example, a deep neural network (DNN) that is pre-trained to receive one piece of situation data and output a situation data feature vector. The feature quantity calculation model for situation data is configured to output similar feature vectors as situations represented by situation data become more similar. Therefore, the feature quantity calculation model for situation data outputs similar feature vectors (vectors with a close distance between feature vectors) for, for example, situation data obtained when a vehicle travels through an urban intersection on a sunny daytime and situation data obtained when a vehicle travels through another urban intersection on a sunny daytime. On the other hand, the feature quantity calculation model for situation data outputs different feature vectors (vectors with a large distance between feature vectors) for, for example, situation data obtained when a vehicle travels through an urban intersection on a sunny daytime and situation data obtained when a vehicle travels on a rural road on a rainy evening.

[0035] In this embodiment, the feature calculation unit 33 uses a feature calculation model for situation data when calculating situation data feature quantities from situation data. However, the situation data feature quantities may be substantially the same as the situation data, or may be normalized values ​​of parameters included in the situation data. In this case, the feature calculation unit 33 uses, for example, the situation data as the situation data feature quantities without using the feature calculation model.

[0036] The individual feature amount calculation unit 34 calculates individual feature amounts by combining the target data feature amounts calculated for each target data by the feature amount calculation unit 33 with the feature amounts of the situation data acquired by the situation data acquisition unit. In this embodiment, the individual feature amount calculation unit 34 calculates individual feature vectors by combining the target data feature vectors and the situation data feature vectors.

[0037] Fig. 3 is a diagram showing how a target data feature vector and a situation data feature vector are combined. In the example shown in Fig. 3, the target data feature vector X is a 500-dimensional vector, and the situation data feature vector Y is a 50-dimensional vector. The individual feature amount calculation unit 34 combines the target data feature vector X and the situation data feature vector Y to calculate a 550-dimensional individual feature vector Z.

[0038] In this embodiment, the individual feature is calculated by combining the target data feature and the situation data feature. However, the individual feature may be a feature unrelated to the situation data feature. In this case, the individual feature is, for example, the same feature as the target data feature. In any case, the individual feature for the target data includes the target data feature.

[0039] The selection unit 35 determines whether to select each piece of target data as data to be used for training the machine learning model based on the similarity between the individual features of the target data and the individual features of target data already selected for training the machine learning model. In this embodiment, the selection unit 35 selects the target data as data to be used for training the machine learning model if the similarity between the individual features of the target data and the individual features of target data already selected for training the machine learning model is equal to or less than a reference value. The target data already selected for training the machine learning model may include data already used in training the machine learning model. Therefore, the selection unit 35 may select the target data as data to be used for training the machine learning model if the similarity between the individual features of the target data and the individual features of data already used in training the machine learning model is equal to or less than a reference value.

[0040] In particular, in this embodiment, the selection unit 35 determines the similarity between individual feature amounts based on the distance between individual feature vectors. Therefore, the selection unit 35 determines that the similarity is high when the distance between individual feature vectors is short, and that the similarity is low when the distance between individual feature vectors is long. Any distance, such as Euclidean distance or Manhattan distance, can be used as the distance between individual feature vectors.

[0041] FIG. 4 is a diagram schematically illustrating a vector space of individual feature vectors. Although individual feature vectors are multidimensional vectors, they are conceptually represented as two-dimensional vectors in a two-dimensional vector space. The white circle V1 in FIG. 4 represents the individual feature vectors of target data that have already been selected for training the machine learning model. Furthermore, the dashed circle C in FIG. 4 indicates the range within which the distance from the individual feature vectors of each target data that has already been selected is a predetermined reference distance.

[0042] In this embodiment, if the distance between the individual feature vector of any target data and the individual feature vector of already selected target data is equal to or greater than a predetermined reference distance, the selection unit 35 newly selects the target data as data to be used in training the machine learning model. Therefore, if the individual feature vector of any target data is located outside all of the circles C in Figure 4, i.e., if it is a vector represented by V2 in Figure 4, the target data is newly selected as data to be used in training the machine learning model. On the other hand, if the individual feature vector of any target data is located within any of the circles C in Figure 4, i.e., if it is a vector represented by V3 in Figure 4, the target data is not selected as data to be used in training the machine learning model.

[0043] As a result, according to this embodiment, data with different properties from the already selected target data is newly selected as data to be used in training the machine learning model. Therefore, various data with different properties can be selected as learning targets. As a result, it becomes possible to appropriately train the machine learning model using data with different properties. Furthermore, an unnecessary increase in the number of learning targets, which would unnecessarily lengthen the processing time for training the machine learning model, is prevented, and an unnecessary increase in the data to be annotated is prevented.

[0044] Furthermore, in this embodiment, similarity is determined taking into consideration not only feature quantities obtained from the target data (image data) but also feature quantities related to the circumstances under which the target data was obtained. Therefore, for example, when image data are similar but the circumstances under which the image data were obtained are different, these image data are selected as data to be used for training the machine learning model. As a result, data is selected based not only on the similarity of the images themselves but also on the similarity of the circumstances under which the images were obtained, allowing for more appropriate training of the machine learning model.

[0045] The selection unit 35 may determine whether to select each piece of target data as data to be used for training the machine learning model based on the similarity between the individual feature of the piece of target data and the individual feature of other piece of target data in the dataset. In particular, the selection unit 35 selects the target data as data to be used for training the machine learning model when the similarity between the individual feature of the piece of target data and the individual feature of other piece of target data in the dataset is equal to or less than a reference value. In this case, the selection unit 35 may use any search method, such as approximate nearest neighbor search.

[0046] The annotation unit 36 ​​allows a user to annotate the image data selected by the selection unit 35. For example, if the machine learning model is a model that, when image data is input, outputs the type of object included in the image represented by the image data, the annotation unit 36 ​​displays the image on an output device such as a display and allows a user to input the type of object in the image via an input device. The annotation unit 36 ​​stores the image data and the input correct answer data together in the storage unit 20 as training data.

[0047] 5 is a flowchart showing the flow of the data selection process executed in the data selection device 1 according to one embodiment. The data selection process shown in FIG.

[0048] 5, when the data selection process starts, first, the target data acquisition unit 31 acquires one image data to be selected as data to be used for training the machine learning model from the storage unit 20 (step S11). Next, the feature calculation unit 33 calculates target data features from the image data acquired by the target data acquisition unit 31 using a feature calculation model for the target data (step S12).

[0049] Next, the situation data acquisition unit 32 acquires situation data relating to the situation when the image data acquired by the target data acquisition unit 31 was generated from the storage unit 20 (step S13). At this time, the feature calculation unit 33 may calculate situation data feature amounts from the situation data acquired by the situation data acquisition unit using a feature calculation model for situation data.

[0050] Next, the individual feature calculation unit 34 calculates individual features by combining the target data features calculated in step S12 and the situation data features calculated based on the situation data acquired in step S13 (step S14).

[0051] Next, the selection unit 35 calculates the distance between the individual feature vector calculated in step S14 and the individual feature vectors of image data already selected for training the machine learning model (i.e., the similarity between the individual feature amounts) (step S15). If multiple image data have already been selected for training the machine learning model, the selection unit 35 calculates the distance between the individual feature vectors of all the image data already selected.

[0052] Next, the selection unit 35 determines whether the shortest distance among the distances calculated in step S15 is equal to or greater than a reference distance (whether the similarity is equal to or less than a reference value) (step S16). If it is determined in step S16 that the distance between the individual feature vectors is equal to or greater than the reference distance, the selection unit 35 selects the image data acquired in step S11 as training image data (step S17). Note that the selection unit 35 selects the image data acquired in step S11 as training image data even if no image data has already been selected for training the machine learning model.

[0053] On the other hand, if it is determined in step S16 that the distance between the individual feature vectors is less than the reference distance, the selection unit 35 does not select the image data acquired in step S11 as training image data (step S18).

[0054] When it is determined in step S17 or S18 whether image data can be selected, the selection unit 35 determines whether a predetermined number of image data have been selected (step S19). The predetermined number is, for example, a number required for sufficient learning of the machine learning model. If it is determined in step S19 that the predetermined number of image data have not been selected, steps S11 to S18 are repeated. On the other hand, if it is determined in step S19 that the predetermined number of image data have been selected, the data selection process is terminated. Note that the selection unit 35 may also terminate the data selection process when it has completed determining whether image data can be selected for all image data included in the dataset.

[0055] Although preferred embodiments according to the present disclosure have been described above, the present disclosure is not limited to these embodiments, and various modifications and changes can be made within the scope of the claims. [Explanation of symbols]

[0056] 1...data selection device, 10...communication interface, 20...storage unit, 30...processor, 31...target data acquisition unit, 32...situation data acquisition unit, 33...feature amount calculation unit, 34...individual feature amount calculation unit, 35...selection unit, 36...annotation unit

Claims

1. A data selection device that selects data to be used for training a machine learning model from a dataset including a plurality of target data, a feature calculation unit that calculates target data feature values ​​of one target data item in the data set using a feature calculation model; a selection unit that determines whether to select the target data as data to be used for training the machine learning model based on the similarity between individual features of the target data including the target data features and individual features of other target data in the dataset or individual features of data already selected for training the machine learning model.

2. a situation data acquisition unit that acquires situation data relating to the situation when each piece of target data was generated; 2. The data selection device according to claim 1, further comprising: an individual feature calculation unit that calculates individual features by combining the target data features calculated by the feature calculation unit for each target data with features of situation data acquired by the situation data acquisition unit.

3. 3. The data selection device according to claim 1, wherein the selection unit selects the target data as data to be used for training the machine learning model when a similarity between individual features of data already selected for training the machine learning model and individual features of the target data is equal to or less than a predetermined value.

4. 3. The data selection device according to claim 1, wherein the selection unit selects the target data as data to be used in training the machine learning model when a similarity between individual features of data already used in training the machine learning model and individual features of the target data is equal to or less than a predetermined value.

5. A data selection program that selects data to be used for training a machine learning model from a dataset including multiple target data, calculating a target data feature of one target data from the data set using a feature calculation model; determining whether to select the target data as data to be used for training the machine learning model based on the similarity between individual features of the target data including the target data features and individual features of other target data in the dataset or individual features of data already selected for training the machine learning model; A data selection program that causes a computer to execute the above.

Images