Perceptual model updating device, perceptual model updating method, and computer program for perceptual model updating

Abstract

To provide a perceptual model update device capable of appropriately updating a perceptual model.SOLUTION: A perceptual model update device inputs each of a plurality of pieces of target data and modified data obtained by making a predetermined modification to the target data into a perceptual model to predict a label corresponding to the target data and calculate an uncertainty score and an inconsistency score, classify the plurality of pieces of target data into first data, second data, and third data based on the uncertainty score and the inconsistency score, give the first data an input label inputted via an input device as a label of the first data, give the second data a label by a predetermined labeling model, and update the perceptual model by training the perceptual model with the first data and the second data given the labels as well as the third data.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to a perceptual model updating device, a perceptual model updating method, and a computer program for updating a perceptual model that trains a perceptual model trained to predict a label corresponding to input data. [Background technology]

[0002] A perception model can predict labels appropriately for input data by learning using appropriate training data. Training data is created by annotating target data extracted from a large amount of original data with correct labels. Manual annotation work by workers requires a lot of time and effort.

[0003] By including as much data as possible that is effective for learning a perception model in the target data extracted from the original data to perform annotation and use as training data, the time and workload required for creating training data can be reduced. Data for which the perception model cannot predict the label with confidence is an example of data that is effective for learning a perception model. Patent Document 1 describes an active learning service that extracts target data to be labeled from a large amount of original data using criteria such as randomness, uncertainty, and representativeness, and updates the perception model using the extracted target data. Uncertainty is a measure that indicates the uncertainty of the label predicted by the perception model. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Pat. No. 1,104,8979 Summary of the Invention [Problem to be solved by the invention]

[0005] If target data is extracted from the original data randomly or based on representativeness, the target data will contain a certain proportion of data that is not effective for training a perceptual model, making it impossible to reduce the time and effort required to create training data.

[0006] When target data is extracted from the original data based on the uncertainty, it is possible to ensure that the target data contains as much data as possible that is effective for training a perception model, thereby reducing the time and workload required to create training data. However, the target data extracted from the original data based on the uncertainty contains many outliers in the entire original data. If a perception model is trained using such target data, the perception model may overlearn the outliers and be unable to properly predict the labels corresponding to the input data.

[0007] An object of the present disclosure is to provide a perceptual model updating device that appropriately updates a perceptual model. [Means for solving the problem]

[0008] The gist of the present disclosure is as follows.

[0009] (1) a model calculation unit that inputs each of a plurality of target data and modified data obtained by adding a predetermined change to the target data into a perceptual model trained to predict a label corresponding to input data and output an uncertainty score indicating the uncertainty of the prediction, thereby predicting the label corresponding to the target data and calculating the uncertainty score corresponding to the target data and an inconsistency score indicating the magnitude of difference between the label predicted for the target data and the label predicted for the modified data; a classification unit that classifies, among the plurality of target data, the target data whose uncertainty score is higher than a predetermined uncertainty threshold and whose inconsistency score is higher than a predetermined inconsistency threshold into first data, the target data whose uncertainty score is lower than the uncertainty threshold and whose inconsistency score is higher than the inconsistency threshold into second data, and the target data whose uncertainty score is lower than the uncertainty threshold and whose inconsistency score is lower than the inconsistency threshold into third data; an assignment unit that assigns an input label inputted via an input device to the first data as a label of the first data, and assigns the label to the second data according to a predetermined label assignment model for assigning the label to input data; an update unit that updates the perceptual model by learning the perceptual model using the first data to which the label has been assigned, the second data to which the label has been assigned, and the third data; A perceptual model updating device comprising:

[0010] (2) The perceptual model updating device described in (1) above, further comprising an extraction unit that extracts the target data from the plurality of data with a predetermined probability for each class into which the plurality of data are classified based on the situation in which the plurality of data are acquired.

[0011] (3) A perceptual model updating device as described in (1) or (2) above, wherein each of the multiple target data is image data having multiple pixels, and the perceptual model predicts the label for each of the multiple pixels in the image data.

[0012] (4) The perceptual model updating device according to (3) above, wherein the model calculation unit sets as the changed data data obtained by horizontally inverting the positions of each pixel in the image data.

[0013] (5) A perceptual model updating device updates a trained perceptual model to predict a label corresponding to input data and output an uncertainty score indicating the uncertainty of the prediction, inputting each of a plurality of target data and modified data obtained by applying a predetermined modification to the target data into the perceptual model, thereby predicting the label corresponding to the target data, and calculating the uncertainty score corresponding to the target data and an inconsistency score indicating a magnitude of difference between the label predicted for the target data and the label predicted for the modified data; Among the plurality of target data, the target data whose uncertainty score is higher than a predetermined uncertainty threshold and whose inconsistency score is higher than a predetermined inconsistency threshold are classified as first data, the target data whose uncertainty score is lower than the uncertainty threshold and whose inconsistency score is higher than the inconsistency threshold are classified as second data, and the target data whose uncertainty score is lower than the uncertainty threshold and whose inconsistency score is lower than the inconsistency threshold are classified as third data; assigning an input label inputted via an input device to the first data as a label of the first data; assigning the label to the second data using a predetermined labeling model that assigns the label to input data; updating the perceptual model by training the perceptual model using the first data to which the labels have been manually assigned, the second data to which the labels have been assigned by the labeling model, and the third data; A method for updating a perceptual model comprising:

[0014] (6) inputting each of a plurality of target data and modified data obtained by adding a predetermined change to the target data into a perceptual model trained to predict a label corresponding to input data and output an uncertainty score indicating the uncertainty of the prediction, thereby predicting the label corresponding to the target data, and calculating the uncertainty score corresponding to the target data and an inconsistency score indicating the magnitude of difference between the label predicted for the target data and the label predicted for the modified data; classifying, among the plurality of target data, the target data whose uncertainty score is higher than a predetermined uncertainty threshold and whose inconsistency score is higher than a predetermined inconsistency threshold into first data, the target data whose uncertainty score is lower than the uncertainty threshold and whose inconsistency score is higher than the inconsistency threshold into second data, and the target data whose uncertainty score is lower than the uncertainty threshold and whose inconsistency score is lower than the inconsistency threshold into third data; assigning an input label inputted via an input device to the first data as a label of the first data; assigning the label to the second data using a predetermined labeling model that assigns the label to input data; updating the perceptual model by training the perceptual model using the first data that has been manually labeled, the second data that has been labeled by the labeling model, and the third data; A computer program for updating a perception model that causes a computer to execute the above.

[0015] According to the perceptual model updating device according to the present disclosure, it is possible to appropriately update the perceptual model. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a schematic diagram of a perceptual model updating system. [Diagram 2] FIG. 1 is a schematic configuration diagram of a vehicle. [Diagram 3] FIG. 2 is a hardware configuration diagram of the perceptual model updating device. [Figure 4] FIG. 2 is a functional block diagram of a processor included in the perceptual model updating device. [Diagram 5] 11A and 11B are diagrams illustrating an example of prediction of labels for target data and calculation of scores corresponding to the target data. [Figure 6] 13 is a flowchart of a perceptual model updating process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] The perceptual model updating device will be described in detail below with reference to the drawings. The perceptual model updating device predicts a label corresponding to the target data and calculates an uncertainty score and an inconsistency score by inputting each of a plurality of target data and modified data obtained by adding a predetermined modification to the target data into a perceptual model. The perceptual model is trained to predict a label corresponding to the input data and to output an uncertainty score for the label. The label represents, for example, a type of object represented in the data. The inconsistency score indicates the magnitude of the difference between a label predicted for the target data and a label predicted for the modified data corresponding to the target data. The perceptual model is composed of a neural network such as Segnet or U-Net that performs semantic segmentation to assign a label corresponding to the object represented by each pixel included in an input image.

[0018] The perceptual model updating device classifies, among the plurality of target data, target data having an uncertainty score higher than a predetermined uncertainty threshold and an inconsistency score higher than a predetermined inconsistency threshold into first data. The perceptual model updating device also classifies, among the plurality of target data, target data having an uncertainty score lower than the uncertainty threshold and an inconsistency score higher than the inconsistency threshold into second data. The perceptual model updating device also classifies, among the plurality of target data, target data having an uncertainty score lower than the uncertainty threshold and an inconsistency score lower than the inconsistency threshold into third data.

[0019] The perceptual model updating device assigns an input label inputted via an input device to the first data as a label of the first data. The perceptual model updating device also assigns a label to the second data by a predetermined label assignment model that assigns a label to the input data. The perceptual model updating device then updates the perceptual model by learning the perceptual model using the first data to which the label has been assigned, the second data to which the label has been assigned, and the third data.

[0020] FIG. 1 is a schematic diagram of a perceptual model updating system. In this embodiment, the perceptual model updating system 1 includes a plurality of vehicles 2 and a perceptual model updating device 3. The perceptual model updating device 3 updates a perceptual model using data collected by each of the plurality of vehicles 2. Each of the plurality of vehicles 2 is communicatively connected to the perceptual model updating device 3 via the wireless base station WBS and the communication network NW by accessing a wireless base station WBS connected to the communication network NW to which the perceptual model updating device 3 is connected via a gateway (not shown) or the like. Hereinafter, one of the plurality of vehicles 2 is also referred to as a "vehicle 2". In the perceptual model updating system 1, a plurality of wireless base stations WBS may be connected to the communication network NW.

[0021] 2 is a schematic configuration diagram of the vehicle 2. The vehicle 2 has a peripheral camera 21, a data communication module (DCM) 22, a Global Navigation Satellite System (GNSS) receiver 23, a storage device 24, and an Electronic Control Unit (ECU) 25. The peripheral camera 21, the data communication module 22, the GNSS receiver 23, the storage device 24, and the ECU 25 are communicatively connected via an in-vehicle network conforming to a standard such as a controller area network.

[0022] The surrounding camera 21 is an example of an update data generating device that generates update data used for updating the perception model by the perception model updating device 3. The surrounding camera 21 generates a surrounding image that represents the surrounding situation of the vehicle 2 as image data included in the update data. The surrounding camera 21 has a two-dimensional detector configured with an array of photoelectric conversion elements sensitive to visible light, such as a CCD or a C-MOS, and an imaging optical system that forms an image of the area to be photographed on the two-dimensional detector. The surrounding camera 21 is disposed, for example, at the front upper part of the vehicle interior, facing forward. The surrounding camera 21 photographs the surrounding situation of the vehicle 2 through the windshield or rear window at a predetermined photographing period (for example, 1 / 30 second to 1 / 10 second) and outputs a surrounding image representing the surrounding situation.

[0023] The vehicle 2 may have, as an update data generating device, a LiDAR (Light Detection And Ranging) sensor that generates a distance image in which each pixel has a value corresponding to the distance to an object in the vicinity of the vehicle 2 represented by the pixel. The LiDAR sensor has a laser that generates infrared laser light and a receiver that two-dimensionally scans the laser light reflected by an object and received through an optical window. The receiver measures the time it takes for the irradiated laser light to be reflected by the object and received, thereby generating a distance image in which each pixel has a value corresponding to the distance to the object represented by the pixel. The LiDAR sensor is disposed, for example, at the front upper part of the vehicle interior, facing forward. The LiDAR sensor outputs a distance image representing the distance to an object in front of the vehicle 2 at a predetermined shooting period (for example, 1 / 30 second to 1 / 10 second). The distance image is an example of image data.

[0024] The data communication module 22 is an example of a communication device, and is a device that executes wireless communication processing conforming to a predetermined wireless communication standard such as so-called 4G (4th Generation) or 5G (5th Generation). For example, the data communication module 22 accesses a wireless base station WBS, and is communicatively connected to the perceptual model updating device 3 via the wireless base station WBS and the communication network NW. The data communication module 22 transmits data received from the ECU 25 to the perceptual model updating device 3 by including the data in an uplink wireless signal and transmitting the wireless signal to the wireless base station WBS. Note that the data communication module 22 may be implemented as a part of the ECU 25.

[0025] The GNSS receiver 23 is an example of an environmental data generating device that generates environmental data representing the environment when the update data generating device generated the data. The GNSS receiver 23 receives GNSS signals from GNSS satellites at predetermined intervals using a GNSS antenna (not shown), and determines the vehicle's own position based on the received GNSS signals. The GNSS receiver 23 generates a positioning signal representing the result of determining the vehicle's own position based on the GNSS signals as environmental data at predetermined intervals, and outputs the signal to the ECU 25 via the in-vehicle network.

[0026] Vehicle 2 may have, as environmental data generating devices, a temperature sensor that outputs temperature data indicating the temperature around vehicle 2, an illuminance sensor that outputs illuminance data indicating the illuminance around vehicle 2, and a rainfall sensor that outputs rainfall data indicating the amount of rainfall around vehicle 2.

[0027] The storage device 24 is an example of a vehicle storage device, and includes, for example, a hard disk drive or a non-volatile semiconductor memory. The storage device 24 stores the surrounding images output by the surrounding camera 21. The storage device 24 may store map data including information on features such as lane markings in association with positions.

[0028] The ECU 25 includes a communication interface, a memory, and a processor. The ECU 25 acquires a surrounding image from the surrounding camera 21 via the communication interface, and stores the image in the storage device 24. The ECU 25 also acquires map information of the vicinity of a position represented by a positioning signal acquired from the GNSS receiver 23 at a time corresponding to the generation time of the surrounding image from the storage device 24, and identifies a road shape type (sharp curve, branch point, etc.) at the time the surrounding image was generated by comparing the shape of the lane markings included in the map information with a road shape template stored in advance in the memory, and stores the identified road shape type in the storage device 24 in association with the surrounding image as environmental data representing the environment at the time the surrounding image was generated. The ECU 25 may also store the temperature, illuminance, or amount of rain at the time the surrounding image was generated in the storage device 24 in association with the surrounding image as environmental data. The ECU 25 also transmits the surrounding image and the associated environmental data stored in the storage device 24 to the perception model update device 3 via the communication interface and the data communication module 22.

[0029] The ECU 25 may acquire map information of the vicinity of the vehicle's own position represented by the positioning signal output by the GNSS receiver 23 from the storage device 24, and transmit a control signal to a driving mechanism of the vehicle 2 so that the vehicle 2 drives in a lane defined by lane markings included in the map information, thereby executing automatic driving of the vehicle 2. The driving mechanism includes, for example, an engine or a motor that supplies power to the vehicle 2, a brake that reduces the driving speed of the vehicle 2, and a steering mechanism that steers the vehicle 2.

[0030] The ECU 25 may further control the traveling of the vehicle 2 so that the distance between the peripheral image output by the peripheral camera 21 and the detected peripheral object becomes larger than a predetermined distance. The ECU 25 can detect the peripheral object by inputting the peripheral image to a classifier that has been trained in advance to detect the object from the image. The classifier may be a perception model that is updated by the perception model updating device of the present disclosure.

[0031] 2 is a diagram showing a hardware configuration of the perceptual model updating device 3. The perceptual model updating device 3 includes a communication interface 31, a storage device 32, a memory 33, and a processor .

[0032] The communication interface 31 is an example of a communication unit, and has an interface circuit for connecting the perceptual model updating device 3 to the communication network NW. The communication interface 31 is configured to be able to communicate with other devices via the communication network NW. That is, the communication interface 31 passes data and the like received from other devices via the communication network NW to the processor 34. In addition, the communication interface 31 transmits data and the like received from the processor 34 to other devices via the communication network.

[0033] The storage device 32 is an example of a storage unit, and includes a storage device such as a hard disk device or a non-volatile semiconductor memory device. The storage device 32 stores surrounding images and environmental data transmitted from each of the multiple vehicles 2. The surrounding images are an example of multiple data stored in the storage device 32.

[0034] The memory 33 is another example of a storage unit, and includes a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 33 stores various data used in processing by the processor 34, such as a parameter group for defining a perceptual model and a labeling model, target data extracted from a plurality of data stored in the storage device 32, and an uncertainty threshold and an inconsistency threshold used to classify target data whose label is predicted by the perceptual model. The memory 33 also stores various application programs, such as a computer program for perceptual model updating for causing the perceptual model updating device 3 to execute a perceptual model updating method.

[0035] The processor 34 includes one or more central processing units (CPUs) and their peripheral circuits, and may further include other arithmetic circuits such as a logic arithmetic unit or a numeric arithmetic unit.

[0036] FIG. 4 is a functional block diagram of the processor 34 included in the perceptual model updating device 3. As shown in FIG.

[0037] The processor 34 of the perceptual model updating device 3 has, as functional blocks, an extraction unit 341, a model calculation unit 342, a classification unit 343, an assignment unit 344, and an update unit 345. Each of these units in the processor 34 is a functional module implemented by a program executed on the processor 34. A computer program for realizing the functions of each unit of the processor 34 may be provided in a form recorded on a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium. Alternatively, each of these units in the processor 34 may be implemented in the perceptual model updating device 3 as an independent integrated circuit, a microprocessor, or firmware.

[0038] The extraction unit 341 extracts multiple target data from multiple data stored in the storage device 32 with a predetermined probability. The extraction unit 341 may extract, as target data, a surrounding image randomly selected with a predetermined probability (for example, 1%) from the surrounding images stored in the storage device 32. The extraction unit 341 may also extract target data from the multiple data with a predetermined probability for each class into which the multiple data are classified based on the situation in which each of the multiple data is acquired. For example, the extraction unit 341 may extract, as target data corresponding to a predetermined road shape type, a surrounding image randomly selected with a predetermined probability from the surrounding images associated with a predetermined road shape type among the surrounding images stored in the storage device 32. The class may be a time period (daytime, nighttime, etc.) determined based on the illuminance included in the environmental data, a weather condition (rainfall, snowfall, etc.) determined based on the temperature and precipitation included in the environmental data, or a combination of these (a sharp curve during nighttime rainfall, a lane branch point during daytime snowfall, etc.).

[0039] The model calculation unit 342 inputs each of multiple target data and modified data obtained by making predetermined modifications to the target data into the perceptual model, thereby predicting a label corresponding to the target data and calculating an uncertainty score and an inconsistency score corresponding to the target data.

[0040] FIG. 5 is a diagram illustrating an example of prediction of a label for target data and calculation of a score corresponding to the target data.

[0041] The model calculation unit 342 inputs the surrounding image SP to the perception model PM to generate a label image LP in which a label is predicted for each of a plurality of pixels in the surrounding image SP. In the label image LP, a label La is predicted for a pixel predicted to correspond to the lane in which the vehicle 2 is traveling among the roads on which the vehicle 2 is traveling. A label Lb is predicted for a pixel predicted to correspond to an area other than the lane in which the vehicle 2 is traveling among the roads on which the vehicle 2 is traveling. A label Lc is predicted for a pixel predicted to correspond to an area other than the road on which the vehicle 2 is traveling. In addition, an uncertainty indicating the uncertainty of the label prediction for the pixel is associated with each of a plurality of pixels in the label image LP.

[0042] The model calculation unit 342 calculates an uncertainty score corresponding to the peripheral image SP based on the uncertainty associated with each of the pixels in the labeled image LP. The model calculation unit 342 may calculate the sum of the product of the uncertainty associated with each of the pixels in the labeled image LP and the logarithm of the uncertainty, and may use the sign-inverted value (Shannon's entropy) as the uncertainty score corresponding to the peripheral image SP.

[0043] Moreover, the model calculation unit 342 inputs the modified peripheral image SP′, which is obtained by performing a predetermined modification on the peripheral image SP, into the perception model PM to generate a modified label image LP′ in which labels corresponding to each of the multiple pixels in the modified peripheral image SP′ are predicted. The modified peripheral image SP′ may be an image in which the positions of each pixel in the peripheral image SP are horizontally inverted. In the modified label image LP′, a label La′ is predicted for pixels predicted to correspond to the lane in which the vehicle 2 is traveling among the roads on which the vehicle 2 is traveling. A label Lb′ is predicted for pixels predicted to correspond to an area other than the lane in which the vehicle 2 is traveling among the roads on which the vehicle 2 is traveling. A label Lc′ is predicted for pixels predicted to correspond to an area other than the roads on which the vehicle 2 is traveling.

[0044] The model calculation unit 342 calculates an inconsistency score indicating the magnitude of difference between the label predicted for the peripheral image SP and the label predicted for the modified peripheral image SP'. For example, the model calculation unit 342 compares the distribution of the uncertainty associated with each of the multiple pixels of the labeled image LP with the distribution of the uncertainty associated with each of the multiple pixels of the modified labeled image LP'. Graph G represents the distribution of the uncertainty associated with each of the multiple pixels of the labeled image LP and the distribution of the uncertainty associated with each of the multiple pixels of the modified labeled image LP'. In graph G, the horizontal axis represents the uncertainty and the vertical axis represents the frequency. The model calculation unit 342 calculates the inconsistency score so that the larger the difference between the distribution of the uncertainty associated with each of the multiple pixels of the labeled image LP and the distribution of the uncertainty associated with each of the multiple pixels of the modified labeled image LP', the larger the value of the inconsistency score becomes.

[0045] 4, the classification unit 343 classifies, among the plurality of target data, target data whose uncertainty score is higher than the uncertainty threshold stored in the memory 33 and whose inconsistency score is higher than the inconsistency threshold stored in the memory 33 into first data. The classification unit 343 classifies, among the plurality of target data, target data whose uncertainty score is lower than the uncertainty threshold and whose inconsistency score is higher than the inconsistency threshold into second data. The classification unit 343 classifies, among the plurality of target data, target data whose uncertainty score is lower than the uncertainty threshold and whose inconsistency score is lower than the inconsistency threshold into third data.

[0046] The classifier 343 may treat the target data whose uncertainty score matches the uncertainty threshold as having an uncertainty score higher than the uncertainty threshold or lower than the uncertainty threshold. Similarly, the classifier 343 may treat the target data whose inconsistency score matches the inconsistency threshold as having an inconsistency score higher than the inconsistency threshold or lower than the inconsistency threshold.

[0047] The classification unit 343 may classify, among the multiple target data, target data whose uncertainty score is higher than the uncertainty threshold and whose inconsistency score is higher than the inconsistency threshold, into any of the first data to the third data, or into other data.

[0048] The assigning unit 344 assigns an input label inputted to the first data via an input device as a label of the first data (manual labeling). For example, the assigning unit 344 generates label assignment screen data for displaying a label assignment screen for assigning a label to the first data, and transmits the label assignment screen data to a display device (not shown) having a display for displaying an image visibly recognizable to a user via the communication interface 31. A user who is in contact with the display on which the label assignment screen is displayed operates an input device connected via the communication interface 31 to specify an input label to be assigned to the first data. The input label is stored in the memory 33 in response to a user's operation of specifying a predetermined area of ​​the label assignment screen. The assigning unit 344 assigns the input label as a label of the first data, and stores the label in the memory 33. The assigning unit 344 may transmit the label assignment screen data to an information processing device other than the perceptual model updating device 3 via the communication interface 31.

[0049] Furthermore, the assigning unit 344 assigns a label to the second data by a label assignment model that assigns a label to input data. The label assignment model may be a model other than the perceptual model PM (e.g., a model having a network configuration different from that of the perceptual model PM), or may be the perceptual model PM. The assigning unit 344 stores the second data to which the label has been assigned by the label assignment model in the memory 33.

[0050] The update unit 345 updates the perception model PM by learning the perception model PM using the first data to which a label has been manually assigned, the second data to which a label has been assigned by the label assignment model, and the third data. The update unit 345 uses the first data to which a label has been manually assigned, the second data to which a label has been assigned by the label assignment model, and the third data as teacher data, and learns the perception model PM according to a predetermined learning method such as an error backpropagation method. The update unit 345 replaces the perception model PM before the update with the updated perception model. For example, the update unit 345 transmits a parameter group for defining the updated perception model to the vehicle 2 via the communication interface 31. The ECU 25 of the vehicle 2 executes subsequent object detection using the updated perception model.

[0051] Fig. 6 is a flowchart of the perceptual model update process. The processor 34 of the perceptual model update device 3 executes the perceptual model update process shown in Fig. 6 in response to a predetermined operation by the user. The processor 34 may execute the perceptual model update process shown in Fig. 6 when a predetermined condition is satisfied, such as a predetermined time has elapsed since the previous execution or a predetermined number of peripheral images have been accumulated in the storage device 32. Furthermore, the processor 34 may repeatedly execute the perceptual model update process shown in Fig. 6 a plurality of times each time the perceptual model update process is started.

[0052] First, the extraction unit 341 of the processor 34 extracts a plurality of target data from a plurality of data stored in the storage device 32 with a predetermined probability (step S1).

[0053] Next, the model calculation unit 342 of the processor 34 inputs each of the multiple target data and modified data that makes predetermined changes to the target data into the perceptual model PM, thereby predicting a label corresponding to the target data and calculating an uncertainty score and an inconsistency score corresponding to the target data (step S2).

[0054] The classification unit 343 of the processor 34 classifies, among the plurality of target data, target data whose uncertainty score is higher than the uncertainty threshold stored in the memory 33 and whose inconsistency score is higher than the inconsistency threshold stored in the memory 33 into first data. The classification unit 343 also classifies, among the plurality of target data, target data whose uncertainty score is lower than the uncertainty threshold and whose inconsistency score is higher than a predetermined inconsistency threshold into second data. The classification unit 343 also classifies, among the plurality of target data, target data whose uncertainty score is lower than the uncertainty threshold and whose inconsistency score is lower than the predetermined inconsistency threshold into third data (step S3).

[0055] For the first data (step S3:A), the assignment unit 344 of the processor 34 assigns the input label input via the input device as a label for the first data (step S4), and proceeds to step S6, which will be described later.

[0056] For the second data (step S3:B), the attachment unit 344 attaches a label according to the label attachment model (step S5), and the process proceeds to step S6, which will be described later.

[0057] Regarding the third data (step S3:C), the classification unit 343 proceeds to step S6 described below without modifying the labels assigned by the model calculation unit 342 using the perceptual model PM.

[0058] The update unit 345 of the processor 34 updates the perceptual model PM by learning the perceptual model PM using the first data to which labels have been manually assigned, the second data to which labels have been assigned by the label assignment model, and the third data (step S6), and then terminates the perceptual model update process.

[0059] By performing the perceptual model updating process in this manner, the perceptual model updating device 3 can appropriately update the perceptual model.

[0060] It should be understood that those skilled in the art can make various changes, substitutions, and alterations thereto without departing from the spirit and scope of the present disclosure. [Explanation of symbols]

[0061] 1 Perceptual model updating system 2 Vehicles 3 Perceptual model update device 341 Extraction part 342 Model Calculation Unit 343 Classification Department 344 Grant Department 345 Update Department

Claims

1. a model calculation unit that inputs each of a plurality of target data and modified data obtained by applying a predetermined modification to the target data into a perceptual model trained to predict a label corresponding to input data and output an uncertainty score indicating the uncertainty of the prediction, thereby predicting the label corresponding to the target data and calculating the uncertainty score corresponding to the target data and an inconsistency score indicating the magnitude of difference between the label predicted for the target data and the label predicted for the modified data; a classification unit that classifies, among the plurality of target data, the target data whose uncertainty score is higher than a predetermined uncertainty threshold and whose inconsistency score is higher than a predetermined inconsistency threshold into first data, the target data whose uncertainty score is lower than the uncertainty threshold and whose inconsistency score is higher than the inconsistency threshold into second data, and the target data whose uncertainty score is lower than the uncertainty threshold and whose inconsistency score is lower than the inconsistency threshold into third data; an assignment unit that assigns an input label inputted via an input device to the first data as a label of the first data, and assigns the label to the second data according to a predetermined label assignment model that assigns the label to input data; an update unit that updates the perceptual model by learning the perceptual model using the first data to which the label has been assigned, the second data to which the label has been assigned, and the third data; A perceptual model updating device comprising:

2. 2. The perceptual model updating device according to claim 1, further comprising an extraction unit configured to extract the target data from the plurality of data with a predetermined probability for each class into which the plurality of data are classified based on a situation in which the plurality of data are acquired.

3. 2. The perceptual model updating device according to claim 1, wherein each of the plurality of target data is image data having a plurality of pixels, and the perceptual model predicts the label for each of a plurality of pixels contained in the image data.

4. 4. The perceptual model updating device according to claim 3, wherein the model calculation section sets as the modified data data obtained by horizontally inverting the positions of each pixel in the image data.

5. a perceptual model updating device that updates a trained perceptual model to predict a label corresponding to input data and output an uncertainty score indicating the uncertainty of the prediction, inputting each of a plurality of target data and modified data obtained by applying a predetermined modification to the target data into the perceptual model, thereby predicting the label corresponding to the target data, and calculating the uncertainty score corresponding to the target data and an inconsistency score indicating a magnitude of difference between the label predicted for the target data and the label predicted for the modified data; Among the plurality of target data, the target data whose uncertainty score is higher than a predetermined uncertainty threshold and whose inconsistency score is higher than a predetermined inconsistency threshold are classified as first data, the target data whose uncertainty score is lower than the uncertainty threshold and whose inconsistency score is higher than the inconsistency threshold are classified as second data, and the target data whose uncertainty score is lower than the uncertainty threshold and whose inconsistency score is lower than the inconsistency threshold are classified as third data; assigning an input label inputted via an input device to the first data as a label of the first data; assigning the label to the second data using a predetermined labeling model that assigns the label to input data; updating the perceptual model by training the perceptual model using the first data to which the labels have been manually assigned, the second data to which the labels have been assigned by the labeling model, and the third data; A method for updating a perceptual model comprising:

6. a perceptual model trained to predict a label corresponding to input data and output an uncertainty score indicating the uncertainty of the prediction, inputting each of a plurality of target data and modified data obtained by applying a predetermined modification to the target data, thereby predicting the label corresponding to the target data, and calculating the uncertainty score corresponding to the target data and an inconsistency score indicating the magnitude of difference between the label predicted for the target data and the label predicted for the modified data; classifying, among the plurality of target data, the target data whose uncertainty score is higher than a predetermined uncertainty threshold and whose inconsistency score is higher than a predetermined inconsistency threshold into first data, the target data whose uncertainty score is lower than the uncertainty threshold and whose inconsistency score is higher than the inconsistency threshold into second data, and the target data whose uncertainty score is lower than the uncertainty threshold and whose inconsistency score is lower than the inconsistency threshold into third data; assigning an input label inputted via an input device to the first data as a label of the first data; assigning the label to the second data using a predetermined labeling model that assigns the label to input data; updating the perceptual model by training the perceptual model using the first data that has been manually labeled, the second data that has been labeled by the labeling model, and the third data; A computer program for updating a perception model that causes a computer to execute the above.

Images