Data acquisition device, data acquisition method, and program

Abstract

To provide a technique that acquires training data useful for retraining a machine learning model.SOLUTION: A data acquisition device according to an embodiment of the present disclosure includes: a determination unit that determines, on the basis of second type data related to first type data to be processed by the machine learning model, whether a predetermined event related to a machine learning model occurs; and an acquisition unit that acquires, when the predetermined event occurs, the first type data.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a data collection device, a data collection method, and a program. [Background technology]

[0002] In recent years, the use of machine learning models has progressed in various technical fields. The accuracy of a machine learning model depends on the training data used to train the machine learning model, and if data to be processed that deviates from the training data is input to the machine learning model, it may be difficult for the machine learning model to achieve the expected performance.

[0003] For this reason, an approach is sometimes used in which the machine learning model is retrained using data from the field in which the machine learning model is operated. Several techniques have been proposed for collecting training data to be used for retraining such machine learning models. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-24534 A [Patent Document 2] JP 2020-135494 A [Patent Document 3] JP 2014-59729 A Summary of the Invention [Problem to be solved by the invention]

[0005] Instead of simply collecting data from operational sites and retraining machine learning models with these training data, there is a need to collect data on which operational machine learning models are unable to achieve good performance, and retrain machine learning models with the collected data.

[0006] In view of the above problems, one objective of the present disclosure is to provide a technique for collecting training data useful for retraining a machine learning model. [Means for solving the problem]

[0007] One aspect of the present disclosure relates to a data collection device having a determination unit that determines whether a specified event related to a machine learning model occurs based on a second type of data related to a first type of data to be processed by the machine learning model, and a collection unit that collects the first type of data when the specified event occurs. Effect of the Invention

[0008] According to the present disclosure, it is possible to provide a technique for collecting training data useful for retraining a machine learning model. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic diagram illustrating an image processing system according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a schematic diagram illustrating a data collection process according to one embodiment of the present disclosure. [Diagram 3] FIG. 3 is a block diagram illustrating a hardware configuration of a data collection device according to an embodiment of the present disclosure. [Figure 4] FIG. 4 is a block diagram illustrating a functional configuration of a data collection device according to an embodiment of the present disclosure. [Diagram 5] FIG. 5 is a flow chart illustrating a data collection process according to one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0011] In the following examples, a data collection device is disclosed for collecting training data for retraining a machine learning model.

[0012] [Image processing system] First, an image processing system including a data collection device according to an embodiment of the present disclosure will be described. Fig. 1 is a schematic diagram showing an image processing system according to an embodiment of the present disclosure. As shown in Fig. 1, an image processing system 10 includes a camera 20, a training data database (DB) 30, a machine learning model database (DB) 40, a data collection device 100, a training device 200, and an image processing device 300.

[0013] The camera 20 acquires video data to be processed by the image processing device 300 during operation, and acquires video data of the operation site to be input to a trained machine learning model used by the image processing device 300. In addition, the camera 20 provides the data collection device 100 with video data captured of the operation site in response to a data acquisition request from the data collection device 100.

[0014] The data collection device 100 collects training image data for training a machine learning model from the camera 20, and stores the training data, which is composed of pairs of each image data and its label, in the training data DB 30.

[0015] In the following embodiment, as shown in FIG. 2, the data collection device 100 acquires non-video data related to the video data to be processed by the machine learning model, and determines whether a predetermined event related to the machine learning model occurs based on the non-video data. Here, the non-video data may be data related to the video data to be processed that is input to the image processing device 300. For example, the non-video data may be data indicating that cows other than Holstein cows have begun to be raised in a ranch where cows are managed using a machine learning model that recognizes Holstein cows. Alternatively, the non-video data may be data indicating that a state change has occurred in which the animals look different in a case where the number of animals is recognized using a machine learning model. Alternatively, the non-video data may be data indicating that a large number of people and animals are densely present, such that people and animals are imaged overlapping each other, in a case where a machine learning model is used to recognize people and animals. When the occurrence of such a predetermined event is detected, the data collection device 100 collects video data for retraining the machine learning model from the camera 20, assigns a label to each video data, and stores training data consisting of the video data and the label in the training data DB 30.

[0016] The training device 200 trains or retrains the machine learning model by using the training data stored in the training data DB 30. When retraining the machine learning model, the training device 200 may retrain the machine learning model by the training data collected for retraining by the data collection device 100.

[0017] The image processing device 300 acquires image data to be processed that is captured by the camera 20, inputs the acquired image data to a trained machine learning model, and acquires processing results from the machine learning model.

[0018] According to an embodiment described later, when the data collection device 100 determines that the video data to be processed input to the machine learning model in operation significantly deviates from the training data used to train the machine learning model based on the non-video data related to the video data to be processed, it determines that retraining of the machine learning model is necessary, collects video data showing the operation site from the camera 20, and causes the training device 200 to retrain the machine learning model using the collected video data as training data. This makes it possible to retrain the machine learning model in operation at an appropriate time, and to continue good image processing.

[0019] Here, the data collection device 100 may be realized by a computing device such as a server, a personal computer (PC), a smartphone, a tablet, etc., and may have a hardware configuration such as that shown in Fig. 3. That is, the data collection device 100 has a drive device 101, a storage device 102, a memory device 103, a processor 104, a user interface (UI) device 105, and a communication device 106, which are interconnected via a bus B.

[0020] The programs or instructions for implementing various functions and processes in the data collection device 100 may be stored in a removable storage medium such as a CD-ROM (Compact Disk-Read Only Memory) or a flash memory. When the storage medium is set in the drive device 101, the programs or instructions are installed from the storage medium via the drive device 101 into the storage device 102 or the memory device 103. However, the programs or instructions do not necessarily have to be installed from the storage medium, and may be downloaded from any external device via a network or the like.

[0021] The storage device 102 is realized by a hard disk drive or the like, and stores installed programs or instructions as well as files, data, etc. used in executing the programs or instructions.

[0022] The memory device 103 is realized by a random access memory, a static memory, or the like, and when a program or an instruction is activated, reads and stores the program or instruction, data, or the like from the storage device 102. The storage device 102, the memory device 103, and the removable storage medium may be collectively referred to as a non-transitory storage medium.

[0023] The processor 104 may be realized by one or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), processing circuitry, etc., which may be composed of one or more processor cores, and performs various functions and processes of the data collection device 100 in accordance with programs, instructions, data such as parameters necessary to execute the programs or instructions, etc. stored in the memory device 103.

[0024] The user interface (UI) device 105 may be composed of input devices such as a keyboard, a mouse, a camera, a microphone, etc., output devices such as a display, a speaker, a headset, a printer, etc., and input / output devices such as a touch panel, and realizes an interface between a user and the data collection device 100. For example, a user operates the data collection device 100 by operating a GUI (Graphical User Interface) displayed on a display or a touch panel using a keyboard, a mouse, etc.

[0025] The communication device 106 is realized by various communication circuits that execute wired and / or wireless communication processing with an external device, the Internet, a LAN (Local Area Network), a cellular network, or other communication network.

[0026] However, the above-described hardware configuration is merely an example, and the data collection device 100 according to the present disclosure may be realized by any other suitable hardware configuration.

[0027] [Data collection device] Next, a description will be given of the data collection device 100 according to an embodiment of the present disclosure. Fig. 4 is a block diagram showing the data collection device 100 according to an embodiment of the present disclosure.

[0028] 4, the data collection device 100 includes a determination unit 110 and a collection unit 120. Each functional unit of the determination unit 110 and the collection unit 120 may be realized by the processor 104 executing a computer program stored in the memory device 103 of the data collection device 100.

[0029] The determination unit 110 determines whether a predetermined event related to the machine learning model occurs based on the second type of data related to the first type of data to be processed by the machine learning model. For example, when the machine learning model is an image processing model, the first type of data to be processed may be video data input to the machine learning model, and the second type of data may be non-video data related to a video object captured in the video data. Specifically, the non-video data may indicate that the video object captured in the video data to be processed is different from the video object captured in the video data used to train the machine learning model in operation, and / or has changed significantly. For example, the non-video data may indicate that a new video object appears in the video data to be processed, that an existing video object has changed to become an unknown video object, and the like. In these cases, it is expected that the accuracy of the machine learning model may be degraded, and the determination unit 110 may determine that a performance degradation event of the machine learning model may occur.

[0030] In the following examples, the first type of data to be processed by the machine learning model is focused on video data, but the first type of data to be processed in the present disclosure is not limited to this and may be data in other formats. Also, the second type of data is not limited to non-video data and may be data in other formats that can explain the first type of data.

[0031] In one embodiment, when the non-video data indicates that the video data includes an unknown video object that is not used to train the machine learning model, the determination unit 110 may determine that a performance degradation event of the machine learning model occurs as a predetermined event. For example, the non-video data may be management information of the facility in which the camera 20 is installed, and the management information may indicate that a new breed of cattle has been received, or that a resident of a different age group has moved into a nursing home. When the management information as the non-video data indicates that a different breed of cattle will be received into a ranch where only Holstein cattle are present, the determination unit 110 may determine that a new video object will appear in the video data to be processed, and a performance degradation event of the machine learning model may occur. Similarly, when the management information as the non-video data indicates that a young person will move into a facility where only gray-haired old people are present, the determination unit 110 may determine that a new video object will appear in the video data to be processed, and a performance degradation event of the machine learning model may occur.

[0032] Also, in one embodiment, when the non-video data indicates that the video data includes a video object in a second state different from the video object in a first state used to train the machine learning model, the determination unit 110 may determine that a performance degradation event of the machine learning model occurs as a predetermined event. For example, the non-video data may be management information of the facility in which the camera 20 is installed, and the management information may indicate health information (e.g., illness, estrus, etc.) of animals in the facility or an event such as shearing. When the management information as the non-video data indicates that animals in the farm have entered estrus, the determination unit 110 may assume that the existing video object changes to an unknown video object and determine that a performance degradation event of the machine learning model may occur. Similarly, when the management information as the non-video data indicates that a shearing event has been performed, the determination unit 110 may assume that the existing video object changes to an unknown video object and determine that a performance degradation event of the machine learning model may occur.

[0033] Such an existing video object may change to an unknown video object not only due to the video object itself, but also due to other factors. For example, the non-video data may be audio data collected in an area captured by the camera 20, and the audio data may indicate a crowded state of the video object. When the audio level (e.g., decibels, etc.) of the audio data as non-video data is equal to or higher than a predetermined threshold due to crowding of cows in a ranch, the determination unit 110 may determine that a performance degradation event of the machine learning model may occur, assuming that the images of the cows overlap each other due to crowding, resulting in video data to be processed that has a different appearance from the training data of the machine learning model. Similarly, when the audio level (e.g., decibels, etc.) of the audio data as non-video data is equal to or higher than a predetermined threshold due to crowding of people at an intersection to be captured, the determination unit 110 may determine that a performance degradation event of the machine learning model may occur, assuming that the images of the people overlap each other due to crowding, resulting in video data to be processed that has a different appearance from the training data of the machine learning model. In this way, the determination unit 110 may determine that a performance degradation event of the machine learning model occurs as a predetermined event based on audio data as non-video data.

[0034] In this way, the determination unit 110 may determine the occurrence of a predetermined event depending on whether the first type of data to be processed and the training data used to train the machine learning model diverge based on the second type of data. That is, the determination unit 110 may obtain non-video data related to the video data to be processed, and determine that a performance degradation event may occur if the non-video data indicates that the video data to be processed and the training data used to train the machine learning model in operation diverge. On the other hand, if the non-video data does not indicate that the video data to be processed and the training data used to train the machine learning model in operation diverge, the determination unit 110 may determine that a performance degradation event will not occur.

[0035] The collection unit 120 collects the first type of data when a predetermined event occurs. For example, when the determination unit 110 determines that a performance degradation event of the machine learning model has occurred, the collection unit 120 may start collecting video data from the camera 20 for retraining the machine learning model. The newly collected video data may be stored in the training data DB 30 after being assigned a correct answer label or the like.

[0036] Also, the collector 120 may end collection of the first type of data when a predetermined event ends or when a predetermined number of the first type of data has been collected. For example, the collector 120 may end collection of video data when the non-video data indicates that the video data to be processed no longer includes unknown video objects that are not used to train the machine learning model, when the non-video data indicates that existing video objects have stopped changing and become known video objects, etc. Alternatively, the collector 120 may end collection of video data when a sufficient number of new video data has been collected to retrain the machine learning model.

[0037] [Data collection and processing] Next, a data collection process according to an embodiment of the present disclosure will be described. The data collection process is for collecting training data for retraining an image processing model, but is not limited thereto. The data collection process may be performed by the data collection device 100, and more specifically, may be realized by one or more processors 104 of the data collection device 100 executing one or more programs or instructions stored in one or more memory devices 103. FIG. 5 is a flow chart illustrating the data collection process according to an embodiment of the present disclosure.

[0038] As shown in Fig. 5, in step S101, non-video data related to the video data to be processed is acquired. For example, the non-video data may indicate that a new video object appears in the video data to be processed, that an existing video object changes to become an unknown video object, or the like, or may indicate that the video data to be processed is diverging from the training data used to train the machine learning model in operation. Specifically, the non-video data may be operation information of the facility in which the camera 20 is installed, or may be audio data of the imaging area of ​​the camera 20, or the like.

[0039] In step S102, the data collection device 100 determines whether a predetermined event related to the machine learning model occurs based on the non-video data. Specifically, if the non-video data indicates that the video data to be processed and the training data used to train the machine learning model in operation are diverged, the data collection device 100 may determine that a performance degradation event of the machine learning model occurs. On the other hand, if the non-video data does not indicate that the video data to be processed and the training data used to train the machine learning model in operation are diverged, the data collection device 100 may determine that a performance degradation event of the machine learning model has not occurred. If the data collection device 100 determines that a predetermined event has occurred (S102: YES), it proceeds to step S103, and if the data collection device 100 determines that a predetermined event has not occurred (S102: NO), it returns to step S101 and acquires the next non-video data.

[0040] In step S103, the data collection device 100 collects video data for retraining the machine learning model. Specifically, the data collection device 100 collects video data captured by the camera 20 based on the non-video data. For example, if the non-video data indicates that a new breed of cattle will arrive at a farm on a certain date and time, the data collection device 100 may start collecting video data of the farm captured by the camera 20 after that date and time.

[0041] According to the above-described embodiment, when the data collection device 100 determines that the video data to be processed input to the machine learning model in operation significantly deviates from the training data used to train the machine learning model based on the non-video data related to the video data to be processed, it determines that retraining of the machine learning model is necessary, collects video data showing the operation site from the camera 20, and causes the training device 200 to retrain the machine learning model using the collected video data as training data. This makes it possible to retrain the machine learning model in operation at an appropriate time, and to continue good image processing.

[0042] In addition, the following supplementary notes are provided in relation to the above description. (Appendix 1) A determination unit that determines whether a predetermined event related to the machine learning model occurs based on second type data related to the first type data to be processed by the machine learning model; a collection unit that collects the first type of data when the predetermined event occurs; A data collection device comprising: (Appendix 2) the first type of data is video data that is input to the machine learning model; 2. The data collection device of claim 1, wherein the second type of data is non-video data relating to an object captured in the video data. (Appendix 3) The data collection device according to claim 1 or 2, wherein the determination unit determines the occurrence of the specified event based on whether or not there is a divergence between the first type data to be processed and training data used to train the machine learning model, based on the second type data. (Appendix 4) A data collection device as described in any one of Appendix 1 to 3, wherein when the non-video data indicates that the video data includes an unknown video object that is not used to train the machine learning model, the determination unit determines that a performance degradation event of the machine learning model occurs as the specified event. (Appendix 5) A data collection device as described in any one of Appendixes 1 to 4, wherein when the non-video data indicates that the video data includes a video object in a second state that is different from a video object in a first state used to train the machine learning model, the determination unit determines that a performance degradation event of the machine learning model has occurred as the specified event. (Appendix 6) The data collection device according to any one of appendices 1 to 5, wherein the determination unit determines that a performance degradation event of the machine learning model occurs as the specified event based on audio data as the non-video data. (Appendix 7) 7. The data collection device of claim 1, wherein the collection unit terminates collection of the first type of data when the specified event ends or a specified number of the first type of data has been collected. (Appendix 8) determining whether a predetermined event occurs related to the machine learning model based on a second type of data related to the first type of data processed by the machine learning model; collecting said first type of data upon occurrence of said predetermined event; Computer implemented data collection methods. (Appendix 9) determining whether a predetermined event occurs related to the machine learning model based on a second type of data related to the first type of data processed by the machine learning model; collecting said first type of data upon occurrence of said predetermined event; A program that causes a computer to execute the following.

[0043] Although the examples of the present disclosure have been described in detail above, the present disclosure is not limited to the specific embodiments described above, and various modifications and variations are possible within the scope of the gist of the present disclosure described in the claims. [Explanation of symbols]

[0044] 10 Image Processing System 20 Camera 30 Training Data DB 40 Machine Learning Model DB 100 Data Acquisition Device 110 Judgment section 120 Collection Department

Claims

1. A determination unit that determines whether a predetermined event related to the machine learning model occurs based on second type data related to the first type data to be processed by the machine learning model; a collection unit that collects the first type of data when the predetermined event occurs; A data collection device comprising:

2. the first type of data is video data that is input to the machine learning model; The data collection device of claim 1 , wherein the second type of data is non-video data relating to video objects captured in the video data.

3. The data collection device according to claim 1 , wherein the determination unit determines the occurrence of the specified event based on whether or not there is a divergence between the first type of data to be processed and training data used to train the machine learning model, based on the second type of data.

4. The data collection device of claim 1 , wherein when the non-video data indicates that the video data includes an unknown video object that is not used to train the machine learning model, the determination unit determines that a performance degradation event of the machine learning model occurs as the specified event.

5. The data collection device of claim 1 , wherein when the non-video data indicates that the video data includes a video object in a second state that is different from a video object in a first state used to train the machine learning model, the determination unit determines that a performance degradation event of the machine learning model occurs as the specified event.

6. The data collection device according to claim 1 , wherein the determination unit determines that a performance degradation event of the machine learning model occurs as the predetermined event based on audio data as the non-video data.

7. The data collection device according to claim 1 , wherein the collection unit stops collecting the first type of data when the predetermined event ends or when a predetermined number of the first type of data has been collected.

8. determining whether a predetermined event occurs related to the machine learning model based on a second type of data related to the first type of data to be processed by the machine learning model; collecting said first type of data upon occurrence of said predetermined event; Computer implemented data collection methods.

9. determining whether a predetermined event occurs related to the machine learning model based on a second type of data related to the first type of data to be processed by the machine learning model; collecting said first type of data upon occurrence of said predetermined event; A program that causes a computer to execute the following.

Images