Data management method, data management system and program

Abstract

To provide a data management method, a data management system and a program for efficiently managing data used for maintaining and managing models. [Solution] A data management method executed by one or more computers, comprising: selecting a group of learning data from a group of field data in a first database used to operate a model in accordance with predetermined selection criteria; and registering the selected group of learning data in a second database that stores data for training the model.

Description

[Technical Field]

[0001] The present disclosure relates to a data management method, a data management system, and a program. [Background technology]

[0002] Recent advances in deep learning technology have expanded the opportunities for using machine learning models. MLOps (Machine Learning Operations) is known as a scheme for maintaining and managing models in operation. In a typical MLOps scenario, field data is collected in the operational environment during model operation through a cycle consisting of data collection, annotation, model learning, model implementation, deployment, field use, input / output logging, log collection, data visualization, and data analysis. The collected field data can then be used to maintain and manage the operational model in the maintenance environment.

[0003] Several techniques have been proposed for updating and / or developing models using data collected during model operation (Patent Documents 1 and 2, etc.). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-157654 [Patent Document 2] Japanese Patent Publication No. 2020-135513 Summary of the Invention [Problem to be solved by the invention]

[0005] Typically, the model is operated on a local server that realizes the operational environment, and is updated and / or developed on a cloud server that realizes the maintenance environment, etc., using field data collected from the local server via a network.

[0006] On the other hand, if field data collected on a local server in the operational environment is transferred to a cloud server in the maintenance environment without being sorted, the huge amount of data may increase data transfer costs, and it may be necessary to secure a large-capacity cloud database (DB) to store the transferred field data.

[0007] In view of the above problems, one objective of the present disclosure is to provide a technology for efficiently managing data used for maintaining and managing models. [Means for solving the problem]

[0008] One aspect of the present disclosure is selecting an additional training data group from a first database of on-site data collected during operation of the model based on a comparison result between on-site data collected during operation of the model and training data for the model; registering the selected additional training data group in a second database that stores the training data; The present invention relates to a data management method executed by one or more computers, comprising: [Effects of the Invention]

[0009] According to the present disclosure, it is possible to provide a technique for efficiently managing data used for maintaining and managing models. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram illustrating a data management system according to one embodiment of the present disclosure. [Figure 2] FIG. 2 is a schematic diagram illustrating a data management process in a data management system according to an embodiment of the present disclosure. [Figure 3] FIG. 3 is a schematic diagram illustrating a data management process in a data management system according to an embodiment of the present disclosure. [Figure 4] FIG. 4 is a schematic diagram illustrating a data management process in a data management system according to an embodiment of the present disclosure. [Figure 5] FIG. 5 is a block diagram illustrating a hardware configuration of a server according to an embodiment of the present disclosure. [Figure 6] FIG. 6 is a flowchart illustrating a data management process according to an embodiment of the present disclosure. [Figure 7] FIG. 7 is a schematic diagram illustrating a data selection process according to one embodiment of the present disclosure. [Figure 8] FIG. 8 is a flowchart illustrating a data selection process according to an embodiment of the present disclosure. [Figure 9] FIG. 9 is a schematic diagram illustrating a data selection process according to one embodiment of the present disclosure. [Figure 10] FIG. 10 is a flowchart illustrating a data selection process according to an embodiment of the present disclosure. [Figure 11] FIG. 11 is a flowchart illustrating a data selection process according to an embodiment of the present disclosure. [Figure 12] FIG. 12 is a flowchart illustrating a data selection process according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0012] In the following embodiment, a data management system for managing data used for maintaining and managing an operational model is disclosed.

[0013] [Summary of the Disclosure] In MLOps, field data collected during model operation is stored in a local DB in the model's operating environment, and the field data group stored in the local DB is used for model adjustment, re-learning, etc. in a maintenance environment realized by a server on the cloud (cloud server). In a data management system according to an embodiment described below, a learning data group used for model adjustment, re-learning, etc. in the maintenance environment is selected from the field data group stored in the local DB according to predetermined selection criteria, and the selected field data is transferred as a learning data group to a cloud DB for the maintenance environment.

[0014] As a result, instead of transferring all of the site data collected in the operational environment to the cloud server, site data suitable for use in model adjustment, re-learning, development, etc. is selected, and the selected site data group is transferred to and registered in the cloud DB as a learning data group to be used in the maintenance environment. In this way, only site data suitable for use in model adjustment, re-learning, development, etc. is transferred to the cloud DB for the maintenance environment, making it possible to reduce the data transmission load on the network and also reduce the storage capacity of the cloud DB.

[0015] [Data Management System] A data management system 10 according to various embodiments of the present disclosure will be described first with reference to Figures 1 to 5. Figure 1 is a schematic diagram illustrating a data management system 10 according to one embodiment of the present disclosure.

[0016] As shown in FIG. 1, the data management system 10 includes a local server 100 that realizes a model operation environment and a cloud server 200 that realizes a model maintenance environment. The data management system 10 maintains and manages an operational model according to, for example, MLOps. The model here may be any type of machine learning model (e.g., neural network, etc.) trained to perform any inference processing (e.g., object detection, anomaly detection, biometric authentication, etc.). Typically, a model is operated in one or more operation environments, and as shown in the figure, field data collected during the operation of the model may be stored in multiple local servers 100 corresponding to each operation environment.

[0017] The cloud server 200 in the maintenance environment acquires a group of on-site data from each local server 100 in the operational environment, and uses the acquired group of on-site data as a group of training data to adjust, retrain, and develop (hereinafter, may be collectively referred to as "maintenance") the model. In this way, the most recently collected on-site data can be used to change the model in operation to one that is suitable for the current state of the operational environment.

[0018] In the embodiments described below, the learning data used by the cloud server 200 to maintain the model in operation does not have to be all of the site data stored in the local server 100, but may be site data selected from the site data group stored in the local server 100 according to predetermined selection criteria.

[0019] As an example, the predetermined selection criterion may be the content similarity between the data content of the site data stored in the local server 100 and the data content of existing data already used for model maintenance by the cloud server 200 and / or the data content of existing data stored in the cloud DB. Specifically, when the site data and the existing data are image data, the content similarity may be based on the pixel-related difference between the site image data and the existing image data, or may be based on the feature-related difference between the site image data and the existing image data.

[0020] As another example, the predetermined selection criterion may be the tag similarity between tag information associated with the site data and tag information of existing data already used for model maintenance by the cloud server 200 and / or existing data stored in the cloud DB. Specifically, the tag information may be any metadata associated with the site data and existing data, such as the acquisition location, acquisition date and time of the site data and existing data, and the presence or absence of a detection target object.

[0021] As yet another example, the predetermined selection criteria may be upload conditions set based on existing data already used by cloud server 200 for model maintenance and / or existing data stored in the cloud DB. Specifically, field data that is significantly different from the existing data stored in cloud server 200 and / or the existing data stored in the cloud DB (for example, if the existing data is data obtained during fine weather, the field data to be uploaded may be data obtained during rainy weather; if the existing data is data obtained during the day, the field data to be uploaded may be data obtained at night) may be selected as data that satisfies the upload conditions. It is believed that model performance can be significantly improved by maintaining a model using field data selected in this manner.

[0022] As yet another example, the predetermined selection criteria may be based on an evaluation of the model's output for the site data or the familiarity of the site data with the model. Specifically, if the output results of an operational model for the site data are insufficiently accurate, the site data may be selected as training data to maintain the model so that appropriate inferences can be made for the site data. Also, if the site data is of a type unknown to the model, i.e., if the site data significantly deviates from the range of the existing data group used to train the model, the site data may be selected as training data to maintain the model so that appropriate inferences can be made for the site data.

[0023] 2 is a schematic diagram illustrating a data management process in a data management system 10 according to an embodiment of the present disclosure. As shown in FIG. 2, in the data management system 10 according to this embodiment, a local server 100 is provided with a data selection and provision function. According to the illustrated embodiment, the local server 100 may select a group of learning data to be used for model maintenance by the cloud server 200 from a group of field data collected during model operation according to predetermined selection criteria, and provide the selected field data to the cloud server 200. Field data that is not selected may be retained in the local server 100 or may be deleted.

[0024] Next, FIG. 3 is a schematic diagram illustrating a data management process in a data management system 10 according to another embodiment of the present disclosure. As shown in FIG. 3 , in the data management system 10 according to this embodiment, the cloud server 200 is provided with a data selection and provision function. According to the illustrated embodiment, the cloud server 100 may select a group of training data to be used for model maintenance by the cloud server 200 from a group of site data collected by the local server 100 during model operation according to predetermined selection criteria, and instruct the local server 100 to provide the selected site data to the cloud server 200. For example, the cloud server 200 may compare the feature values ​​or tag information of the site data to be selected with the feature values ​​or tag information of the training data stored in the cloud database of the cloud server 200, and if the site data to be selected and the training data are significantly different, acquire the site data from the local server 100. Site data that was not selected may be retained in the local server 100 or may be deleted.

[0025] Next, FIG. 4 is a schematic diagram illustrating a data management process in a data management system 10 according to another embodiment of the present disclosure. As shown in FIG. 4, in the data management system 10 according to this embodiment, a data selection and provision function is provided in a data selection server 300, which is separate from the local server 100 and the cloud server 200. According to the illustrated embodiment, the data selection server 300 may select a set of training data to be used for model maintenance by the cloud server 200 from a set of site data collected by the local server 100 during model operation according to predetermined selection criteria, and instruct the local server 100 to provide the selected site data to the cloud server 200. In the illustrated embodiment, the selected site data is transmitted to the cloud server 200 via the data selection server 300. However, the present disclosure is not limited thereto, and the selected site data may be transmitted from the local server 100 to the cloud server 200 without passing through the data selection server 300. Unselected site data may be retained in the local server 100 or deleted.

[0026] Here, the local server 100, the cloud server 200, and the data selection server 300 may each be realized by a computing device such as a server, a personal computer (PC), a smartphone, or a tablet, and may have, for example, a hardware configuration as shown in Fig. 5. That is, the local server 100, the cloud server 200, and the data selection server 300 each have a drive device 101, a storage device 102, a memory device 103, a processor 104, a user interface (UI) device 105, and a communication device 106, which are interconnected via a bus B.

[0027] The programs or instructions that realize the various functions and processes in the local server 100, the cloud server 200, and the data selection server 300 may be stored in a removable storage medium such as a CD-ROM (Compact Disk-Read Only Memory) or a flash memory. When the storage medium is set in the drive device 101, the programs or instructions are installed from the storage medium to the storage device 102 or the memory device 103 via the drive device 101. However, the programs or instructions do not necessarily have to be installed from the storage medium, and may be downloaded from any external device via a network or the like.

[0028] The storage device 102 is realized by a hard disk drive or the like, and stores installed programs or instructions as well as files, data, etc. used to execute the programs or instructions.

[0029] The memory device 103 is realized by a random access memory, a static memory, or the like, and when a program or instruction is activated, reads and stores the program, instruction, data, or the like from the storage device 102. The storage device 102, the memory device 103, and the removable storage medium may be collectively referred to as a non-transitory storage medium.

[0030] The processor 104 may be realized by one or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), processing circuitry, etc., which may be composed of one or more processor cores, and performs various functions and processes of the local server 100, cloud server 200, and data selection server 300 in accordance with programs, instructions, data such as parameters required to execute the programs or instructions, etc. stored in the memory device 103.

[0031] The user interface (UI) device 105 may be composed of input devices such as a keyboard, mouse, camera, microphone, etc., output devices such as a display, speaker, headset, printer, etc., and input / output devices such as a touch panel, and realizes an interface between the user and the local server 100, the cloud server 200, and the data selection server 300. For example, the user operates a GUI (Graphical User Interface) displayed on a display or touch panel using a keyboard, mouse, etc. to operate the local server 100, the cloud server 200, and the data selection server 300.

[0032] The communication device 106 is realized by various communication circuits that execute wired and / or wireless communication processing with external devices, the Internet, a LAN (Local Area Network), a cellular network, or other communication networks.

[0033] However, the above-described hardware configuration is merely an example, and the local server 100, the cloud server 200, and the data selection server 300 according to the present disclosure may be realized by any other appropriate hardware configuration.

[0034] [Data Management Processing] Next, a data management process according to various embodiments of the present disclosure will be described with reference to FIG. 6. FIG. 6 is a flowchart illustrating a data management process according to one embodiment of the present disclosure. The data management process may be initiated periodically, by instruction from an operations manager, or in response to the occurrence of an event such as degradation of model performance. The data management process is performed by the data management system 10, and more specifically, may be realized by one or more processors 104 of the local server 100, cloud server 200, and data selection server 300 of the data management system 10 executing one or more programs or instructions stored in one or more memory devices 103.

[0035] In step S101, the local server 100 acquires on-site data. Specifically, the on-site data used in a model operating in an operating environment is collected by the local server 100 and stored in a local database (DB) of the local server 100. For example, if the model is an object detection model that detects objects in an image, the on-site data may be image data captured by a camera or the like and input to the model in operation. Also, if the model is a voice recognition model that recognizes spoken content, the on-site data may be voice data collected by a microphone and input to the model in operation.

[0036] In step S102, the local server 100, the cloud server 200, or the data selection server 300 determines whether to select the site data stored in the local DB as training data. Specifically, the local server 100, the cloud server 200, or the data selection server 300 selects the training data group from the site data group in the local DB according to predetermined selection criteria. As an example, the predetermined selection criteria may be content similarity between the data content of the site data stored in the local DB and the data content of existing data already used by the cloud server 200 to train the model and / or existing data stored in the cloud DB. As another example, the predetermined selection criteria may be tag similarity between tag information associated with the site data and tag information of existing data already used by the cloud server 200 to train the model and / or existing data stored in the cloud DB. As yet another example, the predetermined selection criteria may be upload conditions set based on the existing data already used by the cloud server 200 to train the model and / or existing data stored in the cloud DB. As yet another example, the predetermined selection criteria may be based on an evaluation of the model's output against the field data or on the familiarity of the field data with the model. Details of the data selection process for each of the above selection criteria will be described below with reference to the drawings.

[0037] If the site data stored in the local DB is selected as learning data (S102: YES), the data management process proceeds to step S103. On the other hand, if the site data stored in the local DB is not selected as learning data (S102: NO), the data management process returns to step S101 and repeats the above steps for the next site data stored in the local DB.

[0038] In step S103, the local server 100 transmits the selected site data to the cloud server 200. Specifically, the local server 100 transmits the selected site data to the cloud server 200 as learning data to be used for maintaining the model in operation.

[0039] In step S104, the local server 100, the cloud server 200, or the data selection server 300 determines whether the above steps have been executed for all of the site data stored in the local DB of the local server 100. If all of the site data in the local DB has been processed (S104: YES), the data management process ends. On the other hand, if all of the site data in the local DB has not been processed (S104: NO), the data management process returns to step S101, and the above steps are repeated for the next site data stored in the local DB.

[0040] [Data selection process] Next, an embodiment in which the predetermined selection criterion is based on content similarity will be described with reference to Figures 7 and 8. Figure 7 is a schematic diagram illustrating a data selection process according to an embodiment of the present disclosure, and Figure 8 is a flowchart illustrating the data selection process according to an embodiment of the present disclosure. The data selection process shown in Figure 8 relates to more detailed processing of step S102 in Figure 6.

[0041] As shown in FIG. 7, the on-site data to be selected may be compared with each of the existing data #1, #2, #3, . . . stored in the cloud DB of the cloud server 200 based on the content similarity.

[0042] Regarding the data selection process based on content similarity, as shown in FIG. 8, in step S201, the local server 100, the cloud server 200, or the data selection server 300 determines the content similarity between the on-site data and the existing data in the cloud DB of the cloud server 200.

[0043] For example, if the on-site data is image data, the local server 100, the cloud server 200, or the data selection server 300 may calculate the difference between the pixel values ​​of the on-site image data and the corresponding pixel values ​​of each existing image data #1, #2, #3, etc., and determine the content similarity between the on-site image data and the existing image data based on the difference in pixel values.

[0044] Alternatively, if a model for determining the features of on-site data and existing data is available, the local server 100, the cloud server 200, or the data selection server 300 may use the model to derive the features of the on-site data and the features of each existing image data #1, #2, #3, ..., and determine the difference between the derived features as the content similarity.

[0045] Note that the content similarity is not limited to the above, and may be based on, for example, the coordinates of feature points of objects in an image, the Euclidean distance between objects in an image, or the like.

[0046] In step S202, the local server 100, the cloud server 200, or the data selection server 300 determines whether the derived content similarity is less than a predetermined threshold. Here, the predetermined threshold may be set to any value that can be determined to be significantly different between the field data and the existing data for the model in operation. If the content similarity is less than the predetermined threshold (S202: YES), the data selection process proceeds to step S203. On the other hand, if the content similarity is equal to or greater than the predetermined threshold (S202: NO), the data selection process proceeds to step S204.

[0047] In step S203, the local server 100, the cloud server 200, or the data selection server 300 selects the field data as learning data, ends the data selection process, and proceeds to step S103 in FIG.

[0048] In step S204, the local server 100, the cloud server 200, or the data selection server 300 does not select the field data as learning data, ends the data selection process, and returns to step S101 in FIG.

[0049] Next, an embodiment in which the predetermined selection criterion is based on tag similarity will be described with reference to Figures 9 and 10. Figure 9 is a schematic diagram illustrating a data selection process according to an embodiment of the present disclosure, and Figure 10 is a flowchart illustrating the data selection process according to an embodiment of the present disclosure. The data selection process shown in Figure 10 relates to more detailed processing of step S102 in Figure 6.

[0050] 9, the site data to be selected may be compared with each of the existing data #1, #2, #3, ... stored in the cloud DB of the cloud server 200 based on tag similarity. As shown in the figure, tag information may be associated with the site data and each of the existing data #1, #2, #3, ..., and tag similarity may be determined based on the degree of agreement between the items of the tag information. Note that the tag information according to the present disclosure is not limited to the tag information items shown in the figure and may be composed of any other appropriate items representing the site data or the existing data.

[0051] Regarding the data selection process based on tag similarity, as shown in FIG. 10, in step S301, the local server 100, the cloud server 200, or the data selection server 300 determines the tag similarity between the on-site data and the existing data in the cloud DB of the cloud server 200.

[0052] For example, if the on-site data is image data of a road as shown in the figure, the local server 100, the cloud server 200, or the data selection server 300 may compare the item values ​​of the tag information of the on-site image data with the item values ​​of the tag information of each existing image data #1, #2, #3, ..., and determine the tag similarity based on the degree of agreement of the item values.

[0053] 9, the tag information of the site data has item values ​​of "highway," "morning," "car present," and "photographed obliquely above," and the tag information of existing data #1 has item values ​​of "highway," "morning," "car present," and "photographed obliquely above." All four items of the tag information of the site data and the existing data match, resulting in a match rate of 100%, and the tag similarity (0 to 1) may be set to 1, indicating a perfect match.

[0054] On the other hand, the tag information of existing data #2 has item values ​​of "general road", "morning", "cars present", and "photographed directly above", and two items match between the site data and existing data #2, resulting in a match rate of 50%, and the tag similarity may be determined to be 0.5, indicating a partial match. Similarly, the tag information of existing data #3 has item values ​​of "highway", "night", "cars present", and "photographed directly above", and two items match between the site data and existing data #3, resulting in a match rate of 50%, and the tag similarity may be determined to be 0.5, indicating a partial match.

[0055] The tag information items are not limited to those described above, and may include information about the data acquisition location, acquisition time, annotation information, acquisition conditions, acquisition environment, object information, and the like.

[0056] In step S302, the local server 100, the cloud server 200, or the data selection server 300 determines whether the derived tag similarity is less than a predetermined threshold. Here, the predetermined threshold may be set to any value that can be determined to be significantly different between the field data and the existing data for the model in operation. If the tag similarity is less than the predetermined threshold (S302: YES), the data selection process proceeds to step S303. On the other hand, if the tag similarity is equal to or greater than the predetermined threshold (S302: NO), the data selection process proceeds to step S304.

[0057] In step S303, the local server 100, the cloud server 200, or the data selection server 300 selects the field data as learning data, ends the data selection process, and proceeds to step S103 in FIG.

[0058] In step S304, the local server 100, the cloud server 200, or the data selection server 300 does not select the field data as learning data, ends the data selection process, and returns to step S101 in FIG.

[0059] Next, an embodiment in which the predetermined selection criteria are based on upload conditions will be described with reference to Fig. 11. Fig. 11 is a flowchart showing a data selection process according to an embodiment of the present disclosure. The data selection process shown in Fig. 11 relates to more detailed processing of step S102 in Fig. 6.

[0060] Regarding the data selection process based on upload conditions, as shown in FIG. 11 , in step S401, the local server 100, the cloud server 200, or the data selection server 300 determines whether the site data satisfies the predetermined upload conditions. For example, if the predetermined upload condition is an image taken during rainfall, the local server 100, the cloud server 200, or the data selection server 300 may determine whether the site data is an image taken during rainfall, and if the site data is an image taken during rainfall, determine that the site data satisfies the predetermined upload conditions. On the other hand, if the site data is not an image taken during rainfall, determine that the site data does not satisfy the predetermined upload conditions. Typically, training data taken during rainfall is relatively less abundant and more scarce than that taken during clear weather. In other words, the upload conditions may be set based on the scarcity of existing data used to train the model in operation and / or existing data stored in the cloud database.

[0061] Here, the weather information associated with the on-site data and the existing data may be acquired based on the location and date and time when the data was acquired. Furthermore, the predetermined upload conditions are not limited to those described above, and may be related to any information related to the on-site data (e.g., acquisition conditions such as brightness at the time of shooting).

[0062] If the site data satisfies the predetermined upload conditions (S401: YES), the data selection process proceeds to step S402. On the other hand, if the site data does not satisfy the predetermined upload conditions (S401: NO), the data selection process proceeds to step S403.

[0063] In step S402, the local server 100, the cloud server 200, or the data selection server 300 selects the field data as learning data, ends the data selection process, and proceeds to step S103 in FIG.

[0064] In step S403, the local server 100, the cloud server 200, or the data selection server 300 does not select the field data as learning data, ends the data selection process, and returns to step S101 in FIG.

[0065] Next, an embodiment in which the predetermined selection criterion is based on the model output evaluation or the familiarity of the site data will be described with reference to FIG. 12. Here, the familiarity of the site data may be any indicator indicating whether or not the site data has already been used to train a model in operation, or to what extent the site data has already been used. FIG. 12 is a flowchart showing a data selection process according to an embodiment of the present disclosure. The data selection process shown in FIG. 12 relates to more detailed processing of step S102 in FIG. 6.

[0066] Regarding the data selection process based on the model output evaluation or the degree of familiarity of the field data, as shown in FIG. 12, in step S501, the local server 100, the cloud server 200, or the data selection server 300 determines the model output evaluation for the field data or the degree of familiarity of the field data for the model.

[0067] For example, the local server 100, the cloud server 200, or the data selection server 300 may acquire, as the output evaluation, an evaluation of the output result of the model for the site data. The evaluation of the output result may be based on, for example, the accuracy or confidence level of the output result of the model for the site data.

[0068] Furthermore, the local server 100, the cloud server 200, or the data selection server 300 may refer to the activity status of the model's neurons to determine whether the site data is existing data used in model training or unknown data not used in model training, and derive the degree of familiarity of the site data with respect to the model based on the determination result. For example, if the model includes a neuron (called a "redundant neuron" (see, for details, https: / / shingi.jst.co.jp / pdf / 2020 / 2020_kyutech_5_tamuko.pdf, etc.)) that determines whether input data is known or unknown to the model, the value of the redundant neuron may be used to determine whether the site data is existing data used in model training or unknown data not used in model training. Specifically, if the redundant neuron outputs a confidence level indicating the likelihood that the site data input to the model is known data, the confidence level may be determined as the degree of familiarity.

[0069] In step S502, the local server 100, the cloud server 200, or the data selection server 300 determines whether the output rating or the degree of knownness is less than a predetermined threshold. Here, the predetermined threshold may be set to any value that can be determined to satisfy the required model performance for the output rating or the degree of knownness of the field data used for the field data. If the output rating or the degree of knownness is less than the predetermined threshold (S502: YES), the data selection process proceeds to step S503. On the other hand, if the output rating or the degree of knownness is equal to or greater than the predetermined threshold (S502: NO), the data selection process proceeds to step S504.

[0070] In step S503, the local server 100, the cloud server 200, or the data selection server 300 selects the field data as learning data, ends the data selection process, and proceeds to step S103 in FIG.

[0071] In step S504, the local server 100, the cloud server 200, or the data selection server 300 does not select the field data as learning data, ends the data selection process, and returns to step S101 in FIG.

[0072] According to the above-described embodiment, among the field data collected in the operational environment, field data that can be effectively used for model adjustment, relearning, development, etc. can be selectively transferred and stored in the maintenance environment, thereby reducing the data transfer load and enabling efficient use of the computational resources required for adjustment, relearning, development, etc.

[0073] In addition, the following supplementary notes are provided in relation to the above description. (Appendix 1) Selecting a group of training data from a group of field data in a first database used for operating the model according to predetermined selection criteria; registering the selected group of training data in a second database that stores training data for the model; 1. A data management method executed by one or more computers, comprising: (Appendix 2) 2. The data management method of claim 1, wherein the predetermined selection criteria is based on content similarity between the site data of the site data group and existing data in the second database. (Appendix 3) The data management method according to claim 2, wherein the selecting step selects, as the learning data group, on-site data whose content similarity is determined to be less than a predetermined threshold. (Appendix 4) A data management method described in any one of Appendices 1 to 3, wherein the predetermined selection criteria are based on tag similarity between tag information associated with site data in the site data group and tag information associated with existing data in the second database. (Appendix 5) The data management method according to claim 4, wherein the selecting step selects, as the learning data group, on-site data whose tag similarity is determined to be less than a predetermined threshold. (Appendix 6) 6. The data management method according to any one of appendices 1 to 5, wherein the predetermined selection criteria relate to upload conditions that are set based on existing data stored in the second database. (Appendix 7) The data management method according to claim 6, wherein the selecting step selects field data determined to satisfy the upload conditions as the learning data group. (Appendix 8) 8. A data management method according to any one of appendices 1 to 7, wherein the predetermined selection criteria are based on an output evaluation or familiarity of the model for the site data of the site data group. (Appendix 9) The data management method according to claim 8, wherein the selecting step selects field data for which the output evaluation or known degree is determined to be less than a predetermined threshold as the learning data group. (Appendix 10) Selecting a group of training data from a group of field data in a first database used for operating the model according to predetermined selection criteria; registering the selected group of training data in a second database that stores training data for the model; A program that causes one or more computers to execute the following: (Appendix 11) a first database that stores a set of field data used in the operation of the model; a second database that stores training data for the model; one or more processors; and the one or more processors: Selecting a learning data group from the on-site data group in accordance with predetermined selection criteria; registering the selected training data group in the second database; A data management system that runs

[0074] Although the examples of the present disclosure have been described in detail above, the present disclosure is not limited to the specific embodiments described above, and various modifications and variations are possible within the scope of the gist of the present disclosure as set forth in the claims.

[0075] The disclosures of the specification, drawings and abstract contained in Japanese Patent Application No. 2023-014069, filed February 1, 2023, are incorporated herein by reference in their entirety. [Explanation of symbols]

[0076] 10 Data Management System 100 local servers 200 cloud servers 300 Data Selection Server

Claims

1. selecting an additional training data group from a first database of on-site data collected during operation of the model based on a comparison result between on-site data collected during operation of the model and training data for the model; registering the selected additional training data group in a second database that stores the training data; 1. A data management method executed by one or more computers, comprising:

2. the first database is managed on a first server that operates the model; The data management method according to claim 1 , wherein the second database maintains the model and is managed on a second server different from the first server.

3. selecting an additional training data group from a first database of on-site data collected during operation of the model based on a comparison result between on-site data collected during operation of the model and training data for the model; registering the selected additional training data group in a second database that stores the training data; A program that causes one or more computers to execute the following.

4. a first database storing field data collected during operation of the model; a second database that stores training data for the model; a processor; and the processor: selecting an additional training data group from a group of on-site data in the first database collected during operation of the model based on a comparison result between the on-site data and the training data; registering the selected additional training data group in the second database; A data management system that runs

Images