Information processing system, information processing method, and program

Abstract

To enable more efficient checking of information composed of multiple parameters.SOLUTION: An information processing system is provided, comprising display control means for controlling display of a list of records composed of values related to multiple parameter items, and control means for controlling a display format of the list display-controlled by the display control means according to a unique number of each parameter item.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] In recent years, AI (artificial intelligence, trained models) generated by deep learning has been used in a variety of fields, including recognizing objects in images, demand forecasting, and text analysis.

[0003] To develop AI with sufficient accuracy, it is important to record the generated trained model in association with the training data and various parameter values ​​(setting values) used to generate it, so that it can be confirmed later on what level of accuracy the trained model was generated when what training data and parameter values ​​were used for training.

[0004] Patent Document 1 discloses a screen that displays a list of various information about a trained model for each version of the trained model that has been generated. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2020-113319 DISCLOSURE OF THE INVENTION [Problem to be solved by the invention]

[0006] As mentioned above, AI development is carried out through repeated trial and error, with various learning data and parameter values ​​being varied.

[0007] Furthermore, since the number of types of parameters is extremely large, it is necessary to efficiently grasp which parameters have an effect on accuracy and which parameters have no effect.

[0008] Therefore, an object of the present invention is to provide a mechanism that enables information made up of multiple parameters to be checked more efficiently. [Means for solving the problem]

[0009] a display control means for controlling the display of information composed of values ​​relating to a plurality of parameter items; a control means for controlling a display form of the information controlled by the display control means in accordance with the unique number for each parameter item; The present invention is characterized by comprising: [Effects of the Invention]

[0010] According to the present invention, it is possible to more efficiently check information made up of a plurality of parameters. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system. [Figure 2] FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing device. [Figure 3] Flowchart showing the overall process of generating a trained model [Figure 4] 10 is a flowchart showing details of the learning result analysis process in step S303. [Figure 5] Flowchart showing details of the process in step S405 [Figure 6] FIG. 10 is a diagram illustrating the processing results of steps S404 and S405. [Figure 7] FIG. 10 is a diagram illustrating the processing result of step S406. [Figure 8] FIG. 10 is a diagram illustrating another embodiment of step S404. [Figure 9] FIG. 10 is a diagram illustrating another embodiment of step S404. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0013] FIG. 1 is a diagram showing an example of the configuration of an information processing system according to the present invention.

[0014] As shown in FIG. 1, a client terminal 101 and a server device 102 are connected via a network 110 so as to be able to communicate with each other.

[0015] The client terminal 101 is a terminal that accepts operations from a user and displays various data acquired from the server device 102 on the display unit. The server device 102 is a terminal that executes learning processing and the like in accordance with the operations accepted by the client terminal 101.

[0016] 1 shows only two terminals, the client terminal 101 and the server device 102, but the system may include other terminals, such as those communicably connected to an external storage device that stores learning data. Also, the system may be completed by a single terminal, with the client terminal 101 performing the learning process and the like.

[0017] FIG. 2 is a block diagram showing an example of the hardware configuration of an information processing device that can be applied to the client terminal 101 and the server device 102 of the present invention.

[0018] As shown in FIG. 2, the information processing device is connected to a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, a storage device 204, a GPU 215, an input controller 205, an audio controller 206, a video controller 207, a memory controller 208, and a communication I / F controller 209 via a system bus 200.

[0019] The CPU 201 and the GPU 215 comprehensively control the devices and controllers connected to the system bus 200 .

[0020] Since the GPU 215 can perform efficient calculations by processing a larger amount of data in parallel, it is effective to use the GPU 215 for processing when performing learning multiple times using a learning model such as deep learning. Therefore, in this embodiment, the GPU 215 is used in addition to the CPU 201 for the learning process. Specifically, when a learning program including a learning model is executed, the CPU 201 and the GPU 215 cooperate to perform calculations to perform learning. Note that the learning process may be performed by only the CPU 201 or the GPU 215.

[0021] ROM202 or external memory 213 stores the BIOS (Basic Input / Output System) and OS (Operating System), which are control programs executed by CPU201, computer-readable and executable programs for realizing this information processing method, and various necessary data (including data tables).

[0022] The RAM 203 functions as a main memory, a work area, etc. for the CPU 201. The CPU 201 loads programs and the like required for executing processing from the ROM 202 or the external memory 213 into the RAM 203, and executes the loaded programs to realize various operations.

[0023] The input controller 205 controls input from input devices such as a keyboard 210 and a pointing device such as a mouse (not shown). If the input device is a touch panel, the user can issue various instructions by pressing (touching with a finger or the like) icons, cursors, or buttons displayed on the touch panel.

[0024] The touch panel may also be a touch panel capable of detecting positions touched by multiple fingers, such as a multi-touch screen.

[0025] The video controller 207 controls the display on an external output device such as a display 212. The display also includes the display of a notebook computer integrated with the main body. Note that the external output device is not limited to a display, and may be, for example, a projector. In addition, for devices capable of receiving the above-mentioned touch operation, an input device is also provided.

[0026] The video controller 207 can control a video memory (VRAM) for display control, and can use part of the RAM 203 as a video memory area, or can provide a separate dedicated video memory.

[0027] The memory controller 208 controls access to the external memory 213. The external memory may be an external storage device (hard disk) that stores a boot program, various applications, font data, user files, edited files, and various data, a flexible disk (FD), or a CompactFlash (registered trademark) memory connected to a PCMCIA card slot via an adapter.

[0028] The communication I / F controller 209 connects and communicates with external devices via a network, and executes communication control processing on the network. For example, communication using TCP / IP, telephone lines such as ISDN, and 4G and 5G mobile phone lines are possible.

[0029] The CPU 201 enables display on the display 212 by, for example, executing a process of expanding (rasterizing) an outline font into a display information area in the RAM 203. The CPU 201 also enables user instructions using a mouse cursor (not shown) on the display 212.

[0030] Next, an overview of the "processing for generating a trained model" as an example to which the present invention is applied will be described using the flowchart in Figure 3.

[0031] In step S301, learning data designated by the user via the client terminal 101 is acquired. Parameter values ​​are set according to instructions received from the user via the client terminal 101.

[0032] In step S302, a learning process is performed by executing a learning program using the learning data acquired in step S301 and the parameter values ​​that have been set, and a trained model is generated.

[0033] In step S303, the results of the learning process executed in step S302 are analyzed. Details of the process in step S303 will be described with reference to the flowchart in FIG.

[0034] In step S304, if an instruction to the effect that learning is complete is received from the user, the process of this flowchart ends (S304: YES). If an instruction to perform learning again is received (S304: NO) because a trained model with the accuracy desired by the user has not been generated, the process proceeds to step S305, where changes to parameter values ​​and learning data are received, and the learning process (S301 to S303) is performed again using the received parameter values ​​and learning data.

[0035] Even on the screen for accepting new learning data and parameter value settings, the display of input fields for parameter items with a large number of unique entries is controlled. For example, input fields for parameter items with a large number of unique entries are displayed together in one place, or they are highlighted (by adding a bold border to the input field, using a different color for the input field, or displaying the parameter item in bold or red).

[0036] In this way, it becomes possible to set parameters efficiently.

[0037] The parameter values ​​used in the learning process are stored for each learning process in the form of Learning ID: 1 for the first learning process in this flowchart, Learning ID: 2 for the second learning process, etc., and the learning processes performed from the start to the end of this flowchart are managed as a single unit.

[0038] Next, the details of the learning result analysis process in step S303 will be described with reference to the flowchart in FIG.

[0039] In step S401, the parameter values ​​used in the learning process in step S302 (parameters set in steps S301 and S305) are acquired. At this time, not only the parameter values ​​used in the most recent learning process but also the parameter values ​​used in past learning processes (from the above learning ID: 1 to the most recent learning) are acquired.

[0040] In step S402, the unique number (the number of types of parameter values ​​in the parameter item) is acquired for the parameter item, which is the item of the parameter value.

[0041] For example, for the training data file path 601 in FIG. 6, there are two types of parameter values, ". / train.csv" and ". / train_added.csv", and therefore the unique number is "2".

[0042] In step S403, a threshold value for the number of unique parameters to be sorted is acquired. The threshold value may be a preset value, or may be a value determined from the number of unique parameters for each parameter acquired in step S402 (for example, a value that identifies the top X number of unique parameter items, or the average or median of the number of unique parameters).

[0043] In step S404, the parameter items having a unique number equal to or greater than the threshold value acquired in step S403 (parameter items having a unique number that satisfies a predetermined number) are sorted in descending order of unique number.

[0044] In step S405, the color of the cell is changed for each unique value of each parameter. Details of the process in step S405 will be described with reference to FIG.

[0045] FIG. 5 is a flowchart illustrating the details of the process in step S405.

[0046] In step S501, one parameter string (parameter item and parameter value associated with the parameter item) is acquired.

[0047] In step S502, it is determined whether the parameter value acquired in step S501 is a numeric value.

[0048] If it is a numeric value (S502: YES), the process proceeds to step S503, and if it is not a numeric value (S502: NO), the process proceeds to step S505.

[0049] In step S503, the maximum and minimum values ​​of the parameter values ​​obtained in step S501 are obtained.

[0050] In step S504, the color of each parameter value is determined and drawn so as to create a gradation according to the value, with the minimum value acquired in step S503 being the lightest color and the maximum value being the darkest color.

[0051] Then, the process proceeds to step S506.

[0052] In step S505, the color of each parameter value is determined and drawn so that each unique value has a different color, and the process then proceeds to step S506.

[0053] In step S506, it is determined whether any unprocessed parameter strings remain, and if so (S506: YES), the process proceeds to step S507; if not (S506: NO), the process of this flowchart ends.

[0054] In step S507, a parameter string that has not been processed is acquired, and the process proceeds to step S502 and subsequent steps.

[0055] Returning to the explanation of Figure 4.

[0056] In step S406, the size of the column is adjusted so that parameters having a unique number that satisfies a predetermined number can fit on one screen.

[0057] In step S407, a list of parameters for which the processes in steps S404 to S406 have been performed is displayed.

[0058] In this embodiment, the list after the processing of steps S404 to S406 has been described as being displayed, but the list before the processing (for example, (A) of FIG. 6) may be displayed, and the processing of steps S404, S405, and S406 may be executed in accordance with instructions from the user. In this case, buttons and check boxes for receiving instructions to execute the processing of S404, S405, and S406 are displayed, and the processing of S404 to S406 is executed when the button is operated or the check box is checked, as one condition.

[0059] Next, the processing in step S404 will be described with reference to FIG.

[0060] FIG. 6A is a diagram showing a list of parameters before the processing of step 404 is executed.

[0061] This shows an example in which records consisting of parameter values ​​related to a training data file path 601, an evaluation data file path 602, a learning rate 603, the number of network layers 604, and an image size 605 are displayed in list format with five learning IDs: 1 to 5.

[0062] As shown in this figure, the number of unique training data file paths 601 is 2 (two types: “. / train.csv” and “. / train_added.csv”), the number of unique evaluation data file paths 602 is 1, the number of unique learning rates 603 is 3, the number of unique network layer numbers is 2, and the number of unique image sizes is 1.

[0063] FIG. 6B is a diagram showing an example of a display after the processing of step S404 has been executed.

[0064] As shown in the figure, the learning rate 603, which has the highest unique number of 3, is on the far left, and to the right of it are the training data file path 601 and the number of network layers 604, both of which have a unique number of 2. In this way, by executing the process of step S404, the parameters can be sorted in descending order of unique number.

[0065] In this way, by controlling the order of parameters according to the number of uniques (displaying them in descending order of the number of uniques, displaying parameters with a large number of uniques at the top), it becomes possible to efficiently check parameters that differ when comparing records.

[0066] In particular, when there are a large number of parameter items and they cannot all be displayed without significant scrolling, by displaying parameter items with a large number of unique numbers close to each other, it becomes possible to check the parameters with a large number of unique numbers at a glance, thereby enabling efficient checking.

[0067] Next, the processing in step S405 will be described with reference to FIG.

[0068] As shown in Figure 6 (B), the learning rate value is a number, and there are three unique values: "0.1," "0.01," and "0.001." The cell with the largest value, "0.1," is displayed in black, and the cell with the smallest value, "0.001," is displayed in white. The cell with the intermediate value, "0.01," is displayed in a light gray color. In this way, the magnitude relationship between the values ​​is displayed in a distinguishable manner using the intensity of the color.

[0069] In this embodiment, the largest value is represented by the darkest color and the smallest value by the lightest color, but the largest value may be represented by the lightest color and the smallest value by the darkest color. In this way, any representation that allows the magnitude relationship of the numerical values ​​to be intuitively grasped is acceptable.

[0070] The training data file paths shown in Figure 6(B) are not numeric, but have two unique values: ". / train.csv" and ". / train_added.csv." To make these two values ​​intuitively distinguishable, the cell for ". / train.csv" is displayed filled with a lane pattern, and the cell for ". / train_added.csv" is displayed filled with a light gray color.

[0071] In this way, by displaying the data in different display formats, such as by color-coding each unique value, it becomes possible to efficiently identify records that have the same parameter values ​​and records that have different parameter values ​​when comparing records.

[0072] Furthermore, by displaying parameter items with a large number of unique values ​​close to each other and controlling the display format for each unique value, it is possible to efficiently check whether there are any combinations of parameter values ​​that have not been tried, thereby reducing the number of missing or overlooked combinations of parameter values. This can be used as a reference for parameter settings in the next trial.

[0073] Furthermore, although not shown, the accuracy (quality) of the trained model generated by each learning job may be displayed in association with the job. This allows the user to understand what level of accuracy will be achieved with certain parameter settings, and therefore to understand which parameters have affected the accuracy of the trained model and which have not. This can be used as a reference for parameter settings in the next trial.

[0074] In particular, in AI development, learning processing is extremely time-consuming, so it is often difficult to change parameters randomly and perform comprehensive trials. Therefore, it is necessary to properly understand which parameters should be changed and which parameter combinations have not yet been implemented, and to utilize this information for the next trial. Therefore, a system is required that displays learning results and various parameter values ​​in a list, as in this embodiment, and allows efficient understanding.

[0075] Next, the processing in step S406 will be described with reference to FIG.

[0076] FIG. 7A is a diagram for explaining the state before the process of step S406 is executed. As shown in this figure, the parameter list does not fit in the window area, so the parameter value for "Number of network layers," a parameter item for which the number of uniques meets the specified number, cannot be displayed.

[0077] In contrast, Figure 7(B) shows the state after the processing of step S406 has been executed. As shown in this figure, by adjusting the width of each column (in Figure 7(B), the width of the columns for "learning rate" and "number of network layers" has been adjusted), all of the parameter items for which the number of uniques satisfies the predetermined number, namely "evaluation data file path," "learning rate," "learning data file path," and "number of network layers," are displayed in the window area.

[0078] This type of display allows you to compare, examine, and confirm parameters that are thought to have affected the training results without scrolling, making it possible to efficiently compare records (training jobs).

[0079] In this embodiment, the width is adjusted for display, but the entire list may be displayed by reducing its size.

[0080] Next, another embodiment will be described with reference to FIG.

[0081] FIG. 8 shows an example of a display in which the unique numbers are made identifiable by changing the display format of the parameter items depending on the unique numbers.

[0082] As shown in Figure 8, the parameter item with the most unique numbers (learning rate in the example in Figure 8) is displayed in the darkest color, the parameter item with the next most unique numbers, "learning data file path," is displayed in a lighter color than "learning rate," and the parameter with the next most unique numbers, "number of network layers," is displayed in an even lighter color.

[0083] This display format also makes it possible to efficiently grasp which parameter items have the most unique counts.

[0084] Next, another embodiment will be described with reference to FIG.

[0085] Figure 9 is an example in which only parameter items whose unique count meets a predetermined number are displayed. As shown in Figure 9, parameter items whose unique count does not meet the predetermined number are hidden (including a display format that narrows the display width to the extent that it can be evaluated as not being displayed), making it possible to efficiently grasp parameter items whose unique count meets the predetermined number.

[0086] While the above description assumes application to AI development, the application of the present invention is not limited to AI development. For example, it can be applied to search systems for patent documents and the like. Specifically, search items in patent searches, such as "specification," "IPC," "claims," ​​and "applicant," correspond to parameter items in the AI ​​development example, and keywords for each search item correspond to parameter values ​​in the AI ​​development example. In patent searches, appropriate search formulas are created through trial and error while changing various keywords. Therefore, applying the present invention makes it possible to efficiently understand what search results are obtained by changing keywords and which items have been changed, thereby improving the efficiency of search operations.

[0087] The present invention can also be applied to image processing, in which various parameters (brightness, saturation, white balance, contrast, sharpness, filter effects, cropping, resolution conversion, blurring effect, etc.) are appropriately changed to adjust the image to a desired one, and by applying the present invention, it is possible to understand what kind of image will be produced under what settings, what settings have been tried, and what settings have not been tried, thereby realizing efficient image processing.

[0088] In this way, the present invention is a technology that can be applied to a system that requires trial and error while changing data for various items.

[0089] The present invention can be embodied as, for example, a system, an apparatus, a method, a program, or a recording medium, etc. Specifically, the present invention may be applied to a system consisting of multiple devices, or may be applied to an apparatus consisting of a single device.

[0090] Furthermore, the program of the present invention is a program that enables a computer to execute the processing methods of the flowcharts shown in Figures 3 to 5, and the storage medium of the present invention stores a program that enables a computer to execute the processing methods of Figures 3 to 5. Note that the program of the present invention may be a program for each processing method of each device in Figures 3 to 5.

[0091] As described above, it goes without saying that the object of the present invention can also be achieved by supplying a recording medium on which a program that realizes the functions of the above-mentioned embodiments is recorded to a system or device, and having the computer (or CPU or MPU) of that system or device read and execute the program stored on the recording medium.

[0092] In this case, the program itself read from the recording medium will realize the novel functions of the present invention, and the recording medium on which the program is recorded will constitute the present invention.

[0093] Examples of recording media for supplying the program include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, and silicon disks.

[0094] Furthermore, it goes without saying that not only are the functions of the above-mentioned embodiments realized by the computer executing a program it has read, but also cases are included in which an OS (operating system) running on the computer performs some or all of the actual processing based on the instructions of the program, and the functions of the above-mentioned embodiments are realized through that processing.

[0095] Furthermore, it goes without saying that this also includes cases where a program read from a recording medium is written into a memory provided on a function expansion board inserted into a computer or a function expansion unit connected to the computer, and then a CPU or the like provided on the function expansion board or function expansion unit performs some or all of the actual processing based on the instructions of the program code, thereby realizing the functions of the above-mentioned embodiments.

[0096] Furthermore, the present invention may be applied to a system consisting of multiple devices, or to a device consisting of a single device. It goes without saying that the present invention can also be applied to a case where the present invention is achieved by supplying a program to a system or device. In this case, the system or device can enjoy the effects of the present invention by reading a recording medium containing a program for achieving the present invention into the system or device.

[0097] Furthermore, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or device can enjoy the effects of the present invention. Note that the present invention also includes configurations that combine the above-mentioned embodiments and their modified examples. [Explanation of symbols]

[0098] 101 client terminals 102 Server device 110 Network

Claims

1. It includes a display control means that controls the display of information consisting of values ​​related to multiple items in a list format, The display control means is If the value is a number, the display format of the value related to the item is controlled to display it in a format that allows the magnitude relationship of the numbers to be identified. If the aforementioned value is not a number, control the display format of the value related to the aforementioned item so that it is displayed in a different format for each unique value. An information processing system characterized by the following.

2. The values ​​related to the aforementioned multiple items are parameters used to generate the trained model. Control the system to display information relating to the accuracy of the trained model in association with the identification information of the trained model. The information processing system according to claim 1, characterized by the following:

3. The display control means controls items whose uniqueness count is a predetermined number to be displayed in an order corresponding to the uniqueness count of the values ​​related to that item. The information processing system according to claim 1, characterized by the following:

4. The values ​​related to the aforementioned multiple items are parameters used to generate the trained model. When the display control means receives an instruction to retrain the trained model, it controls the screen that accepts the setting of values ​​for the items to display the input fields for the values ​​of each item in an identifiable manner according to the unique number of values ​​for each item. The information processing system according to claim 1, characterized by the following:

5. The display control means controls the display of the items in an order corresponding to the number of unique values ​​related to the items. The information processing system according to claim 1, characterized by the following:

6. The display control means controls the display to show items in the higher position if the number of unique values ​​related to the item satisfies a predetermined number. The information processing system according to claim 1, characterized by the following:

7. The display control means controls the items to be displayed in descending order of the number of unique values ​​related to the item. The information processing system according to claim 1, characterized by the following:

8. The display control means controls the display of items whose unique count does not meet a predetermined number. The information processing system according to claim 1, characterized by the following:

9. The values ​​related to the aforementioned multiple items are parameters used to generate the trained model. The information processing system according to claim 1, characterized by the following:

10. The display control means controls the display of each item by varying the intensity of its color according to the number of unique values ​​for each item. The information processing system according to claim 1, characterized by the following:

11. The display control means controls the display of the color associated with the first value to be darker than the color associated with the second value, if the value is a numerical value and the first value is greater than the second value. The information processing system according to claim 1, characterized by the following:

12. The information consisting of values ​​relating to the plurality of items is a list of records consisting of the plurality of items. The information processing system according to claim 1, characterized by the following:

13. The display control means of the information processing system, It includes a display control process that controls the display of information consisting of values ​​related to multiple items in a list format, In the display control step, If the value is a number, the display format of the value related to the item is controlled to display it in a format that allows the magnitude relationship of the numbers to be identified. If the aforementioned value is not a number, control the display format of the value related to the aforementioned item so that it is displayed in a different format for each unique value. An information processing method characterized by the following.

14. A program for causing at least one computer to function as one of the means described in any one of claims 1 to 12.

Images