Information processing apparatus and method, and program

Abstract

To improve a technique relating to application development of machine learning.SOLUTION: An information processing apparatus includes: an input unit 13 which accepts selection of one of a plurality of process flows relating to machine learning; and a control unit 11 which, on the basis of the selected process flow, controls a display unit 14 to selectably display at least one process corresponding to the selected process flow.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a method, and a program. [Background technology]

[0002] Conventionally, technologies have been proposed that allow users to easily perform tasks from image processing settings to data utilization without having to acquire programming skills (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2024-004266 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional technology, the development of machine learning applications has not been considered. However, the development of machine learning applications is complex, involving various processes, and application development is not easy. As such, there is room for improvement in the technology related to the development of machine learning applications.

[0005] In view of the above circumstances, the purpose of the present disclosure is to improve techniques related to the development of machine learning applications. [Means for solving the problem]

[0006] (1) An information processing device according to an embodiment of the present disclosure includes: an input unit that accepts a selection of one of a plurality of process flows related to machine learning; a control unit that controls the display unit to selectably display at least one process corresponding to the selected process flow based on the selected process flow; It has.

[0007] (2) An information processing device according to an embodiment of the present disclosure is the information processing device according to (1), The plurality of process flows includes an annotation process flow, a training process flow, and a runtime process flow.

[0008] (3) An information processing device according to an embodiment of the present disclosure is the information processing device according to (2), The process corresponding to the annotation process flow includes pre-AI processing and annotation.

[0009] (4) An information processing device according to an embodiment of the present disclosure is the information processing device according to (2) or (3), The processes corresponding to the learning process flow include pre-processing, AI, and post-processing.

[0010] (5) An information processing device according to an embodiment of the present disclosure is the information processing device according to any one of (2) to (4), The processes corresponding to the runtime process flow include input, pre-processing, AI, post-processing, and output.

[0011] (6) A method according to an embodiment of the present disclosure is a method executed by an information processing device, receiving a selection of one of a plurality of process flows related to machine learning; displaying, based on the selected process flow, at least one process corresponding to the selected process flow in a selectable manner; Includes.

[0012] (7) A program according to an embodiment of the present disclosure includes: On the computer, receiving a selection of one of a plurality of process flows related to machine learning; displaying, based on the selected process flow, at least one process corresponding to the selected process flow in a selectable manner; Execute the following. [Effects of the Invention]

[0013] According to one embodiment of the present disclosure, techniques for developing machine learning applications are improved. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of an information processing device. [Figure 2] 10 is a flowchart illustrating an operation of the information processing device. [Figure 3] 10 is an example of a selection screen. [Figure 4] 10 is an example of a user interface that is displayed when an annotation process flow is selected. [Figure 5] 10 is an example of a user interface related to an annotation process flow. [Figure 6] 10 is an example of a user interface that is displayed when a learning process flow is selected. [Figure 7] 10 is an example of a user interface related to a learning process flow. [Figure 8] 1 is an example of a user interface that is displayed when a runtime process flow is selected. [Figure 9] 1 is an example of a user interface for a runtime process flow. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, embodiments of the present disclosure will be described.

[0016] (Outline of the embodiment) First, an overview of this embodiment will be described, and details will be described later. An information processing device 10 accepts a selection of one of a plurality of process flows related to machine learning. Based on the selected process flow, the information processing device 10 then controls a display unit to selectably display at least one process corresponding to the selected process flow.

[0017] As described above, according to this embodiment, the information processing device 10 accepts the selection of one of a plurality of process flows and displays at least one process corresponding to each process flow in a selectable manner. Therefore, the selectable processes in each process flow become clear, facilitating application development, thereby improving the technology related to machine learning application development.

[0018] Next, each component of the information processing device 10 will be described in detail. (Configuration of information processing device) As shown in FIG. 1, the information processing device 10 includes a control unit 11, a storage unit 12, an input unit 13, a display unit 14, and a communication unit 15.

[0019] The control unit 11 includes at least one processor, at least one dedicated circuit, or a combination thereof. The processor is a general-purpose processor such as a central processing unit (CPU) or a graphics processing unit (GPU), or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, a field-programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The control unit 11 executes processes related to the operation of the information processing device 10 while controlling each unit of the information processing device 10.

[0020] The storage unit 12 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The semiconductor memory is, for example, a random access memory (RAM) or a read only memory (ROM). The RAM is, for example, a static random access memory (SRAM) or a dynamic random access memory (DRAM). The ROM is, for example, an electrically erasable programmable read only memory (EEPROM). The storage unit 12 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 12 stores data used in the operation of the information processing device 10 and data obtained by the operation of the information processing device 10.

[0021] The input unit 13 includes at least one input interface. The input interface is, for example, a physical key, a capacitance key, a pointing device, or a touch screen integrated with a display. The input interface may also be, for example, a sound sensor that accepts voice input, or a camera that accepts gesture input. The input unit 13 accepts an operation to input data used for the operation of the information processing device 10. The input unit 13 may be connected to the information processing device 10 as an external input device instead of being provided in the information processing device 10. Any connection method may be used, for example, a Universal Serial Bus (USB), a High-Definition Multimedia Interface (HDMI) (registered trademark), or Bluetooth (registered trademark).

[0022] The display unit 14 includes at least one display output interface. The output interface is, for example, a display. The display is, for example, an LCD (liquid crystal display) or an organic EL (electro luminescence) display. The display unit 14 displays and outputs data obtained by the operation of the information processing device 10. The display unit 14 may be connected to the information processing device 10 as an external output device instead of being provided in the information processing device 10. Any connection method may be used, for example, USB, HDMI (registered trademark), or Bluetooth (registered trademark).

[0023] The communication unit 15 includes at least one external communication interface. The communication interface may be either a wired communication interface or a wireless communication interface. In the case of wired communication, the communication interface is, for example, a LAN (Local Area Network) interface or a USB (Universal Serial Bus). In the case of wireless communication, the communication interface is, for example, an interface compatible with mobile communication standards such as LTE (Long Term Evolution), 4G (4th generation), or 5G (5th generation), or an interface compatible with short-range wireless communication such as Bluetooth (registered trademark). The communication unit 15 receives data used in the operation of the information processing device 10 and transmits data obtained by the operation of the information processing device 10.

[0024] The functions of the information processing device 10 are realized by executing a program according to this embodiment on a processor corresponding to the information processing device 10. That is, the functions of the information processing device 10 are realized by software. The program causes a computer to execute the operations of the information processing device 10, thereby causing the computer to function as the information processing device 10. That is, the computer functions as the information processing device 10 by executing the operations of the information processing device 10 in accordance with the program.

[0025] In this embodiment, the program can be recorded on a computer-readable recording medium. The computer-readable recording medium includes non-transitory computer-readable media, such as a magnetic recording device, an optical disc, a magneto-optical recording medium, or a semiconductor memory. The program can be distributed, for example, by selling, transferring, or lending a portable recording medium, such as a DVD (digital versatile disc) or a CD-ROM (compact disc read only memory), on which the program is recorded. The program can also be distributed by storing the program in the storage of an external server and transmitting the program from the external server to another computer. The program can also be provided as a program product.

[0026] Some or all of the functions of the information processing device 10 may be implemented by a dedicated circuit equivalent to the control unit 11. In other words, some or all of the functions of the information processing device 10 may be implemented by hardware.

[0027] (Operation of information processing device) The operation of the information processing device 10 according to this embodiment will be described with reference to FIG.

[0028] Step S100: The input unit 13 of the information processing device 10 accepts selection of one of multiple process flows related to machine learning. The multiple process flows include an annotation process flow, a learning process flow, and a runtime process flow. The annotation process flow is a process flow related to the process of creating a dataset used for training a machine learning model. The learning process flow is a process flow related to the process of creating a learning model based on the dataset created by the annotation process flow. In other words, the learning process flow is a process flow related to the process of creating a new AI. The runtime process flow is a process flow related to the process of creating a desired estimation processing program using the learning model created by the learning process flow. In other words, the runtime process flow is a process flow related to the process of creating an operation flow related to the created new AI.

[0029] When accepting the selection of one of the multiple process flows related to machine learning, the control unit 11 may display a selection screen on the display unit 14. FIG. 3 is an example of such a selection screen 100. The selection screen 100 includes objects 110, 120, and 130 corresponding to the multiple process flows related to machine learning. The objects 110, 120, and 130 correspond to an annotation process flow, a learning process flow, and a runtime process flow, respectively. The input unit 13 accepts the selection of one of the multiple process flows related to machine learning by accepting a selection operation, such as a click or tap, by the user on the objects 110, 120, and 130.

[0030] Step S200: Based on the process flow selected in step S100, the control unit 11 of the information processing device 10 controls the display unit 14 to selectably display at least one process corresponding to the selected process flow. The display unit 14 selectably displays at least one process corresponding to the selected process flow.

[0031] 4 is an example of a user interface 111 that is displayed when the annotation process flow is selected. The processes corresponding to the annotation process flow include "pre-AI processing" and "annotation."

[0032] "Preliminary AI processing," included in the process corresponding to the annotation process flow, is a process of annotating using pre-created AI. The pre-created AI includes AI (learning models) created by the learning process flow described below. For example, "pre-AI processing" includes a process of automatically annotating image data using a pre-created machine learning model.

[0033] The "annotation" included in the process corresponding to the annotation process flow is a process of manually annotating an image. For example, "annotation" includes a process in which a user visually determines which of multiple products A in an image are defective and specifies the defective product by clicking on it in the image.

[0034] As shown in FIG. 4, the user interface 111 includes a start object 112 and an object 113. The start object 112 is an object that indicates the starting location of the annotation process flow. Processes connected to the start object 112 are executed in order. The object 113 indicates an arbitrary process added by the user. When the object 113 is placed, a pull-down menu 114 for selecting the process of the object 113 is displayed. That is, the control unit 11 controls the display unit 14 to selectably display at least one process corresponding to the selected process flow (here, the annotation process flow). Specifically, the display unit 14 displays "pre-AI processing" and "annotation," which are processes corresponding to the annotation process flow, as items in the pull-down menu 114. The user can determine the process of the object 113 by selecting one of the processes displayed in the pull-down menu 114.

[0035] Figure 5 is an example of a completed annotation process flow. As shown in Figure 5, by connecting processes such as pre-AI processing and annotation to a start object in order, an annotation process flow can be created without coding. Various settings and parameter changes related to each process can be adjusted as appropriate using the settings menu 115.

[0036] 6 is an example of a user interface 121 that is displayed when a learning process flow is selected. The processes that correspond to the learning process flow include "pre-processing," "AI," and "post-processing."

[0037] "Preprocessing," included in the processes corresponding to the learning process flow, is a process in which predetermined processing is performed on images before AI learning. Predetermined processing includes, for example, cutting out an image and processing it to only the ROI (Region of Interest), binarization, color change, positioning, etc. In image processing, computer vision, etc., the ROI refers to a partial region of interest in an image to which you want to apply filtering, recognition, etc. In other words, the ROI refers to the area you want to cut out. Regarding positioning, for example, in the case of comparative inspection, a reference image containing a component is registered. In this case, accurate comparative inspection cannot be performed unless the positions of the components in the image input during inspection and the reference image match. Therefore, it is preferable to perform a process to align the positions of the components. This process is called positioning.

[0038] The "AI" included in the process corresponding to the learning process flow is an AI learning process. In other words, "AI" is a process for training a machine learning model. The machine learning model may be any learning model, for example, a machine learning model built based on a decision tree. Examples of machine learning models built based on a decision tree include, but are not limited to, Light GBM and XGBoost. Alternatively, the machine learning model may be a model generated based on a machine learning algorithm such as a convolutional neural network (CNN), a recurrent neural network (RNN), or other deep learning.

[0039] "Post-processing," included in the process corresponding to the learning process flow, is a process of applying specified processing to images after AI learning. "Post-processing" also includes, for example, onnx conversion. The model file created after learning is a block of numerical values ​​that matches the model structure. The model structure is defined in Python code. During inference, a process is carried out in which the model is constructed from the Python code and the numerical values ​​of the learning results are reflected in the model. Since this can only be used in a Python environment, it is preferable to convert the model structure and the numerical values ​​of the learning results together into a format called onnx. This conversion process allows the model file to be used in a variety of environments.

[0040] As shown in FIG. 6 , the user interface 121 includes a start object 122 and an object 123. The start object 122 is an object that indicates the start location of the learning process flow. Processes connected to the start object 122 are executed in order. The object 123 indicates an arbitrary process added by the user. When the object 123 is placed, a pull-down menu 124 for selecting the process of the object 123 is displayed. That is, the control unit 11 controls the display unit 14 to selectably display at least one process corresponding to the selected process flow (here, the learning process flow). Specifically, the display unit 14 displays "pre-processing," "AI," and "post-processing," which are processes corresponding to the learning process flow, as items in the pull-down menu 124. The user can determine the process of the object 123 by selecting one of the processes displayed in the pull-down menu 124.

[0041] Figure 7 is an example of a completed learning process flow. As shown in Figure 7, by connecting processes such as AI to a start object in order, a learning process flow can be created without coding. Various settings and parameter changes related to each process can be adjusted as needed using the setting menu 125.

[0042] 8 is an example of a user interface 131 that is displayed when a runtime process flow is selected. The processes that correspond to the runtime process flow include "Input," "Pre-processing," "AI," "Post-processing," and "Output."

[0043] The "input" included in the process corresponding to the runtime process flow is a process for inputting an image. In the "input" process, the camera controls the camera to capture an image. In the case of handling saved images, the process reads a file.

[0044] The "preprocessing" included in the processes corresponding to the runtime process flow is a process that performs predetermined processing on the image before AI inference. The predetermined processing includes, for example, cutting out the image and processing it to only the ROI portion, binarization, color change, positioning, etc.

[0045] The "AI" included in the process corresponding to the runtime process flow is an AI inference process, which uses trained AI to perform processing on input images.

[0046] "Post-processing," included in processes corresponding to the runtime process flow, is a process that applies predetermined processing to images after AI inference. Processing includes combining multiple result images. Images may be combined horizontally or vertically. For example, if you want to count boards stacked one meter high, not all of the boards can fit in a single image. Therefore, such objects may be photographed in segments. Although the images are segmented, this type of combining process is performed when you want to treat the stacked group as a single inspection object. Images may be combined with mask images, etc.

[0047] The "output" included in the processes corresponding to the runtime process flow is a process that outputs the results of AI inference or post-processing. Specifically, output includes image display, image saving, and display and saving of AI processing values ​​(number, pixel-based area, etc.).

[0048] As shown in FIG. 8 , the user interface 131 includes a start object 132 and an object 133. The start object 132 is an object that indicates the start location of the learning process flow. Processes connected to the start object 132 are executed in order. The object 133 indicates an arbitrary process added by the user. When the object 133 is placed, a pull-down menu 134 for selecting the process of the object 133 is displayed. That is, the control unit 11 controls the display unit 14 to selectably display at least one process corresponding to the selected process flow (here, the runtime process flow). Specifically, the display unit 14 displays “input,” “pre-processing,” “AI,” “post-processing,” and “output,” which are processes corresponding to the runtime process flow, as items in the pull-down menu 134. The user can determine the process of the object 133 by selecting one of the processes displayed in the pull-down menu 134.

[0049] Figure 9 is an example of a completed runtime process flow. As shown in Figure 9, input, AI, output, and other processes are connected to a start object in order, allowing a runtime process flow to be created without coding. Various settings and parameter changes for each process can be adjusted as needed using the settings menu 135.

[0050] As described above, the information processing device 10 according to this embodiment accepts the selection of one of a plurality of process flows, and displays at least one selectable process corresponding to each process flow.

[0051] This configuration improves the technology related to machine learning application development by clarifying the selectable processes in each process flow and facilitating application development.

[0052] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art may make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each component or step can be rearranged so as not to be logically inconsistent, and multiple components or steps can be combined or divided into one.

[0053] For example, in the above-described embodiment, the configuration and operation of the information processing device 10 may be distributed among a plurality of computers that can communicate with each other.

[0054] In the present embodiment, the annotation process flow, learning process flow, and runtime process flow shown in Figures 5, 7, and 9 all show examples in which the processes are arranged in series, but this is not limiting. For example, the processes in the process flow may be arranged in parallel. [Explanation of symbols]

[0055] 10. Information processing equipment 11 Control section 12 Storage section 13 Input section 14 Display section 15 Communications Department 100 Selection Screen 110, 120, 130 objects 111, 121, 131 User Interface 112, 122, 132 Starting Objects 113, 123, 133 objects 114, 124, 134 pull-down menu 115, 125, 135 Settings Menu

Claims

1. an input unit that accepts a selection of one of a plurality of process flows related to machine learning; a control unit that controls the display unit to selectably display at least one process corresponding to the selected process flow based on the selected process flow; An information processing device having the above.

2. The information processing apparatus according to claim 1 , wherein the plurality of process flows include an annotation process flow, a learning process flow, and a runtime process flow.

3. The information processing device according to claim 2 , wherein the process corresponding to the annotation process flow includes a pre-AI process and an annotation.

4. The information processing device according to claim 2 , wherein the processes corresponding to the learning process flow include pre-processing, AI, and post-processing.

5. The information processing device according to claim 2 , wherein the processes corresponding to the runtime process flow include input, pre-processing, AI, post-processing, and output.

6. A method executed by an information processing device, receiving a selection of one of a plurality of process flows related to machine learning; displaying, based on the selected process flow, at least one process corresponding to the selected process flow in a selectable manner; A method comprising:

7. On the computer, receiving a selection of one of a plurality of process flows related to machine learning; displaying, based on the selected process flow, at least one process corresponding to the selected process flow in a selectable manner; A program that executes the following.

Images