Image processing device, program, and image processing method
The image processing apparatus automates preprocessing and command generation using AI models to address the challenges of manual adjustment in conventional segmentation, achieving efficient and accurate segmentation results.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-01-22
- Publication Date
- 2026-04-09
AI Technical Summary
Conventional image segmentation techniques require manual adjustment of preprocessing, model commands, and parameters, making it difficult to achieve suitable segmentation results.
An image processing apparatus and method that automates the adjustment of preprocessing, execution commands, and parameter setting using AI models to facilitate easy acquisition of suitable segmentation results.
Enables easy and accurate segmentation results by automating preprocessing and command generation, supporting various input forms and allowing for easy correction and refinement of classification results.
Smart Images
Figure JP2025001928_09042026_PF_FP_ABST
Abstract
Description
Image Processing Apparatus, Program, and Image Processing Method
[0001] The present disclosure relates to an image processing apparatus, a program, and an image processing method.
[0002] Techniques have been developed for performing segmentation using a model learned for performing segmentation that classifies regions from an image (see, for example, Patent Document 1).
[0003] International Publication No. 2020 / 092590
[0004] However, in the conventional technique, it is necessary for the user to appropriately adjust preprocessing of the image input to the model, commands input to the model, parameters set in the model, etc., and it may be difficult to obtain a suitable segmentation result.
[0005] Therefore, one or more aspects of the present disclosure aim to enable easy acquisition of a suitable segmentation result.
[0006] An image processing apparatus according to one aspect of the present disclosure inputs image data and request data indicating a request to classify an object from the image data into a generation AI model, thereby obtaining parameters to be set in a segmentation model, input image data to be input to the segmentation model, and elements to be classified using the segmentation model. A preprocessing unit, a execution command generation unit that generates an execution command for setting the parameters in the segmentation model, inputting the input image data to the segmentation model, and classifying the elements in the input image data, and according to the execution command, using the segmentation model And a segmentation unit that performs segmentation to classify the elements and obtains a classification result.
[0007] A program according to one aspect of the present disclosure is characterized in that a computer functions as a preprocessing unit that inputs image data and request data indicating a request to classify objects from the image data into a generating AI model to acquire parameters to be set in a segmentation model, input image data to be input into the segmentation model, and elements to be classified using the segmentation model; an execution command generation unit that sets the parameters in the segmentation model, inputs the input image data into the segmentation model, and generates an execution command to classify the elements in the input image data; and a segmentation unit that acquires a classification result by performing segmentation to classify the elements from the input image data using the segmentation model according to the execution command.
[0008] An image processing method according to one aspect of the present disclosure is characterized by inputting image data and request data indicating a request to classify objects from the image data into a generating AI model to obtain parameters to be set in a segmentation model, input image data to be input into the segmentation model, and elements to be classified using the segmentation model; setting the parameters in the segmentation model; inputting the input image data into the segmentation model; generating an execution command to classify the elements in the input image data; and performing segmentation to classify the elements from the input image data using the segmentation model according to the execution command, thereby obtaining a classification result.
[0009] According to one or more aspects of this disclosure, suitable segmentation results can be easily obtained.
[0010] This is a schematic block diagram showing the configuration of the image processing apparatus according to Embodiment 1. (A) to (D) are schematic diagrams illustrating an example of processing in the post-processing unit. This is a schematic block diagram showing the configuration of the PC. This is a flowchart showing the operation of the image processing apparatus according to Embodiment 1. This is a schematic block diagram showing the configuration of the image processing apparatus according to Embodiment 2. This is a flowchart showing the operation of the image processing apparatus according to Embodiment 2. This is a schematic block diagram showing the configuration of the image processing apparatus according to Embodiment 3. This is a flowchart showing the operation of the image processing apparatus according to Embodiment 3.
[0011] Embodiment 1. Figure 1 is a block diagram schematically showing the configuration of the image processing device 100 according to Embodiment 1. The image processing device 100 comprises a data acquisition unit 101, a pre-processing unit 102, an execution command generation unit 103, a segmentation unit 104, a post-processing unit 105, and an output unit 106.
[0012] The data acquisition unit 101 acquires image data showing an image containing the object to be classified, and request data indicating a request to classify the object from the image data. The request data can be any data that can indicate a request to classify the object from the image data. The method of indicating the object can be anything; for example, it may be a string of characters indicating the object, a region in the image, or a point in the image. Here, as an example, it is assumed that the preprocessing unit 102 uses a trained model, which is a multimodal generative AI (Artificial Intelligence) model such as GPT (Generative Pre-trained Transformer)-4V (Vision). In this case, the request data becomes a prompt, which is text data requesting the classification of an object from the image. The acquired image data and request data are provided to the preprocessing unit 102.
[0013] The preprocessing unit 102 uses at least one of the image data from the data acquisition unit 101 and the requested data to perform preprocessing in order to enable segmentation in the segmentation unit 104.
[0014] For example, if the contrast of the image shown in the image data is low and the subject is not clear, the preprocessing unit 102 may perform a process to increase the contrast of the image as a preprocessing step. Also, if the brightness of the image shown in the image data is low and the subject is not clear, the preprocessing unit 102 may perform a process to increase the brightness of the image as a preprocessing step. The image data showing the image that has been preprocessed in this manner is provided to the execution command generation unit 103 as input image data. If the preprocessing unit 102 does not perform any preprocessing on the image shown in the image data, it provides the image data from the data acquisition unit 101 to the execution command generation unit 103 as input image data.
[0015] Furthermore, if the request data indicates multiple targets, the preprocessing unit 102 may divide the multiple targets into individual targets and identify each target as an element. In other words, if the request data indicates multiple targets, the preprocessing unit 102 identifies multiple elements. The elements are data that identifies the targets in a format that can be input into the segmentation model used by the segmentation unit 104.
[0016] Furthermore, the preprocessing unit 102 may, for example, translate the string indicating the target shown in the request data into that specific language and identify the translated string as an element if the segmentation model used by the segmentation unit 104 has more training data in a particular language, such as English, than in other languages, and if identifying the target in a specific language results in higher segmentation accuracy.
[0017] In addition, the preprocessing unit 102 generates parameters to be set in the segmentation model used by the segmentation unit 104, based on at least one of the image data and the request data, as a preprocessing step.
[0018] Here, the preprocessing unit 102 can perform preprocessing using a trained model, which is a generation AI model. For example, the preprocessing unit 102 can automatically perform the necessary preprocessing according to at least one of the image data and the request data simply by inputting the image data and the request data into the trained model. In other words, the trained model automatically determines the necessary preprocessing according to the image data and the request data and executes the necessary preprocessing on at least one of the image data and the request data. As a result, the preprocessing unit 102 inputs the image data and the request data into the trained model and obtains the input image data, elements, and parameters from the trained model.
[0019] The acquired input image data, elements, and parameters are provided to the execution command generation unit 103. If the input image data differs from the image data provided by the data acquisition unit 101, the pre-processing unit 102 provides the image data to the post-processing unit 105.
[0020] Furthermore, in order to ensure that the preprocessing in the preprocessing unit 102 is performed appropriately, the request data may include examples of preferred preprocessing results, such as methods like Few-Shot Learning or In-Context Learning. The request data may also include identification information for identifying the segmentation model used by the segmentation unit 104.
[0021] Furthermore, the trained model used by the preprocessing unit 102 may be stored in a memory unit (not shown) provided in the image processing device 100, or it may be stored in another device such as a cloud. If the trained model is stored in another device, the preprocessing unit 102 can access that device via a communication unit (not shown) provided in the image processing device 100 and use the trained model.
[0022] The execution command generation unit 103 receives input image data, elements, and parameters from the preprocessing unit 102, sets the parameters in the segmentation model, inputs the input image data into the segmentation model, and generates execution commands to classify the elements in the input image data. If multiple elements are received from the preprocessing unit 102, the execution command generation unit 103 generates an execution command for each element. In other words, the execution command generation unit 103 generates multiple execution commands corresponding to multiple elements. The execution command generation unit 103 provides the generated execution commands and input image data to the segmentation unit 104.
[0023] The execution command generation unit 103 may generate execution commands using a trained model such as a generation AI. In this case, the trained model may be stored in a memory unit (not shown) provided in the image processing device 100, or it may be stored in another device such as the cloud. If the trained model is stored in another device, the execution command generation unit 103 can access that device via a communication unit (not shown) provided in the image processing device 100 and use the trained model.
[0024] The segmentation unit 104 receives an execution command and input image data from the execution command generation unit 103, and performs segmentation to classify elements from the input image data using a segmentation model according to the execution command. Specifically, the segmentation unit 104 sets the parameters indicated by the execution command in the segmentation model and inputs instructions to the segmentation model to classify the elements indicated by the execution command from the input image data. As a result, the segmentation unit 104 can obtain the classification result as the segmentation result from the segmentation model.
[0025] Furthermore, if the segmentation unit 104 receives multiple execution commands from the execution command generation unit 103, it obtains multiple classification results corresponding to each command. The segmentation unit 104 then provides the obtained classification results to the post-processing unit 105.
[0026] The segmentation model may be stored in a memory unit (not shown) provided in the image processing device 100, or it may be stored in another device such as a cloud. If the segmentation model is stored in another device, the segmentation unit 104 can access that device via a communication unit (not shown) provided in the image processing device 100 and use the segmentation model.
[0027] The post-processing unit 105 receives the classification result from the segmentation unit 104 and performs post-processing on the classification result that corresponds to the pre-processing performed by the pre-processing unit 102.
[0028] For example, if the request data contains multiple objects, in other words, if the request data is a request to classify multiple objects, the pre-processing unit 102 acquires multiple elements corresponding to those multiple objects. The execution command generation unit 103 then generates multiple execution commands corresponding to each of those multiple elements, and the segmentation unit 104 acquires multiple classification results by performing segmentation according to each of those multiple execution commands. As a result, the post-processing unit 105 is given multiple classification results. In such a case, the post-processing unit 105 generates an output classification result that shows the multiple classification results using a single image. In this case, it is desirable that the single image is an image represented by image data acquired by the data acquisition unit 101.
[0029] Specifically, as shown in Figure 2(A), when classifying a circular shape 121 and a triangular shape 122 as objects from image 120, the preprocessing unit 102 identifies multiple elements by dividing the multiple objects. Here, data representing "circle" and "triangle" are identified as elements, respectively. In such cases, the segmentation unit 104 provides multiple classification results. For example, classification result 123 as shown in Figure 2(B) and classification result 124 as shown in Figure 2(C) are provided.
[0030] In the classification result 123 shown in Figure 2(B), the circular shape 121 is classified, and in the classification result 124 shown in Figure 2(C), the triangular shape 122 is classified. Therefore, the post-processing unit 105 combines the multiple classification results into one. For example, as shown in Figure 2(D), the post-processing unit 105 generates a single image representing the multiple objects shown in the multiple classification results 123 and 124 as the output classification result 125. In the output classification result 125 shown in Figure 2(D), the circular shape 121 and the triangular shape 122 are classified.
[0031] Furthermore, if the image data acquired by the data acquisition unit 101 is different from the input image data acquired by the pre-processing unit 102, the post-processing unit 105 uses the image data acquired by the data acquisition unit 101 to generate an output classification result that shows the classification result from the segmentation unit 104.
[0032] For example, if the preprocessing unit 102 performs image processing on the image data, such as changing the contrast or brightness, the preprocessing unit 102 generates the output classification result by superimposing the classification result from the segmentation unit 104 onto the image shown by the image data, thereby representing the classification result from the segmentation unit 104 in the image shown by the image data.
[0033] Furthermore, if neither the preprocessing unit 102 has performed segmentation of multiple objects nor image processing, the postprocessing unit 105 will use the classification result from the segmentation unit 104 as the output classification result. The postprocessing unit 105 will then provide the output classification result to the output unit 106.
[0034] The output unit 106 outputs the output classification result. For example, the output unit 106 may display the output classification result in a predetermined format on a display unit (not shown) provided on the image processing device 100, or it may transmit the output classification result to another device via a communication unit (not shown) provided on the image processing device 100.
[0035] The image processing apparatus 100 described above can be implemented by a computer such as the PC 10 shown in Figure 3. The PC 10 includes storage 11 such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive), memory 12, a processor 13 such as a CPU (Central Processing Unit), a communication interface 14 such as a NIC (Network Interface Card), an input interface 15 such as a keyboard and mouse, and a display 16.
[0036] For example, the data acquisition unit 101, pre-processing unit 102, execution command generation unit 103, segmentation unit 104, post-processing unit 105, and output unit 106 can be implemented by loading a program stored in storage 11 into memory 12 and having the processor 13 execute that program. A communication unit (not shown) can be implemented by a communication interface 14. A display unit (not shown) can be implemented by a display 16.
[0037] The program may be downloaded to storage 11 via a reader / writer (not shown) from a recording medium (not shown), or via a communication interface 14 from a network, and then loaded into memory 12 and executed by processor 13. Alternatively, it may be loaded directly into memory 12 via a reader / writer from a recording medium, or via a communication interface 14 from a network, and executed by processor 13. In other words, the program may be provided by a computer program product such as a recording medium.
[0038] FIG. 4 is a flowchart showing the operation of the image processing apparatus 100 according to Embodiment 1. First, the data acquisition unit 101 acquires image data indicating an image including an object to be classified, and request data indicating at least the object (S10).
[0039] The preprocessing unit 102 receives the image data and the request data from the data acquisition unit 101, and executes preprocessing for effectively performing segmentation in the segmentation unit 104 using at least one of them (S11). As a result, the preprocessing unit 102 acquires input image data, elements, and parameters.
[0040] The execution command generation unit 103 receives the input image data, elements, and parameters from the preprocessing unit 102, sets the parameters in the segmentation model, and generates an execution command for classifying the elements from the input image data (S12).
[0041] The segmentation unit 104 receives the execution command and the input image data from the execution command generation unit 103, and executes segmentation for classifying an object corresponding to an element from the input image data according to the execution command (S13).
[0042] The postprocessing unit 105 receives the classification result from the segmentation unit 104, and executes postprocessing corresponding to the preprocessing performed by the preprocessing unit 102 on the classification result (S14).
[0043] Then, the output unit 106 outputs an output classification result that is the classification result after the postprocessing is performed by the postprocessing unit 105 (S15).
[0044] As described above, according to the present embodiment, since image, execution command, and parameter adjustment are automatically performed, a suitable classification result can be easily obtained. Further, according to the present embodiment, multimodal automatic segmentation assuming various input forms can be realized.
[0045] Embodiment 2. FIG. 5 is a block diagram schematically showing the configuration of an image processing apparatus 200 according to Embodiment 2. The image processing apparatus 200 includes a data acquisition unit 201, a preprocessing unit 202, an execution command generation unit 203, a segmentation unit 104, a postprocessing unit 105, an output unit 106, and a history storage unit 207.
[0046] The segmentation unit 104, the postprocessing unit 105, and the output unit 106 of the image processing apparatus 200 in Embodiment 2 are the same as the segmentation unit 104, the postprocessing unit 105, and the output unit 106 of the image processing apparatus 100 in Embodiment 1.
[0047] The data acquisition unit 201 acquires image data and request data in the same manner as in Embodiment 1. In Embodiment 2, the data acquisition unit 201 stores the acquired image data and request data in the history storage unit 207 as acquisition data history. The acquisition data history is also referred to as the first history.
[0048] Further, the data acquisition unit 201 acquires additional data indicating a request input by the user again according to the output classification result output by the output unit 106. The additional data is data to be additionally input to a learned model that is a generation AI model.
[0049] When the user wants to execute classification again by referring to the output classification result, the user may input request data having a content different from the already input request data as additional data. For example, when a part of the object to be classified is not classified as a part of the object, specifically, when the object is a "dog" and the dog's ear is not classified as an image of a dog, the content of requesting to classify that part as an object may be included in the additional data. Specifically, the user can request to classify that part as an object by including in the additional data a text indicating to classify that part as an object, including a rectangle including that part in the additional data, or including the coordinates of points of that part in the additional data.
[0050] Similar to Embodiment 1, the preprocessing unit 202 uses at least one of the image data from the data acquisition unit 101 and the requested data to perform preprocessing for effective segmentation in the segmentation unit 104. In Embodiment 2, the preprocessing unit 202 stores the input image data, elements, and parameters acquired based on the image data and the requested data in the history storage unit 207 as preprocessing data history. The preprocessing data history is also called the second history.
[0051] Furthermore, when the preprocessing unit 202 acquires additional data from the data acquisition unit 201, it uses the acquired data history and preprocessing data history stored in the history storage unit 207, along with the additional data, to acquire new input image data, new elements, and new parameters. Here, the newly acquired input image data, new elements, and new parameters are also referred to as reacquired input image data, reacquired elements, and reacquired parameters. At least one of the reacquired input image data, reacquired elements, and reacquired parameters is different from the input image data, elements, and parameters acquired based on the image data and request data, as in Embodiment 1.
[0052] Specifically, the preprocessing unit 202 inputs the acquired data history, preprocessed data history, and additional data into the trained model to obtain reacquired input image data, reacquired elements, and reacquired parameters.
[0053] In the first embodiment, the execution command generation unit 203 receives input image data, elements, and parameters from the preprocessing unit 202, sets the parameters in the segmentation model, and generates an execution command to classify the elements from the input image data. In the second embodiment, the execution command generation unit 203 receives reacquired input image data, reacquired elements, and reacquired parameters from the preprocessing unit 202, sets the reacquired parameters in the segmentation model, inputs the reacquired input image data into the segmentation model, and generates an execution command to classify the reacquired elements in the reacquired input image data. The execution command generated in this way is provided to the segmentation unit 104.
[0054] The history storage unit 207 is a storage unit that stores the history of acquired data and the history of pre-processed data.
[0055] The image processing device 200 described above can also be implemented by a computer such as the PC 10 shown in Figure 3. Specifically, the history storage unit 207 can be implemented by storage 11 or memory 12.
[0056] Figure 6 is a flowchart showing the operation of the image processing device 200 according to Embodiment 2. Here, among the steps included in the flowchart shown in Figure 6, steps that perform the same processing as the steps included in the flowchart showing the operation of the image processing device 100 according to Embodiment 1, shown in Figure 4, are denoted by the same reference numerals as the steps shown in Figure 4.
[0057] The processing in steps S10 to S15 in Figure 6 is the same as the processing in steps S10 to S15 in Figure 4. However, after the processing in step S15 in Figure 6, the process proceeds to step S26. In step S10 in Figure 6, the data acquisition unit 201 stores the acquired image data and request data in the history storage unit 207 as acquired data history. Also, in step S11 in Figure 6, the pre-processing unit 202 stores the input image data, elements, and parameters acquired based on the image data and request data in the history storage unit 207 as pre-processed data history.
[0058] In step S26, the user refers to the output classification result output by the output unit 106 and decides whether or not to perform resegmentation. If resegmentation is to be performed (Yes in S26), the process proceeds to step S27; if resegmentation is not to be performed (No in S26), the process ends.
[0059] Furthermore, if resegmentation is performed (Yes in S26), the user may input additional data via an input unit (not shown) provided in the image processing device 200. Alternatively, the user may input additional data to another device and transmit that additional data, which the communication unit (not shown) of the image processing device 200 may receive.
[0060] In step S27, the data acquisition unit 201 acquires additional data indicating the user's request again, according to the output classification result output by the output unit 106. The acquired additional data is provided to the pre-processing unit 202.
[0061] When the preprocessing unit 202 acquires additional data from the data acquisition unit 201, it uses the acquired data history and preprocessing data history stored in the history storage unit 207, along with the additional data, to acquire the reacquired input image data, reacquired elements, and reacquired parameters (S28). The reacquired input image data, reacquired elements, and reacquired parameters acquired here are provided to the execution command generation unit 203.
[0062] When the execution command generation unit 203 receives the reacquired input image data, reacquired elements, and reacquired parameters from the preprocessing unit 202, it sets the reacquired parameters in the segmentation model and generates an execution command to classify the reacquired elements from the reacquired input image data (S29). Then the process returns to step S13.
[0063] As described above, according to this embodiment, if the user wishes to apply corrections to the output classification results, they can easily do so by providing additional data to the image processing device 200. Therefore, multiple classifications can be easily put into practical use.
[0064] Embodiment 3. Figure 7 is a block diagram schematically showing the configuration of the image processing apparatus 300 according to Embodiment 3. The image processing apparatus 300 comprises a data acquisition unit 301, a pre-processing unit 302, an execution command generation unit 303, a segmentation unit 104, a post-processing unit 105, an output unit 106, a history storage unit 307, and an evaluation unit 308.
[0065] The segmentation unit 104, post-processing unit 105, and output unit 106 of the image processing device 300 in Embodiment 3 are the same as those of the image processing device 100 in Embodiment 1. However, in the image processing device 300 in Embodiment 3, the output unit 106 receives an output classification result that has received a high evaluation from the evaluation unit 308 and outputs that output classification result.
[0066] The data acquisition unit 301 acquires image data and request data, similar to the first embodiment. In the third embodiment, the data acquisition unit 301 stores the acquired image data and request data as an acquired data history in the history storage unit 307.
[0067] Similar to Embodiment 1, the preprocessing unit 302 uses at least one of the image data from the data acquisition unit 101 and the requested data to perform preprocessing to enable segmentation in the segmentation unit 104. Then, in Embodiment 3, the preprocessing unit 302 stores the input image data, elements, and parameters acquired based on the image data and the requested data in the history storage unit 307 as preprocessing data history.
[0068] Furthermore, when the preprocessing unit 302 receives a reprocessing instruction from the evaluation unit 308, it refers to the preprocessing data history stored in the history storage unit 307 and performs an adjustment that modifies at least one of the previous input image data, elements, and parameters. The input image data, elements, and parameters that have been adjusted are also referred to as the adjusted input image data, adjusted element, and adjusted parameter.
[0069] Specifically, the preprocessing unit 302 may acquire adjustment input image data, adjustment elements, and adjustment parameters by inputting the acquired data history and preprocessing data history into the trained model. Furthermore, if the reprocessing instruction from the evaluation unit 308 includes evaluation information indicating the content of the evaluation performed by the evaluation unit 308, that evaluation information may also be input into the trained model along with the acquired data history and preprocessing data history. In addition, the preprocessing unit 302 may make adjustments to at least one of the previous input image data, elements, and parameters according to predetermined rules.
[0070] Furthermore, the preprocessing unit 302 stores the adjustment input image data, adjustment elements, and adjustment parameters as preprocessing data history in the history storage unit 307. The adjustment input image data, adjustment elements, and adjustment parameters are also provided to the execution command generation unit 303.
[0071] In the first embodiment, the execution command generation unit 303 receives input image data, elements, and parameters from the preprocessing unit 302, sets the parameters in the segmentation model, and generates an execution command to classify the elements from the input image data. In the third embodiment, the execution command generation unit 303 receives the adjusted input image data, adjusted elements, and adjusted parameters from the preprocessing unit 302, and generates an execution command with the adjusted content. In other words, the execution command generation unit 303 sets the adjusted parameters in the segmentation model, inputs the adjusted input image data into the segmentation model, and generates an execution command to classify the adjusted elements in the adjusted input image data. The execution command thus generated is provided to the segmentation unit 104.
[0072] The evaluation unit 308 receives the output classification result from the post-processing unit 105 and evaluates the output classification result. For example, the evaluation unit 308 can evaluate the output classification result by inputting the output classification result and a prompt indicating that the output classification result should be evaluated to the trained model, which is the generated AI model. If necessary for evaluation in the trained model, the objects classified in the output classification result may be explicitly indicated in the prompt. The prompt may also include examples showing what kind of evaluation is needed.
[0073] Then, if the evaluation of the output classification result is lower than a predetermined evaluation, the evaluation unit 308 gives a reprocessing instruction to the preprocessing unit 302. The reprocessing instruction may include evaluation information indicating the content of the evaluation.
[0074] The image processing device 300 described above can also be implemented by a computer such as the PC 10 shown in Figure 3. Specifically, the history storage unit 307 can be implemented by the storage 11 or memory 12. The evaluation unit 308 can also be implemented by loading a program stored in the storage 11 into the memory 12 and having the processor 13 execute that program.
[0075] Figure 8 is a flowchart showing the operation of the image processing device 300 according to Embodiment 3. Here, among the steps included in the flowchart shown in Figure 8, steps that perform the same processing as the steps included in the flowchart showing the operation of the image processing device 100 according to Embodiment 1, shown in Figure 4, are denoted by the same reference numerals as the steps shown in Figure 4.
[0076] The processing in steps S10 to S14 in Figure 8 is the same as the processing in steps S10 to S14 in Figure 4. However, after the processing in step S14 in Figure 8, the process proceeds to step S36. In step S10 in Figure 8, the data acquisition unit 301 stores the acquired image data and request data in the history storage unit 207 as acquired data history. Also, in step S11 in Figure 8, the pre-processing unit 302 stores the input image data, elements, and parameters acquired based on the image data and request data in the history storage unit 307 as pre-processed data history.
[0077] In step S36, the evaluation unit 308 evaluates the output classification result from the post-processing unit 105. Based on the evaluation of the output classification result, the evaluation unit 308 determines whether re-segmentation is necessary (S37). For example, if the evaluation of the output classification result is above a predetermined threshold, the evaluation unit 308 determines that re-segmentation is not necessary (No in S37), and if the evaluation of the output classification result is below the predetermined threshold, it determines that re-segmentation is necessary (Yes in S37). If re-segmentation is not necessary (No in S37), the process proceeds to step S15, and if re-segmentation is necessary (Yes in S37), the process proceeds to step S38.
[0078] The process in step S15 of Figure 8 is the same as the process in step S15 of Figure 4.
[0079] In step S38, the evaluation unit 308 gives a reprocessing instruction to the preprocessing unit 302, and the preprocessing unit 302 adjusts the input image data, elements, and parameters by changing at least one of them, according to the instruction, using the acquired data history and preprocessing data history stored in the history storage unit 307. The adjusted input image data, elements, and parameters are given to the execution command generation unit 303, and the process returns to step S12. The preprocessing unit 302 also stores the adjusted input image data, elements, and parameters in the history storage unit 307 as preprocessing data history.
[0080] As described above, according to this embodiment, the image processing device 300 automatically evaluates the classification result and repeatedly performs segmentation by changing at least one of the input image data, elements, and parameters until the evaluation reaches or exceeds a predetermined evaluation. This allows the user to easily obtain classification results with high classification accuracy.
[0081] In embodiments 2 and 3 described above, the image processing devices 200 and 300 are equipped with history storage units 207 and 307, but embodiments 2 and 3 are not limited to such examples. For example, some or all of the data stored in the history storage units 207 and 307 may be stored in another device such as a cloud. Specifically, some or all of the data stored in the history storage units 207 and 307 may be stored in another device that stores the trained models used by the pre-processing units 202 and 302. If all of the data stored in the history storage units 207 and 307 is stored in another device, the image processing devices 200 and 300 do not need to be equipped with history storage units 207 and 307.
[0082] 100, 200, 300 Image processing device, 101, 201, 301 Data acquisition unit, 102, 202, 302 Pre-processing unit, 103, 203, 303 Execution command generation unit, 104 Segmentation unit, 105 Post-processing unit, 106 Output unit, 207, 307 History storage unit, 308 Evaluation unit.
Claims
1. An image processing apparatus comprising: a preprocessing unit that inputs image data and request data indicating a request to classify objects from the image data into a generating AI model to acquire parameters to be set in a segmentation model, input image data to be input into the segmentation model, and elements to be classified using the segmentation model; an execution command generation unit that sets the parameters in the segmentation model, inputs the input image data into the segmentation model, and generates an execution command to classify the elements in the input image data; and a segmentation unit that acquires a classification result by performing segmentation to classify the elements using the segmentation model according to the execution command.
2. The image processing apparatus according to claim 1, characterized in that the preprocessing unit obtains multiple elements corresponding to the multiple objects when the request data includes multiple objects, the execution command generation unit generates multiple execution commands corresponding to each of the multiple elements, and the segmentation unit performs the segmentation according to each of the multiple execution commands to obtain multiple classification results.
3. The image processing apparatus according to claim 2, further comprising a post-processing unit that generates an output classification result showing the multiple classification results using a single image.
4. The image processing apparatus according to claim 3, characterized in that the one image is the image shown in the image data.
5. The image processing apparatus according to claim 1, further comprising a post-processing unit that generates an output classification result indicating the classification result using the image data when the image data and the input image data are different.
6. The image processing apparatus according to any one of claims 1 to 5, further comprising: a storage unit that stores a first history which is a history of the image data and the request data, and a second history which is a history of the input image data, the elements and the parameters; and a data acquisition unit that acquires additional data to be input to the generating AI model, wherein the preprocessing unit inputs the first history, the second history and the additional data to the generating AI model to acquire new parameters to be set in the segmentation model, new input image data to be input in the segmentation model and new elements to be classified using the segmentation model; the execution command generation unit generates a new execution command which sets the new parameters in the segmentation model, inputs the new input image data in the segmentation model and classifies the new elements in the new input image data; and the segmentation unit performs segmentation which classifies the new elements using the segmentation model according to the new execution command.
7. An image processing apparatus according to any one of claims 1 to 6, further comprising an evaluation unit that evaluates the classification result and instructs the preprocessing unit to reprocess if the evaluation of the classification result is lower than a predetermined evaluation, wherein the preprocessing unit, upon receiving the instruction, performs an adjustment to change at least one of the parameters, the input image data, and the elements, the execution command generation unit generates a new execution command with the adjusted content, and the segmentation unit performs segmentation using the segmentation model according to the new execution command.
8. A program characterized in that it causes a computer to function as a preprocessing unit that inputs image data and request data indicating a request to classify objects from the image data into a generating AI model to acquire parameters to be set in a segmentation model, input image data to be input into the segmentation model, and elements to be classified using the segmentation model; an execution command generation unit that sets the parameters in the segmentation model, inputs the input image data into the segmentation model, and generates an execution command to classify the elements in the input image data; and a segmentation unit that, in accordance with the execution command, performs segmentation to classify the elements from the input image data using the segmentation model and acquires the classification result.
9. An image processing method characterized by inputting image data and request data indicating a request to classify objects from the image data into a generating AI model to obtain parameters to be set in a segmentation model, input image data to be input into the segmentation model, and elements to be classified using the segmentation model; setting the parameters in the segmentation model, inputting the input image data into the segmentation model, generating an execution command to classify the elements in the input image data; and performing segmentation to classify the elements from the input image data using the segmentation model according to the execution command, thereby obtaining a classification result.