Generation program, generation method, and information processing device

Abstract

This generation program causes a computer to perform a process for: encoding features of misclassified images, which are images among input images classified by a machine learning model, into a space different from the image space; expanding the encoded features within the categories containing the features; and generating a factor level table which is used in the machine learning model, and to which the categories have been added as factor candidates and the encoded features and expanded features have been added as levels.

Description

Generation program, generation method, and information processing device

[0001] The present invention relates to a generation program, a generation method, and an information processing device.

[0002] Image recognition AI (Artificial Intelligence) is a useful component in fields such as autonomous driving and medical diagnostic systems, but its reliability must also be ensured.

[0003] When testing image recognition AI, if the test data is not diverse enough, situations (weaknesses) in which the machine learning model is prone to making incorrect judgments may be overlooked, and unexpected problems may occur during operation.

[0004] For this reason, some aspects are systematically tested during development using a wide variety of test data to detect and correct potential weaknesses.

[0005] One such testing technique involves checking the performance of a machine learning model on a factor-level map using human-interpretable features (factors and levels).

[0006] JP 2023-157645 A JP 2019-211342 A U.S. Patent Application Publication No. 2023 / 0129240 U.S. Patent Application Publication No. 2019 / 0385338

[0007] Tahereh Zohdinasab, Vincenzo Riccio, Alessio Gambi, and Paolo Tonella. “DeepHyperion: exploring the feature space of deep learning-based systems through illumination search,” In Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis. 79-90, 2021.

[0008] However, the above-mentioned conventional techniques have room for improvement in that they can only find weaknesses within a preset factor level.

[0009] In one aspect, an object of the present invention is to provide a generation program, a generation method, and an information processing device that can realize automatic generation of new factor levels.

[0010] A generation program according to one aspect causes a computer to execute a process of encoding features of an image that has been misclassified among input images determined by a machine learning model into a space different from the image space, expanding the features in a category that includes the encoded features, setting the categories as factor candidates, and adding the encoded features and the expanded features as levels to generate a factor level table to be used in the machine learning model.

[0011] According to one embodiment, automatic generation of new factor levels can be achieved.

[0012] FIG. 1 is a block diagram showing an example of the functional configuration of a server device. FIG. 2 is a diagram showing an example of a factor level table. FIG. 3 is a diagram showing an example of test data. FIG. 4 is a schematic diagram explaining one aspect of a problem. FIG. 5 is a diagram showing an example of an inappropriate factor level table. FIG. 6 is a diagram showing an example of an appropriate factor level table. FIG. 7 is a diagram showing one aspect of a problem-solving approach. FIG. 8 is a diagram showing an example of a factor level map. FIG. 9 is a diagram showing an example of a factor level map. FIG. 10 is a schematic diagram explaining automatic generation of new factor levels. FIG. 11 is a diagram showing an example of a prompt. FIG. 12 is a diagram showing an example of an erroneous judgment image. FIG. 13 is a diagram showing an example of a characteristic sentence. FIG. 14 is a diagram showing an example of a prompt. FIG. 15 is a diagram showing an example of a fill-in-the-blank question. FIG. 16 is a diagram showing an example of a prompt. FIG. 17 is a diagram showing an example of an answer to a fill-in-the-blank question. FIG. 18 is a diagram showing an example of a process for compressing the number of words in a sentence. FIG. 19 is a diagram showing an example of image generation. FIG. 20 is a diagram showing an example of a new factor level. FIG. 21 is a diagram showing an example of a new factor level. FIG. 22 is a diagram showing an example of a new factor level. Fig. 23 is a flowchart showing the procedure of the generation process, and Fig. 24 is a diagram showing an example of a hardware configuration.

[0013] Hereinafter, embodiments of a generation program, a generation method, and an information processing device according to the present disclosure will be described with reference to the accompanying drawings. Note that these embodiments merely illustrate one example or aspect, and the structure, action, function, properties, characteristics, methods, and uses according to the present disclosure are not limited by such examples. Furthermore, the embodiments can be appropriately combined within the scope of not causing any contradiction in the processing content.

[0014] <Example 1> <Usage Scene> Fig. 1 is a block diagram showing an example of the functional configuration of a server device 10. Fig. 1 shows the server device 10 that provides a generation function that realizes automatic generation of a machine learning model that executes an image recognition task, for example, factor levels that define test conditions for an image recognition AI.

[0015] Such image recognition AI may be applied, as an example only, to fields requiring high reliability, such as systems for autonomous driving and medical diagnosis. Below, as an example of an image recognition task, an example in which image recognition AI performs an image classification task will be given, but other machine learning tasks such as object detection and segmentation may also be performed.

[0016] The server device 10 can provide the above-described generation function as a cloud service by executing a platform as a service (PaaS) type middleware or a software as a service (SaaS) type application. The server device 10 may be included as an example of an information processing device.

[0017] As shown in Fig. 1, the server device 10 can be communicatively connected to the client terminals 30 via a network NW. For example, the network NW may be any type of communication network, whether wired or wireless, such as the Internet or a local area network (LAN). Note that Fig. 1 shows an example in which one client terminal 30 is connected to one server device 10, but any number of client terminals 30 may be connected.

[0018] The client terminal 30 is a terminal device that receives the above-described generation function. For example, the client terminal 30 can be used by anyone involved in a system that includes image recognition AI as a component, such as those who design, develop, operate, or maintain the system. The client terminal 30 may be implemented by any computer, such as a personal computer, a smartphone, a tablet terminal, or a wearable terminal.

[0019] Although the above example illustrates a case in which the generation function is provided as a cloud service, the present invention is not limited to this. For example, the generation function may be provided on-premise. Furthermore, the above example illustrates a case in which the generation function is provided in a client-server system, but the present invention is not limited to this. For example, the generation function may be provided standalone by causing an application running on the client terminal 30 to cause the client terminal 30 to execute processing corresponding to the generation function.

[0020] <Factor Level> The test data used to test the image recognition AI described above must be diverse in order to detect and correct situations (weaknesses) in which the machine learning model is prone to making erroneous judgments.

[0021] For this reason, test data is generated using a factor level table, which is expressed in human-interpretable features. Here, "factors" refer to the conditions imposed on the test subject, and "levels" refer to the patterns that the conditions can take.

[0022] FIG. 2 is a diagram showing an example of a factor level table. As shown in FIG. 2, the factor level table includes three factors: "weather," "time of day," and "location." Of these, the factor "weather" includes four levels, such as "sunny," "cloudy," "rainy," and "snowy." The factor "time of day" includes three levels, such as "daytime," "evening," and "nighttime." Furthermore, the factor "location" includes three levels, such as "urban area," "highway," and "residential area." Test data is generated according to condition patterns corresponding to combinations of the "levels" included in these "factors."

[0023] FIG. 3 is a diagram showing an example of test data. FIG. 3 shows test images generated according to a condition pattern corresponding to a combination of the level "snow" included in the factor "weather," the level "daytime" included in the factor "time of day," and the level "urban area" included in the factor "location" in the factor level table shown in FIG. 2. By combining the levels of each factor in this way, a wide variety of condition patterns can be covered. Combinations that are impossible to realize based on the specifications or logic of the test target can be excluded as prohibited combinations.

[0024] <One Aspect of the Problem> As explained in the Background Art section above, the above-mentioned conventional technology has room for improvement in that it can only find weaknesses within a predetermined factor level.

[0025] Fig. 4 is a schematic diagram illustrating one aspect of the problem. Fig. 4 shows an example of a scenario in which an image recognition AI that performs image classification of vehicles is tested. For example, in the example shown in Fig. 4, factor levels that combine the levels of two factors, "weather" and "time of day," are set in advance.

[0026] The performance of the image recognition AI is evaluated using test images generated according to condition patterns corresponding to combinations of the levels of these two factors. The evaluation results are then mapped for each combination of the levels of each factor to generate a factor level map.

[0027] Here, an example is shown in which, for cells determined by a combination of levels, darker hatching is assigned as the evaluation improves, while lighter hatching is assigned as the evaluation worsens.

[0028] Although such a factor level map can detect that the image recognition AI's weaknesses are the combination of {rain, night} and the combination of {sunny, day}, it is difficult to detect whether there are weaknesses in other factor levels. For example, it is not possible to detect whether there are weaknesses in condition patterns that include combinations of factor levels such as "bus color" or "location," and weaknesses may be hidden in other factor levels that are not included in the pre-set factor levels.

[0029] However, there are technical challenges in automatically generating factor levels that have not been set in advance, i.e., new factor levels. In other words, since an infinite number of factor levels can be conceived, it is difficult to find appropriate factor levels that are useful for determining whether or not a machine learning model needs improvement and for planning countermeasures.

[0030] Figure 5 shows an example of an inappropriate factor level table. Similar to Figure 4, Figure 5 shows a factor level map on which an image recognition AI performing vehicle image classification is tested. Furthermore, Figure 5 shows an example in which cells in the factor level map with good ratings are assigned dark hatching, while cells with poor ratings are assigned light hatching.

[0031] As shown in Figure 5, in the case of a factor level table in which no weaknesses exist to begin with, or a factor level map in which all weaknesses exist, there are no clues for improvement, such as determining whether the machine learning model needs to be improved or formulating countermeasures, making it difficult to approach improvement.

[0032] Furthermore, in general, in combination evaluations using factor levels, the levels within the same factor must be independent. Nevertheless, in a factor level map where the levels are inappropriately selected, the factors are too vague and the levels are not independent of each other. For example, in the example shown in Figure 5, "urban" and "snow" are listed side by side as levels for the factor "environment," but there may be an environment that is both "urban" and "snow." As a result, despite the assumption of a two-factor combination test, the evaluation ends up being a multi-factor combination test.

[0033] <One Aspect of the Problem-Solving Approach> Therefore, the generation function according to this embodiment realizes automatic generation of factor levels that satisfy the following two criteria. The first criterion is that the boundary between weak and non-weak features (hereinafter referred to as "weakness boundary") is included, rather than listing only weak levels. The second criterion is that the levels included in the same factor are closely related, for example, visually similar, or share the same or a common concept.

[0034] FIG. 6 is a diagram showing an example of an appropriate factor level table. Similar to FIG. 4, FIG. 6 shows a factor level map 20A on which an image recognition AI that classifies images of vehicles is tested. Furthermore, FIG. 6 shows newly added levels corresponding to the columns of the levels included in the rows and columns of the factor level map 20A. Furthermore, FIG. 6 also shows an example in which cells with good evaluations in the factor level map are assigned dark hatching, while cells with poor evaluations are assigned light hatching. Hereinafter, factors, levels, and factor levels newly generated by the above-described generation function may be referred to as "new factors," "new levels," and "new factor levels" to distinguish them from pre-set factor levels.

[0035] As shown in Figure 6, the columns of the factor level map 20A include new levels such as "Snowy tracks," "Icy tracks," "Leafy tracks," "Muddy tracks," "Dusty tracks," and "Sandy tracks." These six new levels are all encompassed by the common concept of tire tracks. As such, the new levels are closely related to each other, and therefore satisfy the second criterion above.

[0036] Furthermore, according to the factor level map 20A, a weak boundary WB1 appears that distinguishes between cells that are prone to misjudgment, i.e., cells with dark hatching, and cells that are less likely to be misjudged, i.e., cells with light hatching, so it can be seen that the new level satisfies the first criterion above.

[0037] For example, Fig. 6 shows a test image 41 generated according to a condition pattern corresponding to a combination of "snowy tracks" and "sunny" as an example of a test image corresponding to a cell that is prone to misjudgment. Fig. 6 also shows a test image 42 generated according to a condition pattern corresponding to a combination of "muddy tracks" and "sunny" as an example of a cell test image that is less prone to misjudgment.

[0038] Evaluation using these two test images 41 and 42 reveals that even under the same sunny weather, tire tracks on snow and tire tracks on mud have a different impact on the inference of the image recognition AI. In other words, a boundary appears where tire tracks on snow are more likely to be misjudged, while tire tracks on mud are less likely to be misjudged. A series of boundaries where such differences in evaluation appear appears on the factor level map 20A as a weak boundary WB1.

[0039] From the viewpoint of realizing the automatic generation of new levels as exemplified by the factor level map 20A, the generation function according to the first embodiment automatically generates new factors and new levels by starting from a feature that contributes to erroneous judgment and expanding the features to the surrounding area.

[0040] Fig. 7 is a diagram showing one aspect of the problem-solving approach. Fig. 7 illustrates a feature space in which decision boundaries for vehicles such as "school bus," "streetcar," and "trolley bus" are plotted. Furthermore, Fig. 7 illustrates an example of a test image that is misclassified by the image recognition AI, namely, misclassified image 43, which was misclassified as "streetcar" by the image recognition AI despite the correct label being "trolley bus."

[0041] As shown in Fig. 7, the features of the misclassified image 43 are encoded into a space different from the image space. For example, in the example shown in Fig. 7, the features of the misclassified image 43 are encoded into the language space to obtain the phrase "snowy, tire1 tracks" that represents the feature. By expanding this feature phrase "snowy, tire tracks", new levels "leafy, tire tracks" and "muddy, tire tracks" can be obtained. Furthermore, by generating titles from "snowy, tire tracks", "leafy, tire tracks", and "muddy, tire tracks", the factor "something that covers the road" can be obtained.

[0042] Therefore, the generation function according to the first embodiment can automatically generate new factor levels that satisfy the above two criteria. Such automatic generation of new factor levels also makes it possible to generate the factor level map 20A shown in FIG.

[0043] The example of generating new factor levels shown in Figure 7 is merely an example, and other new factor levels, and even new factor level maps, can be automatically generated. Figures 8 and 9 are diagrams showing examples of factor level maps. In Figures 8 and 9, new factor levels are added in the column direction included in factor level maps 20B and 20C.

[0044] For example, as shown in FIG. 8, the columns of the factor level map 20B include new levels such as "city street," "highway," and "residential." All three of these new levels are encompassed by the common concept of location. As such, the new levels are closely related to each other, and therefore satisfy the second criterion above. Furthermore, a boundary appears where urban areas are prone to misjudgment, while highways are less prone to misjudgment. A series of boundaries where differences in evaluation appear like this appears on the factor level map 20B as a weak boundary WB2. Therefore, it can be seen that the new levels also satisfy the first criterion above.

[0045] As shown in FIG. 9 , the columns of the factor level map 20C include new levels such as "Yellow bus," "Red bus," and "Blue bus." All three of these new levels are encompassed by the common concept of bus color. As such, the new levels are closely related to one another, and therefore satisfy the second criterion. Furthermore, a boundary appears where yellow buses are more likely to be misjudged, while red and blue buses are less likely to be misjudged. This series of boundaries where differences in evaluation appear appears in the factor level map 20C as the weak boundary WB3. Therefore, it can be seen that the new levels also satisfy the first criterion.

[0046] <Configuration of Server Device 10> Next, the functional configuration of server device 10 that provides the above-described generation function will be described. Fig. 1 schematically illustrates blocks related to the generation function possessed by server device 10. As shown in Fig. 1, server device 10 has a communication control unit 11, a storage unit 13, and a control unit 15. Note that Fig. 1 only illustrates a selection of functional units related to the above-described generation function, and server device 10 may also be provided with functional units other than those illustrated.

[0047] The communication control unit 11 is a functional unit that controls communication with other devices such as the client terminal 30. In one embodiment, the communication control unit 11 can be realized by a network interface card such as a LAN card. In one aspect, the communication control unit 11 receives a request for generating a factor level map from the client terminal 30, or outputs a response to the request to the client terminal 30.

[0048] The storage unit 13 is a functional unit that stores various types of data. In one embodiment, the storage unit 13 may be realized by internal, external, or auxiliary storage of the server device 10. For example, the storage unit 13 stores a factor level 13A and a factor level map 13B. The factor level 13A and the factor level map 13B will be described later together with a scene in which reference or registration is performed.

[0049] The control unit 15 is a functional unit that performs overall control of the server device 10. For example, the control unit 15 may be realized by a hardware processor. As shown in FIG. 1 , the control unit 15 includes a reception unit 15A, an image generation unit 15B, an evaluation unit 15C, a factor level map generation unit 15D, an output unit 15E, an erroneous judgment feature extraction unit 15F, a sentence expansion unit 15G, and a new factor level generation unit 15H. Note that the control unit 15 may also be realized by hardwired logic or the like.

[0050] The reception unit 15A is a processing unit that receives various types of information from the client terminal 30. In one aspect, the reception unit 15A can receive a request from the client terminal 30 to generate a factor level map.

[0051] In one aspect, the receiving unit 15A can accept the specification of one or more factor levels when accepting the request. Examples of factor levels accepted at this stage include factor levels manually set in advance by a user who is provided with the generation function under the requirements of the image recognition AI model. The factor levels accepted together with the request are stored in the storage unit 13 as factor levels 13A.

[0052] In another aspect, the reception unit 15A can also accept a designation of the image recognition AI to be tested when accepting the request. For example, the reception unit 15A can accept an upload of a trained machine learning model from the client terminal 30, or can accept a designation from a library of machine learning models published on the Internet.

[0053] The image generating unit 15B is a processing unit that generates a test image. In one aspect, the image generating unit 15B generates a test image based on a factor level table including two or more factors when a manually set factor level, the specification of which is accepted by the accepting unit 15A, is registered in the factor level 13A stored in the storage unit 13, or when a new factor level generated by a new factor level generating unit 15H (described later) is registered.

[0054] The image generation described above can be achieved using any image generation model. For example, the image generation model may be a diffusion model implemented using open source software such as Stable Diffusion. Test images can be generated by inputting, as prompts, into the text encoder of such a diffusion model, the correct answer labels for the image recognition task of the image recognition AI being tested, as well as the combination of factor levels, i.e., the text corresponding to the cells in the factor level table. Needless to say, the image generation unit 15B can generate multiple test images for each cell in the factor level table.

[0055] The evaluation unit 15C is a processing unit that evaluates the performance of the image generation AI to be tested based on the test images generated by the image generation unit 15B. In one embodiment, the evaluation unit 15C executes the following exemplary process for each combination of factor levels, i.e., for each cell in the factor level table. For example, the evaluation unit 15C determines whether each test image is correct or incorrect based on whether the output of the image recognition AI to which the test image is input matches the correct label of the test image. Here, test images determined to be incorrect are identified as misclassified images. The evaluation unit 15C can then calculate the number of correct answers out of the total number of test images, the so-called accuracy rate, as one of the indicators for evaluating the performance of the image recognition AI. This allows for an evaluation result, such as the accuracy rate of the image recognition AI, to be obtained for each cell in the factor level table.

[0056] The factor level map generation unit 15D is a processing unit that generates a factor level map. In one aspect, the factor level map generation unit 15D generates a factor level map by mapping the evaluation results evaluated by the evaluation unit 15C and the accuracy rate of the image recognition AI to combinations of the levels of each factor, i.e., to each cell of the factor level table. The factor level map 13B generated in this way is stored in the storage unit 13.

[0057] The factors for which a factor level map is to be generated may have any breakdown. For example, it is possible to start generating a factor level map corresponding to a factor level table that includes all factors included in the factor level 13A. Furthermore, each time a new factor level is generated by the new factor level generation unit 15H (described later), a factor level table including the new factor level and a manually set factor level may be generated, or a factor level table including only the new factor level may be generated. In addition, the number of factor levels included in the factor level map does not necessarily have to be multiple, and a factor level map including only one factor level may be generated.

[0058] The output unit 15E is a processing unit that executes output control for the client terminal 30. In one aspect, the output unit 15E can display the factor level map generated by the factor level map generation unit 15D on the client terminal 30 as a response to the request accepted by the acceptance unit 15A.

[0059] Here, the automatic generation of new factor levels by the generation function can be realized by the misjudgment feature extraction unit 15F, the sentence expansion unit 15G, and the new factor level generation unit 15H, as shown by hatching in FIG.

[0060] FIG. 10 is a schematic diagram illustrating the automatic generation of new factor levels. As shown in FIG. 10, the misjudgment feature extraction unit 15F expresses, in a sentence, the features of the misjudged images, which were misjudged by the image recognition AI among the test images generated by the image generation unit 15B (S1). Hereinafter, the sentence expressing the features of the misjudged image may be referred to as a "characteristic sentence." Next, the sentence expansion unit 15G expands the characteristic sentence obtained in step S1 to other related characteristic sentences that are related to the characteristic sentence (S2). Then, the new factor level generation unit 15H generates new levels by compressing each sentence, including the characteristic sentence obtained in step S1 and the related characteristic sentence obtained in step S2, into words or phrases of a predetermined number of words or less, and generates new factors by generating titles for the set of new levels (S3). Using the new factor levels thus generated in step S3, the image generation unit 15B generates test images, the evaluation unit 15C evaluates the image recognition AI, and the factor level map generation unit 15D generates a factor level map.

[0061] The misjudgment feature extraction unit 15F is a processing unit that expresses the features of the misjudgment image in sentences. In one aspect, the misjudgment feature extraction unit 15F encodes the features of the misjudgment image into a space different from the image space. For example, the misjudgment feature extraction unit 15F encodes the features of the misjudgment image from the image space into a language space. Such encoding into a language space may be realized by a large-scale language model, so-called LMM (Large Multimodal Model). It is not limited to this LMM, and may also be realized by VQA (Visual Question Answering), Image Captioning, or VLM (Vision Language Model).

[0062] For example, the misjudgment feature extraction unit 15F inputs an misjudgment image to the image encoder of the LMM and inputs a prompt including an instruction element for extracting features of the misjudgment image to the text encoder of the LMM. FIG. 11 is a diagram illustrating an example of the prompt. FIG. 12 is a diagram illustrating an example of an misjudgment image. As shown in FIG. 11, the prompt P1 includes a role for analyzing the features of the misjudgment image 44 shown in FIG. 12 and an instruction for extracting the features of the misjudgment image 44 shown in FIG. 12. Furthermore, the prompt P1 includes rules for feature extraction, such as listing the features in bullet points, specifying the number of features to include in the bullet points, specifying the number of words to describe each feature, and specifying the perspective for extracting the features. Note that the prompt P1 is merely an example, and some or all of the role and rule settings may be omitted.

[0063] The LMM to which the prompt P1 and the misjudged image 44 are input in this manner outputs, as an example, a characteristic sentence in which the characteristics of the misjudged image 44 are encoded. Fig. 13 is a diagram showing an example of a characteristic sentence. Fig. 13 shows an output O1 of the LMM to which the prompt P1 shown in Fig. 11 and the misjudged image 44 shown in Fig. 12 are input. For example, in the example shown in Fig. 13, the output O1 of the LMM includes, from top to bottom, 13 characteristic sentences, such as a sentence expressing the characteristics of a sign, a sentence expressing the characteristics of a vintage design, a sentence expressing the characteristics of overhead lines, ..., and a sentence expressing the characteristics of a landscape.

[0064] The sentence expansion unit 15G is a processing unit that expands the characteristic sentence extracted from the misjudgment image by the misjudgment feature extraction unit 15F to other related characteristic sentences that are related to the characteristic sentence. In one aspect, the sentence expansion unit 15G realizes data expansion of the characteristic sentence by making small changes to the characteristic sentence in the language space.

[0065] As just one example, such data expansion can be achieved by using LLMs (Large Language Models) to generate fill-in-the-blank questions by blanking out some words or phrases in characteristic sentences, and then generating answers to those fill-in-the-blank questions.

[0066] Fig. 14 is a diagram showing an example of a prompt. As shown in Fig. 14, prompt P2 contains an instruction to generate a fill-in-the-blank question from the characteristic sentence highlighted in bold and underlined among the 13 characteristic sentences shown in Fig. 13, i.e., the fifth characteristic sentence from the top, "Snow-covered roads with visible tire tracks, implying recent traffic." When this prompt P2 is input, the LLM outputs the fill-in-the-blank question shown in Fig. 15.

[0067] FIG. 15 is a diagram showing an example of a fill-in-the-blank question. For example, in the example shown in FIG. 15, three fill-in-the-blank questions are generated in the output O2 of the LLM. In the first fill-in-the-blank question, a blank is set for the sixth word "tire" from the beginning of the characteristic sentence "Snow-covered roads with visible tire tracks, implying recent traffic." shown in FIG. 14. In the second fill-in-the-blank question, a blank is set for the first word "Snow" from the characteristic sentence shown in FIG. 14. In the third fill-in-the-blank question, a blank is set for the third word "roads" from the beginning and the first word "traffic" from the end of the characteristic sentence shown in FIG. 14.

[0068] FIG. 16 is a diagram showing an example of a prompt. As shown in FIG. 16, prompt P3 includes the second fill-in-the-blank question " <blank>The LLM receives prompt P3 and outputs the answer to the fill-in-the-blank question shown in FIG. 17.

[0069] Fig. 17 is a diagram showing example answers to fill-in-the-blank questions. For example, in the example shown in Fig. 15, output O3 of the LLM lists 10 answers to fill in the blanks of the fill-in-the-blank question shown in Fig. 16. That is, from top to bottom, the 10 answers are listed as "Snow", "Ice", "Mud", "Leaf", "Dust", "Sand", "Rain", "Frost", "Gravel", and "Water".

[0070] By generating these fill-in-the-blank questions and answers to the fill-in-the-blank questions, it is possible to obtain related feature sentences that are expanded by fixing the type of trace or road and making small changes that change the type of material covering the road.

[0071] In this way, data expansion can be achieved by generating fill-in-the-blank questions and answers to the fill-in-the-blank questions, thereby acquiring related characteristic sentences. For example, when generating similar sentences from characteristic sentences, sentences that lack consistency may be listed, unrelated sentences may be included, or even if related characteristic sentences are successfully generated, only a portion of the answers generated by the fill-in-the-blank questions may be obtained, limiting the variation. However, these failures can be prevented by the data expansion described above.

[0072] While the example described here uses LLM to generate fill-in-the-blank questions, some or all of the generation of fill-in-the-blank questions may be achieved through natural language processing. For example, the remaining words after stop words have been removed from the characteristic sentence can be set as candidates for the blanks in the fill-in-the-blank questions. Furthermore, the remaining words after chunks of the characteristic sentences have been removed of modified words, i.e., words belonging to modifiers, can be set as candidates for the blanks in the fill-in-the-blank questions. Furthermore, a TF-IDF score can be calculated for each remaining word or blank candidate, and words with a TF-IDF score above a certain threshold can be set as the blanks. Additionally, the prompt can be set to an instruction element that performs the above-mentioned stop word removal, modifier word removal (modifier extraction), blank setting using the TF-IDF score, or a combination of these.

[0073] The new factor level generating unit 15H is a processing unit that generates a new factor level by compressing each of the characteristic sentences and related characteristic sentences into a word or a phrase of a predetermined number of words or less.

[0074] In one embodiment, the new factor level generation unit 15H gradually reduces sentences such as characteristic sentences and related characteristic sentences. For example, the new factor level generation unit 15H inputs to the LLM sentences such as characteristic sentences and related characteristic sentences, as well as prompts including instruction elements for gradually reducing the number of words contained in the sentences to a specific number of words, for example, three words. FIG. 18 is a diagram showing an example of a process for compressing the number of words in a sentence. FIG. 18 shows an example in which the related characteristic sentence shown in FIG. 17, "Mud-covered roads with visible tire tracks, implying recent traffic," is compressed to three words or less. For example, in the example shown in FIG. 18, the number of words in the related characteristic sentence containing 10 words is compressed to eight words in the first pass, to six words in the second pass, to four words in the third pass, and to three words for "Muddy tire tracks." in the fourth pass. As a result, "Muddy tire tracks" is set as the label for the new level.

[0075] In this way, by gradually reducing the sentence using LLM, it is possible to obtain a simple expression as a level name while preserving the original meaning of the sentence. In other words, if the original sentence, such as a feature sentence or related feature sentence, is shortened to the target number of words all at once, there is a high possibility that important feature elements, such as "tire tracks," will be lost. Note that, although an example has been given in which data expansion is performed by generating fill-in-the-blank questions and answers before the original feature sentences and original related feature sentences are compressed, data expansion may also be performed after the original feature sentences and original related feature sentences are compressed.

[0076] Furthermore, the new factor level generation unit 15H inputs a prompt containing an instruction element for creating a title to the LLM, along with all the new levels obtained for each sentence, such as the characteristic sentence and the related characteristic sentence, etc. As a result, the title "tire tracks" output by the LLM can be acquired as a new factor.

[0077] The new factors and new levels thus obtained are added to and registered as new factor levels in the factor level 13A of the storage unit 13. Using such new factor levels, the image generation unit 15B can generate test images, the evaluation unit 15C can evaluate the image recognition AI, and the factor level map generation unit 15D can generate a factor level map.

[0078] FIG. 19 is a diagram showing an example of image generation. FIG. 19 shows, by way of example only, an example of generating a test image corresponding to the new level "Muddy tire tracks." As shown in FIG. 19, prompt P4 "a trolley bus, Muddy tire tracks," which includes the correct label "trolley bus" and the new level "Muddy tire tracks," is input to the diffusion model 50. By inputting such prompt P4 to the diffusion model 50, a test image 45 corresponding to the correct label "trolley bus" and the new level "Muddy tire tracks" can be obtained.

[0079] Furthermore, after generating a new factor level, the evaluation unit 15C can also evaluate the effectiveness of the new factor level in addition to evaluating the performance of the image recognition AI. In one aspect, the evaluation unit 15C determines whether the new factor level is effective based on whether there is one or more rows or columns corresponding to the new factor level in a factor level map generated from a factor level table including the new factor level, the rows or columns including one or more high-precision cells where the performance of the image recognition AI is equal to or greater than a first threshold and one or more low-precision cells where the performance of the image recognition AI is less than a second threshold that is smaller than the first threshold. Hereinafter, a row or column corresponding to a new factor level that includes one or more high-precision cells and one or more low-precision cells may be referred to as a "row or column including a weak boundary."

[0080] Here, if there is one or more rows or columns that contain weak boundaries, the new factor level is determined to be valid, whereas if there is not one or more rows or columns that contain weak boundaries that contain one or more high-precision cells and one or more low-precision cells, the new factor level is determined to be ineffective.

[0081] Furthermore, in addition to determining whether a new factor level is effective, the effectiveness can also be quantified into a score. For example, the score can be calculated by increasing the added points as the number of rows or columns containing weak boundaries increases, or by increasing the deducted points as the number of rows or columns containing weak boundaries decreases. The score can also be calculated by increasing the added points as the number of low-accuracy cells increases, or by increasing the deducted points as the number of low-accuracy cells decreases. Alternatively, the score can be calculated by increasing the deducted points as the number of cells for which a test image cannot be generated using an image generation model increases, or by increasing the added points as the number of cells for which a test image cannot be generated decreases. Cells for which a test image cannot be generated may include the following cells. For example, in addition to cells with prohibited or contradictory properties, cells may also be included that cannot be reverse-converted from a test image to the prompt used to generate the test image using VQA, or cells that are incorrectly judged by another image generation AI performing the same image recognition task as the image generation AI being tested.

[0082] Such a new factor level effectiveness evaluation makes it possible to distinguish between inappropriate new factor levels and appropriate new factor levels. Figures 20 to 22 are diagrams showing examples of new factor levels. Figures 20 to 22 show factor level maps in which new factor levels are included in the column direction and manually set factor levels are included in the row direction. Furthermore, among Figures 20 to 22, Figures 20 and 21 show examples corresponding to inappropriate new factor levels, while Figure 22 shows an example corresponding to appropriate new factor levels. For example, the above effectiveness evaluation can determine that a new factor level in which the test images at the new level do not affect the accuracy of the image recognition AI (Figure 20) and a new factor level in which test images cannot be generated for most cells (Figure 22) are ineffective. Furthermore, the above effectiveness evaluation can determine that a new factor level in which the test images at the new level affect the accuracy of the image recognition AI and a large number of low-accuracy cells are ineffective (Figure 23).

[0083] <Processing Flow> Next, a processing flow of the server device 10 according to this embodiment will be described. Fig. 23 is a flowchart showing the procedure of the generation processing. This processing is merely an example and can be started when the specification of the factor level is accepted.

[0084] As shown in FIG. 23, when a specification of a factor level is accepted (step S101), the image generating unit 15B generates a test image for each cell of the factor level table based on the factor level table including the factor levels specified in step S101 (step S102).

[0085] Next, the evaluation unit 15C evaluates the performance of the image generation AI to be tested based on the test images generated for each cell of the factor level table in step S102 (step S103). After that, on the condition that the factor level table includes the new factor level, the evaluation unit 15C evaluates the effectiveness of the new factor level (step S104).

[0086] If the new factor level is effective (Yes in step S104), the factor level map generating unit 15D generates a factor level map by mapping the evaluation results obtained for each cell of the factor level table in step S103 (step S105). If the new factor level is not effective (No in step S104), the process of step S105 is skipped.

[0087] Then, if any termination condition, such as the number of loops, the number of misjudged images, or the target number of new factor levels generated, is not met (No in step S106), the misjudgment feature extraction unit 15F encodes the features of the misjudged images that were misjudged by the image recognition AI among the test images generated in step S102 into a sentence (step S107).

[0088] Next, the sentence expansion unit 15G expands the characteristic sentence obtained in step S107 to other related characteristic sentences that are related to the characteristic sentence (step S108).After that, the new factor level generation unit 15H generates new levels and new factors by compressing each sentence of the characteristic sentence obtained in step S107 and the related characteristic sentence obtained in step S108 into words or phrases of a predetermined number of words or less (step S109).

[0089] With the new factor levels thus generated in step S109 specified, the processes from step S102 to step S105 are repeated.

[0090] Thereafter, if the termination condition is satisfied (Yes in Step S106), the output unit 15E causes the factor level map generated in Step S105 to be displayed on the client terminal 30 (Step S110), and ends the process.

[0091] <Summary of Example 1> As described above, the server device 10 according to this example encodes the features of images that have been misclassified among the images tested by the image recognition AI into a word string, and generates a new level and a new factor from the encoded word string and the expanded word string obtained by expanding it.

[0092] Therefore, the server device 10 according to this embodiment can automatically generate new factor levels that are correlated with each other and include weak point boundaries. This allows for systematic testing of image recognition AI using test data with sufficient diversity. This makes it possible to discover situations (weak points) that are prone to false positives in image recognition AI, thereby reducing the possibility of unexpected problems during operation, such as serious accidents resulting in personal injury. Furthermore, the cost of developing image recognition AI quality can be reduced.

[0093] <Example 2> Although Example 1 of the present disclosure has been described so far, various applications are possible, and further, other than Example 1 described above, the present disclosure may be implemented in various different forms.

[0094] <Exercise of creative ability> The matters described in the first embodiment, such as the types and specific names of the image recognition AI and image generation model, are merely examples and can be changed. In addition, the order of processing in the flowchart described in the first embodiment can also be changed within a consistent range.

[0095] <Application Example 1> In the above-described first embodiment, an example was given in which one new factor level was generated from one misclassified image. However, one new factor level can also be generated from multiple misclassified images. In this case, a prompt including an instruction element for extracting common features from the multiple misclassified images can be input to the LMM along with the multiple misclassified images. This makes it possible to extract characteristic sentences that encode features common to the multiple misclassified images. In this case, it is also possible to skip the extraction of characteristic sentences and extract a word string of up to a predetermined number of words from the multiple misclassified images.

[0096] <Application Example 2> In the above Example 1, an example was given in which related characteristic sentences are generated by performing the above-mentioned data extension on characteristic sentences, but related characteristic sentences can also be further generated by performing the above-mentioned data extension on related characteristic sentences.

[0097] <System> The information including the processing procedures, control procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, one or more of the functional units of the server device 10, including the reception unit 15A, image generation unit 15B, evaluation unit 15C, factor level map generation unit 15D, output unit 15E, misjudgment feature extraction unit 15F, sentence expansion unit 15G, and new factor level generation unit 15H, may be configured as separate devices.

[0098] Furthermore, the components of each device shown in the figure are functional concepts and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown. In other words, all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Note that each configuration may also be a physical configuration.

[0099] Furthermore, all or any part of the processing performed by the illustrated device may be realized by a program executed by a hardware processor such as an MPU (Micro-Processing Unit) or a CPU (Central Processing Unit), or may be realized as hardware using wired logic.

[0100] <Hardware> Next, an example of the hardware configuration of the server device described in the above embodiment will be described. Fig. 24 is a diagram showing an example of the hardware configuration. As shown in Fig. 24, the server device 10 has a communication device 10a, a storage device 10b, a memory 10c, and a processor 10d. Note that the components shown in Fig. 24 may be connected to each other via a bus or the like.

[0101] The communication device 10a is a network interface card, etc. The storage device 10b is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD). For example, the storage device 10b stores programs and databases that operate the functions shown in FIG.

[0102] The processor 10d reads out a program that executes the same processing as the processing unit shown in FIG. 1 from the storage device 10b or the like and loads it into the memory 10c, thereby operating a process that executes the functions described in FIG.

[0103] Such a process realizes the same functions as the processing units of the server device 10. For example, the processor 10d reads from the storage device 10b or the like a program having the same functions as the reception unit 15A, image generation unit 15B, evaluation unit 15C, factor level map generation unit 15D, output unit 15E, misjudgment feature extraction unit 15F, sentence extension unit 15G, new factor level generation unit 15H, etc. Then, the processor 10d executes a process that executes processing similar to that of the reception unit 15A, image generation unit 15B, evaluation unit 15C, factor level map generation unit 15D, output unit 15E, misjudgment feature extraction unit 15F, sentence extension unit 15G, new factor level generation unit 15H, etc.

[0104] In this way, the server device 10 operates as an information processing device that executes the generation method by reading and executing the program. The server device 10 can also realize functions similar to those of the above-described embodiment by reading the program from a recording medium using a media reading device and executing the read program. Note that the program in these other embodiments is not limited to being executed by the server device 10. For example, the functions of the present disclosure can be similarly applied when another computer or server executes the program, or when these execute the program in cooperation with each other.

[0105] The above program can be distributed via a network such as the Internet. The above program can also be recorded on any recording medium and executed by a computer by reading it from the recording medium. For example, the recording medium can be a hard disk, a flexible disk (FD), a CD-ROM, a magneto-optical disk (MO), a digital versatile disk (DVD), or the like.

[0106] REFERENCE SIGNS LIST 10 Server device 11 Communication control unit 13 Storage unit 13A Factor level 13B Factor level map 15 Control unit 15A Reception unit 15B Image generation unit 15C Evaluation unit 15D Factor level map generation unit 15E Output unit 15F Misjudgment feature extraction unit 15G Text expansion unit 15H New factor level generation unit 30 Client terminal< / blank>

Claims

1. A generation program that causes a computer to execute the following process: for input images judged by a machine learning model and misclassified, encode the features of the image into a space different from the image space; expand the encoded features in categories that include the features; and generate a factor level table to be used in the machine learning model, in which the categories are set as factor candidates and the encoded features and the expanded features are added as levels.

2. The generation program according to claim 1, wherein the encoding process includes a process of encoding features of the image from the image space to a language space.

3. The generation program according to claim 2, wherein the encoding process is realized by a large-scale language model.

4. The generation program described in claim 2, characterized in that the expansion process includes a process of expanding the first word string to a second word string by generating an answer to a fill-in-the-blank question in which some of the first word string encoded from the image space to the language space has blanks set.

5. The generation program according to claim 1, further causing the computer to perform the following processes: evaluate the machine learning model using the generated factor level table; determine the effectiveness of the factor level table using the evaluation result; and update the factor level table based on the determination result.

6. The generation program of claim 5, wherein the determining process includes a process of determining whether the factor level table is valid based on whether there is one or more rows or columns corresponding to the added factor levels, which include one or more first cells in the factor level table where the evaluation of the machine learning model is equal to or greater than a first threshold, and one or more second cells where the evaluation of the machine learning model is less than a second threshold that is smaller than the first threshold.

7. A generation method characterized by a computer executing the following processes: for an input image judged by a machine learning model and for an incorrectly judged image, the features of the image are encoded in a space different from the image space; for a category containing the encoded features, the features are expanded; and a factor level table to be used in the machine learning model is generated, in which the categories are set as factor candidates and the encoded features and the expanded features are added as levels.

8. The method of claim 7, wherein said encoding step includes encoding features of said image from said image space into a language space.

9. The method of claim 8, wherein the encoding process is implemented using a large-scale language model.

10. The generation method described in claim 8, characterized in that the expansion process includes a process of expanding the first word string to a second word string by generating an answer to a fill-in-the-blank question in which some of the first word string encoded from the image space to the language space has blanks set.

11. The generation method according to claim 7, wherein the computer further performs the following processes: evaluating the machine learning model using the generated factor level table; determining the effectiveness of the factor level table using the evaluation result; and updating the factor level table based on the determination result.

12. The generation method described in claim 11, characterized in that the determining process includes a process of determining whether the factor level table is valid based on whether there is one or more rows or columns corresponding to the added factor levels, which include one or more first cells in the factor level table where the evaluation of the machine learning model is equal to or greater than a first threshold, and one or more second cells where the evaluation of the machine learning model is less than a second threshold that is smaller than the first threshold.

13. An information processing device comprising: a control unit that executes the following process: for an input image judged by a machine learning model and judged incorrectly, encode the features of the image into a space different from the image space; expand the encoded features in a category containing the features; and generate a factor level table to be used in the machine learning model, in which the categories are set as factor candidates and the encoded features and the expanded features are added as levels.

14. The information processing apparatus according to claim 13, wherein said encoding process includes a process of encoding features of said image from said image space to a language space.

15. The information processing apparatus according to claim 14, wherein said encoding process is realized by a large-scale language model.

16. The information processing device described in claim 14, characterized in that the expansion process includes a process of expanding the first word string to a second word string by generating an answer to a fill-in-the-blank question in which a blank is set in part of the first word string encoded from the image space to the language space.

17. The information processing device according to claim 13, wherein the control unit further executes the following processes: evaluating the machine learning model using the generated factor level table; determining the effectiveness of the factor level table using the evaluation result; and updating the factor level table based on the determination result.

18. The information processing device described in claim 17, characterized in that the determining process includes a process of determining whether the factor level table is valid based on whether there is one or more rows or columns corresponding to the added factor level, which include one or more first cells in the factor level table where the evaluation of the machine learning model is equal to or greater than a first threshold, and one or more second cells where the evaluation of the machine learning model is less than a second threshold that is smaller than the first threshold.

Images