Information processor and information processing program

Abstract

To prevent reduction of information amount in an image caption while generating the image caption with accurate content.SOLUTION: The information processor according to the present application includes: an acquisition unit configured to acquire, for an input target image, a target caption that is a sentence describing contents of the target image by using a machine learning model; a calculation unit configured to calculate an evaluation value for evaluating accuracy of the target caption and a sentence length value indicating the length of the target caption; and a model generation unit configured to generate a machine learning model trained by reinforcement learning by setting a reward value based on the evaluation value and the sentence length value.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing program. [Background technology]

[0002] Conventionally, there are known techniques for generating image captions, which are sentences that explain the contents of an image, from an image. For example, there is known a technique for generating an image caption from an image using a machine learning model that generates an image caption from an image (hereinafter, sometimes referred to as a "caption generation model").

[0003] The Consensus-based Image Description Evaluation (CIDEr) score is known as an evaluation value for assessing the accuracy of image captions generated by a caption generation model, and Self-critical Sequence Training for Image Captioning (SCST), a type of reinforcement learning, is known as a learning method for improving the CIDEr score. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Steven J. Rennie, Etienne Marcheret1, Youssef Mroueh, Jerret Ross and Vaibhava Goel, "Self-critical Sequence Training for Image Captioning", CVPR 2017, 2017, [Retrieved February 15, 2020], Internet <URL:https: / / openaccess.thecvf.com / content_cvpr_2017 / papers / Rennie_Self-Critical_Sequence_Training_CVPR_2017_paper.pdf> Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned conventional techniques tend to shorten the length of image captions, and therefore, the above-mentioned conventional techniques do not necessarily enable the generation of image captions with accurate content while preventing a decrease in the amount of information contained in the image captions.

[0006] The present application aims to provide an information processing device and an information processing program that can generate image captions with accurate content while preventing a decrease in the amount of information contained in the image captions. [Means for solving the problem]

[0007] The information processing device of the present application includes an acquisition unit that acquires a target caption, which is a sentence that explains the content of an input target image, using a machine learning model; a calculation unit that calculates an evaluation value that evaluates the accuracy of the target caption and a sentence length value that indicates the length of the target caption; and a model generation unit that generates the machine learning model that has undergone reinforcement learning by setting a reward value based on the evaluation value and the sentence length value. [Effects of the Invention]

[0008] According to one aspect of the embodiment, it is possible to generate image captions with accurate content while preventing a decrease in the amount of information contained in the image captions. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram for explaining the image caption generation process using the greedy method. [Figure 2] FIG. 2 is a diagram for explaining the process of generating an image caption using a search method. [Figure 3]FIG. 3 is a diagram for explaining a method for learning a visual language model according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of the configuration of the information processing device according to the embodiment. [Figure 5] FIG. 5 is a diagram for explaining a method for calculating a sentence length value according to the embodiment. [Figure 6] FIG. 6 is a flowchart showing the procedure of information processing by the information processing device according to the embodiment. [Figure 7] FIG. 7 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, a detailed description will be given of an information processing device and an information processing program according to the present application (hereinafter referred to as an "embodiment") with reference to the drawings. Note that the information processing device and the information processing program according to the present application are not limited to the embodiment. Furthermore, the same components in the following embodiments are denoted by the same reference numerals, and duplicated descriptions will be omitted.

[0011] (Embodiment) 1. Introduction Conventionally, there is known a technology related to a machine learning model (hereinafter, sometimes referred to as a "caption generation model") that generates an image caption, which is a sentence that explains the content of an image, from an image. In recent years, a machine learning model called a visual language model (VLM) has become mainstream as a caption generation model. The following describes the case where the caption generation model is a visual language model.

[0012] A visual language model is a machine learning model that is pre-trained based on training data including pairs of images and corresponding image captions. For example, a visual language model is a machine learning model trained to output an image caption corresponding to an image when an image included in the training data is input. For example, a visual language model is trained to predict the correct next token from an image and given (partially) tokens. For example, a visual language model is trained to estimate the next token from a sequence of tokens being generated. For example, the visual language model may be CoCa (Contrastive Captioners are Image-Text Foundation Models), BLIP (Bootstrapping Language-Image Pre-training), BLIP2, GIT (Generative Image to Text Transformer), etc.

[0013] The Consensus-based Image Description Evaluation (CIDEr) score is known as an evaluation value for assessing the accuracy of image captions generated by a visual language model. Self-critical Sequence Training for Image Captioning (SCST) is also known as a learning method for improving the CIDEr score. SCST is a reinforcement learning method for a visual language model using the CIDEr score as a reward value. Specifically, there are two methods for generating image captions by a visual language model: the greedy method and the search method. A typical visual language model can switch between the greedy method and the search method when generating image captions. In SCST, the visual language model is reinforced by setting the reward value as the value obtained by subtracting the CIDEr score of the image caption generated by the search method from the CIDEr score of the image caption generated by the greedy method. The difference between the greedy method and the search method is explained in detail using Figures 1 and 2.

[0014] Figure 1 is a diagram for explaining the process of generating an image caption using the greedy method. Figure 1 shows how a visual language model 1 generates an image caption 31 corresponding to an image 20 from an image 20 using the greedy method. The greedy method is a method in which, when the visual language model 1 outputs the next token for each token, it selects and outputs the token with the highest probability of appearing next (also called the occurrence probability) from among multiple next token candidates.

[0015] In FIG. 1, first, the visual language model 1 outputs token 311. Then, when outputting the token next to token 311, the visual language model 1 outputs token 312, which has the highest probability of appearance among multiple next token candidates. Then, when outputting the token next to token 312, the visual language model 1 outputs token 313, which has the highest probability of appearance among multiple next token candidates. In this way, when outputting the token next to each token, the visual language model 1 selects and outputs the token with the highest probability of appearing next from multiple next token candidates, thereby generating image caption 31 using a greedy method.

[0016] Also, in Figure 1, the CIDEr score of the image caption 31 generated by the greedy method is shown as CIDEr score (greedy method) 41. As explained in Figure 1, the greedy method is characterized in that the visual language model 1 continues to select the token with the highest occurrence probability. In other words, the greedy method is characterized in that the visual language model 1 continues to output only the token with the highest occurrence probability. Furthermore, the image caption generated by the visual language model 1 continuing to select the token with the highest occurrence probability can also be said to be the image caption generated as a result of the visual language model 1 continuing to make the best choice at each point in time. Therefore, the image caption generated by the greedy method serves as a benchmark indicating the current performance of the visual language model 1. For example, the CIDEr score of the image caption 31 generated by the greedy method ("CIDEr score (greedy method) 41" in Figure 1) serves as a benchmark score indicating the current performance of the visual language model 1.

[0017] FIG. 2 is a diagram illustrating the process of generating an image caption using a search method. FIG. 2 shows how the visual language model 1 generates an image caption 32 corresponding to an image 20 from an image 20 using a search method. The search method is a method in which, when the visual language model 1 outputs the next token for each token, it selects two or more tokens from multiple next token candidates in order of probability of appearing next, and outputs one token selected from the two or more selected tokens according to a predetermined selection rule. For example, the predetermined selection rule may be a rule that selects one token from the two or more selected tokens according to some probabilistic rule. For example, the predetermined selection rule may be a rule that randomly selects one token from the two or more selected tokens.

[0018] In FIG. 2, first, the visual language model 1 selects two tokens 321 and 322 with the highest probability of appearing next from among token 321 with a 30% probability of appearing next, token 322 with a 50% probability of appearing next, and token 323 with a 20% probability of appearing next. Next, the visual language model 1 outputs one token 321 randomly selected from the two selected tokens 321 and 322. Next, the visual language model 1 selects two tokens 324 and 325 with the highest probability of appearing next from among token 324 with a 40% probability of appearing next, token 325 with a 50% probability of appearing next, and token 326 with a 10% probability of appearing next. Next, the visual language model 1 outputs one token 325 randomly selected from the two selected tokens 324 and 325. In this way, when outputting the next token for each token, the visual language model 1 selects two or more tokens from multiple next token candidates based on the probability of them appearing next, and outputs one token selected from the two or more selected tokens according to a predetermined selection rule, thereby generating an image caption 32 using a search method.

[0019] Also, in FIG. 2, the CIDEr score of the image caption 32 obtained by the search method is indicated by CIDEr score (search method) 42. As explained in FIG. 2, the search method differs from the greedy method in that the visual language model 1 does not necessarily continue to output only tokens with the highest occurrence probability. In other words, the search method may generate an image caption of quality that exceeds the current capabilities of the visual language model 1 by outputting tokens probabilistically selected from tokens that the visual language model 1 has an occurrence probability equal to or higher than a certain level. In other words, the image caption generated by the search method may exceed the current capabilities of the visual language model 1. For example, the CIDEr score of the image caption 32 obtained by the search method ("CIDEr score (search method) 42" in FIG. 2) may exceed the reference score indicating the current capabilities of the visual language model 1 (corresponding to "CIDEr score (greedy method) 41" explained in FIG. 1).

[0020] Therefore, in SCST, the visual language model 1 is reinforced to improve the quality of image captions generated by the search method. That is, in SCST, the visual language model 1 is reinforced to improve the CIDEr score of image captions generated by the search method. Specifically, in SCST, the visual language model 1 is reinforced to learn by setting the CIDEr score of an image caption generated by the search method as a reward value. More specifically, in SCST, the visual language model 1 is reinforced to learn by setting the reward value as the value obtained by subtracting the CIDEr score of an image caption generated by the greedy method, "CIDEr score (greedy method)", from the CIDEr score of an image caption generated by the search method, "CIDEr score (search method)". The reward value of SCST is expressed by the following equation (1):

[0021]

number

[0022] The loss function of SCST is expressed by taking the logarithm of the probability of occurrence of each token included in each of the multiple image captions generated by the search method, multiplying the logarithm of the probability of occurrence of each token, and multiplying this by the reward value expressed in equation (1) above. The loss function of SCST is expressed by equation (2) below. Note that Σ in equation (2) below indicates that the sum for each of the multiple image captions is taken.

[0023]

number

[0024] In SCST, the visual language model 1 is reinforced to learn so as to minimize the value of the loss function expressed by the above formula (2). In SCST, the parameter values ​​of the visual language model 1 are learned so as to minimize the value of the loss function expressed by the above formula (2).

[0025] Furthermore, when SCST is applied to the visual language model 1 using the CIDEr score as a reward value, the length of image captions (hereinafter sometimes referred to as "generated captions") generated by the trained visual language model 1 tends to be shorter. This is because the CIDEr score increases as the similarity between the correct image caption and the generated caption increases. Specifically, the longer the length of the generated caption, the higher the likelihood that it will contain erroneous information not included in the correct image caption, and therefore the higher the likelihood that the CIDEr score will be low. To avoid this, the visual language model 1 is thought to generate shorter image captions. However, a shorter generated caption corresponds to a smaller amount of information contained in the generated caption. Furthermore, a smaller amount of information contained in the generated caption corresponds to a lack of information that should have been included in the generated caption. Therefore, a technology that prevents the length of generated captions from becoming shorter is desired. In other words, a technology that prevents a decrease in the amount of information contained in the generated caption is desired.

[0026] In contrast, an information processing device according to an embodiment generates a reinforcement-learned visual language model by setting a reward value based on the CIDEr score and a sentence length value indicating the length of the generated caption. FIG. 3 is a diagram for explaining a training method for a visual language model according to an embodiment. FIG. 3 differs from FIG. 2 in that a sentence length value indicating the length of the generated caption is set as the reward value for reinforcement learning. The reward value according to the embodiment is expressed by the following formula (3). The following formula (3) is obtained by adding a term corresponding to the sentence length value indicating the length of the generated caption to the above formula (1).

[0027]

number

[0028] In this way, the information processing device according to the embodiment sets a reward value based on the CIDEr score and a sentence length value indicating the length of the generated caption, and performs reinforcement learning on the visual language model. This allows the information processing device to maximize the CIDEr score while preventing the length of the generated caption from becoming too short. This allows the information processing device to generate image captions that have accurate content and are not too short. Therefore, the information processing device can generate image captions with accurate content while preventing a decrease in the amount of information contained in the image captions.

[0029] 2. Configuration of Information Processing Device An example of the configuration of the information processing device 100 according to the embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the configuration of the information processing device 100 according to the embodiment. The information processing device 100 includes a communication unit 110, a storage unit 120, and a control unit 130.

[0030] (Communication unit 110) The communication unit 110 is realized by a NIC (Network Interface Card), an antenna, etc. The communication unit 110 is connected to various networks by wire or wirelessly, and transmits and receives information to and from other information processing devices other than the information processing device 100, for example.

[0031] (Storage unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. Specifically, the storage unit 120 stores various data. For example, the storage unit 120 stores various programs. For example, the storage unit 120 stores an information processing program according to an embodiment. The storage unit 120 may also store information related to a machine learning model that generates an image caption, which is a sentence that describes the content of an image, from an image. For example, the storage unit 120 may store information related to a visual language model that is a machine learning model that generates an image caption from an image. The storage unit 120 may also store information related to a target image acquired by the acquisition unit 131. The storage unit 120 may also store information related to a machine learning model generated by the model generation unit 133. For example, the storage unit 120 may store information related to a visual language model generated by the model generation unit 133.

[0032] (control unit 130) The control unit 130 is a controller, and is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) executing various programs stored in a storage device inside the information processing device 100 using RAM as a work area. The control unit 130 is also a controller, and is realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0033] The control unit 130 has an acquisition unit 131, a calculation unit 132, a model generation unit 133, and a sentence generation unit 134 as functional units, and may realize or execute the information processing actions described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 4, and may be any other configuration that performs the information processing described below. Furthermore, each functional unit indicates a function of the control unit 130, and does not necessarily have to be physically distinct.

[0034] (Acquisition part 131) The acquisition unit 131 may acquire a target image for which an image caption is to be generated. The target image may include not only a still image but also a moving image. Specifically, the acquisition unit 131 may acquire the target image from an external information processing device via the communication unit 110. When the acquisition unit 131 acquires the target image, the acquisition unit 131 may store information related to the target image in the storage unit 120.

[0035] Furthermore, the acquisition unit 131 may acquire a machine learning model that generates an image caption, which is a sentence that describes the content of an image, from an image. Specifically, the acquisition unit 131 may acquire a visual language model, which is a machine learning model that generates an image caption, which is a sentence that describes the content of an image, from an image. More specifically, the acquisition unit 131 may acquire the visual language model by referring to the storage unit 120. For example, when the acquisition unit 131 acquires a target image, it may acquire the visual language model. For example, the acquisition unit 131 may acquire a machine learning model that has been trained in advance based on training data including pairs of images and image captions corresponding to the images. For example, the acquisition unit 131 may acquire a machine learning model that has been trained to output an image caption corresponding to an image when an image included in the training data is input. For example, the acquisition unit 131 may acquire a machine learning model that has been trained to predict the correct next token from an image and given (partially acquired) tokens. For example, the acquisition unit 131 may acquire a machine learning model that has been trained to estimate the next token from a sequence of tokens currently being generated. For example, the acquisition unit 131 may acquire a machine learning model that has been trained to estimate and output the next token from an input image and token sequence, such as a visual language model such as CoCa, BLIP, BLIP2, or GIT.

[0036] Furthermore, the acquisition unit 131 acquires a target caption, which is a sentence that explains the content of the input target image, from the input target image using a machine learning model. Specifically, the acquisition unit 131 generates a target caption from the target image using a machine learning model that generates an image caption, which is a sentence that explains the content of the image, from an image. For example, the acquisition unit 131 may generate a target caption, which is an image caption corresponding to the target image, using a visual language model. For example, when the acquisition unit 131 acquires a target image and a visual language model, it may generate a target caption, which is an image caption corresponding to the target image, using the visual language model. More specifically, the acquisition unit 131 may input the target image to a machine learning model that generates an image caption, which is a sentence that explains the content of the image, from an image, and cause the machine learning model to generate the target caption. The acquisition unit 131 may generate the target caption by causing the machine learning model to generate the target caption. Note that, hereinafter, the "target caption" may be referred to as a "generated caption." For example, the acquisition unit 131 may generate an image caption corresponding to the target image by inputting the target image into a visual language model. For example, the acquisition unit 131 may input the target image into the visual language model to acquire a generated caption, which is an image caption output from the visual language model. In this way, the acquisition unit 131 may acquire a generated caption, which is an image caption generated by a visual language model that is a machine learning model that generates an image caption, which is a sentence that describes the content of the image, from an image. Furthermore, when the acquisition unit 131 acquires the generated caption, it may output the acquired generated caption to the calculation unit 132.

[0037] For example, the acquisition unit 131 may acquire a greedy caption, which is a generated caption generated by a greedy method. Specifically, the acquisition unit 131 may acquire a greedy caption output from the visual language model by inputting a target image into the visual language model with the visual language model switched to a greedy method mode. For example, when the visual language model outputs the next token for each token, the acquisition unit 131 may generate a greedy caption by selecting and outputting a token with the highest probability of appearing next from multiple next token candidates. Furthermore, the acquisition unit 131 may acquire the generated greedy caption. Furthermore, when the acquisition unit 131 acquires a greedy caption, it may output the acquired greedy caption to the calculation unit 132.

[0038] The acquisition unit 131 may also acquire a search caption, which is a target caption generated by the search method. The acquisition unit 131 may also acquire a search caption, which is a generated caption generated by the search method. Specifically, the acquisition unit 131 may acquire a search caption output from the visual language model by inputting a target image into the visual language model while the visual language model is switched to the search method mode. For example, when the visual language model outputs the next token for each token, the acquisition unit 131 may generate a search caption by selecting two or more tokens from multiple next token candidates in order of probability of next appearance, and outputting one token selected from the selected two or more tokens according to a predetermined selection rule. The acquisition unit 131 may also acquire the generated search caption. When the acquisition unit 131 acquires a search caption, it may output the acquired search caption to the calculation unit 132.

[0039] (Calculation unit 132) The calculation unit 132 may acquire a target caption from the acquisition unit 131. The calculation unit 132 may acquire a generated caption from the acquisition unit 131. Furthermore, when the calculation unit 132 acquires a generated caption from the acquisition unit 131, it may calculate an evaluation value that evaluates the accuracy of the acquired generated caption. The evaluation value that evaluates the accuracy of the generated caption is, in other words, an evaluation value that evaluates the quality of the generated caption. Generally, the evaluation value that evaluates the generated caption is calculated based on how similar the generated caption is to a correct caption. Here, the correct caption is a sentence that is prepared in advance as a correct answer for the generated caption. The correct caption is a sentence that explains the content of the target image and is a correct sentence that is prepared in advance as a correct answer for the generated caption. In other words, the more similar the generated caption is to the correct caption, the more accurate the generated caption is. Furthermore, the more similar the generated caption is to the correct caption, the higher the quality of the generated caption.

[0040] Furthermore, the calculation unit 132 may calculate an evaluation value based on the similarity between a correct caption and a target caption. The calculation unit 132 may calculate an evaluation value based on the similarity between a correct caption and a generated caption. For example, the calculation unit 132 may calculate an evaluation value that takes a larger value as the similarity between the correct caption and the generated caption increases. Furthermore, a high degree of match between n-grams (combinations of n consecutive tokens) between the correct caption and the generated caption corresponds to similarity between the correct caption and the generated caption. For example, the calculation unit 132 may calculate an evaluation value that takes a larger value as the degree of match between n-grams between the correct caption and the generated caption increases.

[0041] Furthermore, the calculation unit 132 may calculate an evaluation value that is a CIDEr (Consensus-based Image Description Evaluation) score of the target caption. The calculation unit 132 may calculate an evaluation value that is a CIDEr score of the generated caption. CIDEr is an evaluation index used to evaluate the quality of the generated caption. The evaluation value corresponding to CIDEr is called a CIDEr score. The higher the degree of match between the n-grams of the correct caption for the image and the generated caption, the larger the CIDEr score. The CIDEr score also takes into account TF-IDF, an evaluation index that evaluates word rarity, to evaluate the degree of match between the n-grams of the correct caption for the image and the generated caption. Specifically, the CIDEr score calculates TF-IDF weights so that n-grams that appear in many images are given a smaller weight and n-grams that appear only in specific images are given a larger weight.

[0042] Furthermore, the calculation unit 132 may acquire a greedy caption from the acquisition unit 131. Furthermore, when the calculation unit 132 acquires a greedy caption from the acquisition unit 131, the calculation unit 132 may calculate a greedy evaluation value that evaluates the accuracy of the acquired greedy caption. For example, the calculation unit 132 may calculate a greedy evaluation value that is a CIDEr score of the greedy caption. Furthermore, when the calculation unit 132 calculates the greedy evaluation value, the calculation unit 132 may output the greedy evaluation value to the model generation unit 133.

[0043] Furthermore, the calculation unit 132 may acquire a search caption from the acquisition unit 131. Furthermore, when the calculation unit 132 acquires a search caption from the acquisition unit 131, the calculation unit 132 may calculate a search evaluation value that evaluates the accuracy of the acquired search caption. For example, the calculation unit 132 may calculate a search evaluation value that is a CIDEr score of the search caption. Furthermore, when the calculation unit 132 calculates the search evaluation value, the calculation unit 132 may output the search evaluation value to the model generation unit 133.

[0044] Furthermore, when the calculation unit 132 acquires a generated caption from the acquisition unit 131, the calculation unit 132 may calculate a sentence length value indicating the length of the acquired generated caption. FIG. 5 is a diagram for explaining a method for calculating a sentence length value according to the embodiment. FIG. 5 shows a string of tokens included in the generated caption acquired by the calculation unit 132 from the acquisition unit 131. In FIG. 5, the generated caption is calculated based on a start token <bos>(BOS stands for "Begging of Sequence"), the first token A1, the second token A2, ..., the (N-1)th (N is a natural number) token A(N-1) (not shown), and the end token <eos>(EOS stands for "End of Sequence") and consists of a sequence of tokens arranged in order from left to right.

[0045] Furthermore, the calculation unit 132 may assign an index to each token included in the generated caption. For example, the calculation unit 132 may assign an index to each token, which is a number that increases by one in the order in which the tokens are arranged. In FIG. 5, the calculation unit 132 assigns the number 0 to the starting token. <bos>The calculation unit 132 assigns the number 1 as the index of the first token A1 that follows the start token. The calculation unit 132 assigns the number 2 as the index of the second token A2 that follows the first token A1. Similarly, the calculation unit 132 assigns the number (N-1) as the index of the (N-1)th token. The calculation unit 132 assigns the number N as the index of the end token <eos>is assigned as an index.

[0046] Furthermore, the calculation unit 132 may calculate a sentence length value based on the length of a sequence of tokens included in the target caption. The calculation unit 132 may calculate a sentence length value based on the length of a sequence of tokens included in the generated caption. The calculation unit 132 may calculate a sentence length value based on the length of a sequence of tokens included in the generated caption based on the index of each token included in the generated caption. For example, the calculation unit 132 may calculate a sentence length value based on the length of a sequence of tokens included in the generated caption, excluding a start token and an end token. For example, the calculation unit 132 may calculate a value obtained by subtracting the index of the end token from the index of the start token. Next, the calculation unit 132 may calculate a value obtained by subtracting the index of the end token from the index of the start token, and then subtracting 1 from the value obtained by subtracting the index of the end token from the index of the start token, as the sentence length value of the generated caption. In FIG. 5, the calculation unit 132 may calculate N, which is the value obtained by subtracting 0, which is the index of the start token, from N, which is the index of the end token. Next, the calculation unit 132 may calculate the sentence length value as (N-1), which is the value obtained by subtracting 1 from N. In this way, the calculation unit 132 may calculate the sentence length value based on the length of the string of tokens included in the generated caption.

[0047] Furthermore, when the calculation unit 132 calculates the sentence length value, it may output the calculated sentence length value to the model generation unit 133. For example, when the calculation unit 132 acquires a search caption from the acquisition unit 131, it may calculate a search sentence length value that is the sentence length value of the acquired search caption. Furthermore, when the calculation unit 132 calculates the search sentence length value, it may output the calculated search sentence length value to the model generation unit 133.

[0048] As described above, the calculation unit 132 calculates an evaluation value that evaluates the accuracy of the target caption and a sentence length value that indicates the length of the target caption. For example, the calculation unit 132 calculates a search evaluation value that is an evaluation value of the searched caption and a search sentence length value that is a sentence length value of the searched caption.

[0049] (Model generation unit 133) The model generation unit 133 generates a reinforcement-learned machine learning model by setting a reward value based on the evaluation value and sentence length value calculated by the calculation unit 132. For example, the model generation unit 133 generates a reinforcement-learned visual language model by setting a reward value based on the evaluation value and sentence length value calculated by the calculation unit 132. For example, the model generation unit 133 generates a reinforcement-learned machine learning model by setting a reward value based on the search evaluation value and search sentence length value calculated by the calculation unit 132. For example, the model generation unit 133 generates a reinforcement-learned visual language model by setting a reward value based on the search evaluation value and search sentence length value calculated by the calculation unit 132. For example, the model generation unit 133 acquires the greedy evaluation value, the search evaluation value, and the search sentence length value from the calculation unit 132. When the model generation unit 133 acquires the greedy evaluation value, the search evaluation value, and the search sentence length value from the calculation unit 132, the model generation unit 133 may set the value obtained by subtracting the greedy evaluation value from the search evaluation value and adding the search sentence length value to the search evaluation value as the reward value for reinforcement learning of the visual language model. For example, the model generation unit 133 may set a value obtained by subtracting the CIDEr score of the greedy caption (hereinafter referred to as "CIDEr score (search method)") from the search evaluation value, which is the CIDEr score of the searched caption (hereinafter referred to as "CIDEr score (search method)"), and adding the result to the search sentence length value, as a reward value for reinforcement learning of the visual language model. For example, the model generation unit 133 may generate a reinforcement-learned visual language model by setting a reward value expressed by the above formula (3). Here, the search sentence length value corresponds to the "sentence length value" in the above formula (3). The model generation unit 133 may generate a reinforcement-learned visual language model so as to maximize the reward value expressed by the above formula (3).

[0050] Furthermore, the model generation unit 133 may perform reinforcement learning on the visual language model so as to minimize, for each search caption, the value of a loss function expressed by the sum of the products of the logarithm of the occurrence probability of each token included in the search caption and the reward value expressed by the above formula (3). More specifically, the model generation unit 133 may perform reinforcement learning on the visual language model so as to minimize the value of a loss function expressed by the same formula as the above formula (2). The model generation unit 133 may perform reinforcement learning on the visual language model so as to minimize the value of a loss function obtained by replacing the value of the reward value in the above formula (2) with the reward value expressed by the above formula (3). The model generation unit 133 learns the values ​​of the parameters of the visual language model so as to minimize the value of the loss function obtained by replacing the value of the reward value in the above formula (2) with the reward value expressed by the above formula (3).

[0051] Furthermore, when the model generation unit 133 generates a visual language model through reinforcement learning based on the reward value expressed by the above formula (3), it may store information about the generated visual language model in the storage unit 120. Furthermore, when the model generation unit 133 generates a visual language model through reinforcement learning based on the reward value expressed by the above formula (3), it may output the generated visual language model to the sentence generation unit 134.

[0052] (Sentence generation section 134) The sentence generation unit 134 may acquire a machine learning model generated by the model generation unit 133. For example, the sentence generation unit 134 may acquire a visual language model generated by the model generation unit 133. For example, the sentence generation unit 134 may acquire the visual language model generated by the model generation unit 133 by referring to the storage unit 120. Alternatively, the sentence generation unit 134 may acquire the visual language model from the model generation unit 133. Furthermore, when the sentence generation unit 134 acquires the visual language model generated by the model generation unit 133, it may acquire an image to be processed. For example, the sentence generation unit 134 may acquire an image to be processed from an external information processing device via the communication unit 110.

[0053] Furthermore, the sentence generation unit 134 may generate an image caption corresponding to the processing target image from the processing target image using the machine learning model generated by the model generation unit 133. For example, the sentence generation unit 134 may generate an image caption corresponding to the processing target image from the processing target image using the visual language model generated by the model generation unit 133. For example, when the sentence generation unit 134 acquires the visual language model generated by the model generation unit 133 and the processing target image, the sentence generation unit 134 may generate an image caption corresponding to the processing target image from the processing target image using the visual language model generated by the model generation unit 133. Specifically, the sentence generation unit 134 may generate an image caption corresponding to the processing target image by inputting the processing target image into the visual language model generated by the model generation unit 133. For example, the sentence generation unit 134 may acquire an image caption corresponding to the processing target image output from the visual language model by inputting the processing target image into the visual language model generated by the model generation unit 133.

[0054] [3. Processing Procedure] FIG. 6 is a flowchart showing the procedure of information processing by the information processing device 100 according to the embodiment. In FIG. 6, the acquisition unit 131 of the information processing device 100 acquires a generated caption, which is an image caption generated by a visual language model (step S11). The calculation unit 132 of the information processing device 100 calculates an evaluation value that evaluates the accuracy of the generated caption acquired by the acquisition unit 131 and a sentence length value that indicates the length of the generated caption (step S12). The model generation unit 133 of the information processing device 100 generates a visual language model that has undergone reinforcement learning by setting a reward value based on the evaluation value and sentence length value calculated by the calculation unit 132 (step S13). The sentence generation unit 134 of the information processing device 100 generates an image caption corresponding to the image to be processed from the image to be processed, using the visual language model generated by the model generation unit 133 (step S14).

[0055] [4. Modifications] The processing according to the above-described embodiment may be implemented in various different forms other than the above embodiment.

[0056] In the above-described embodiment, the model generation unit 133 sets the value obtained by subtracting the greedy evaluation value from the search evaluation value and adding the search sentence length value as the reward value for reinforcement learning of the visual language model. However, the reward value for reinforcement learning of the visual language model is not limited to this. For example, the model generation unit 133 may set the value obtained by adding the search evaluation value and the search sentence length value as the reward value for reinforcement learning of the visual language model. For example, the model generation unit 133 may set the value obtained by adding the search evaluation value, which is the CIDEr score of the searched caption, and the search sentence length value as the reward value for reinforcement learning of the visual language model.

[0057] In the above-described embodiment, the CIDEr score is used as an evaluation value for evaluating the accuracy of the generated caption, but the evaluation value for evaluating the accuracy of the generated caption is not limited to the CIDEr score. For example, the evaluation value for evaluating the accuracy of the generated caption may be a score corresponding to an evaluation index such as BLEU (Bilingual Evaluation Understudy), METEOR (Metric for Evaluation of Translation with Explicit Ordering), or SPICE (Semantic Propositional Image Caption Evaluation). Furthermore, the evaluation value for evaluating the accuracy of the generated caption may be an evaluation value calculated by combining multiple evaluation indexes.

[0058] [5. Effects] As described above, the information processing device 100 according to the embodiment includes an acquisition unit 131, a calculation unit 132, and a model generation unit 133. The acquisition unit 131 acquires a target caption, which is a sentence that explains the content of an input target image, for an input target image using a machine learning model. The calculation unit 132 calculates an evaluation value that evaluates the accuracy of the target caption and a sentence length value that indicates the length of the target caption. The model generation unit 133 generates a machine learning model that has undergone reinforcement learning by setting a reward value based on the evaluation value and the sentence length value.

[0059] In this way, the information processing device 100 sets a reward value based on an evaluation value that evaluates the accuracy of an image caption, which is a sentence that describes the content of an image, and a sentence length value that indicates the length of the image caption, and performs reinforcement learning on the machine learning model. This maximizes the evaluation value that evaluates the accuracy of the image caption while preventing the length of the image caption from becoming too short. This allows the information processing device 100 to generate image captions that have accurate content and are not too short. Therefore, the information processing device 100 can generate image captions with accurate content while preventing a decrease in the amount of information contained in the image caption. Furthermore, the information processing device 100 can generate image captions with accurate content while preventing a decrease in the amount of information contained in the image caption, thereby contributing to the achievement of Goal 9 of the Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote inclusive and sustainable industrialization, and promote innovation and infrastructure."

[0060] The acquisition unit 131 acquires a search caption, which is a target caption generated by the search method. The calculation unit 132 calculates a search evaluation value, which is an evaluation value of the search caption, and a search sentence length value, which is a sentence length value of the search caption. The model generation unit 133 generates a machine learning model that has undergone reinforcement learning by setting a reward value based on the search evaluation value and the search sentence length value.

[0061] As a result, the information processing device 100 can generate search captions with accurate content while preventing a decrease in the amount of information contained in the search captions by setting an evaluation value that evaluates the accuracy of the search captions and a reward value based on a sentence length value that indicates the length of the search captions and performing reinforcement learning on the machine learning model.

[0062] Furthermore, the calculation unit 132 calculates an evaluation value based on the similarity between the correct caption and the target caption.

[0063] This enables the information processing device 100 to generate image captions with accurate content that maximizes the evaluation value based on the similarity between the correct caption and the image caption generated by the machine learning model, while preventing a decrease in the amount of information contained in the image caption.

[0064] The calculation unit 132 also calculates an evaluation value, which is a Consensus-based Image Description Evaluation (CIDEr) score of the target caption.

[0065] This allows the information processing device 100 to generate an image caption with appropriate content that maximizes the CIDEr score, while preventing a decrease in the amount of information included in the image caption.

[0066] The calculation unit 132 also calculates a sentence length value based on the length of the string of tokens included in the target caption.

[0067] In this way, the information processing device 100 can prevent the length of the string of tokens included in the image caption from becoming shorter by setting the length of the string of tokens included in the image caption as the reward value. Therefore, the information processing device 100 can prevent a decrease in the amount of information included in the image caption.

[0068] The information processing device 100 further includes a sentence generation unit 134. The sentence generation unit 134 uses the machine learning model generated by the model generation unit 133 to generate an image caption corresponding to the image to be processed from the image to be processed.

[0069] This allows the information processing device 100 to generate image captions with accurate content while preventing a decrease in the amount of information included in the image captions.

[0070] [6. Hardware Configuration] The information processing device 100 according to the embodiment described above is realized by, for example, a computer 1000 configured as shown in Fig. 7. Fig. 7 is a hardware configuration diagram showing an example of a computer that realizes the functions of the information processing device 100. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM 1300, an HDD 1400, a communication interface (I / F) 1500, an input / output interface (I / F) 1600, and a media interface (I / F) 1700.

[0071] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. The ROM 1300 stores a boot program executed by the CPU 1100 when the computer 1000 starts up, programs that depend on the hardware of the computer 1000, and the like.

[0072] The HDD 1400 stores programs executed by the CPU 1100, data used by such programs, etc. The communication interface 1500 receives data from other devices via a predetermined communication network and sends it to the CPU 1100, and transmits data generated by the CPU 1100 to other devices via the predetermined communication network.

[0073] The CPU 1100 controls output devices such as a display and a printer, and input devices such as a keyboard and a mouse, via the input / output interface 1600. The CPU 1100 acquires data from the input devices via the input / output interface 1600. The CPU 1100 also outputs generated data to the output devices via the input / output interface 1600.

[0074] Media interface 1700 reads a program or data stored in recording medium 1800 and provides it to CPU 1100 via RAM 1200. CPU 1100 loads the program or data from recording medium 1800 onto RAM 1200 via media interface 1700 and executes the loaded program. Recording medium 1800 is, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0075] For example, when the computer 1000 functions as the information processing device 100 according to the embodiment, the CPU 1100 of the computer 1000 executes programs loaded onto the RAM 1200 to realize the functions of the control unit 130. The CPU 1100 of the computer 1000 reads and executes these programs from the recording medium 1800, but as another example, the CPU 1100 may obtain these programs from another device via a predetermined communication network.

[0076] Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be implemented in other forms that include the embodiments described in the Disclosure of the Invention section and that have undergone various modifications and improvements based on the knowledge of those skilled in the art.

[0077] [7. Other] Furthermore, among the processes described in the above embodiments and modifications, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the information including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown in the drawings.

[0078] Furthermore, the components of each device shown in the figure are conceptual functional components and do not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0079] Furthermore, the above-described embodiments and modifications can be combined as appropriate within the scope of not causing any contradiction in the processing content. [Explanation of symbols]

[0080] 100 Information processing device 110 Communications Department 120 Storage section 130 Control Unit 131 Acquisition Department 132 Calculation Unit 133 Model Generation Unit 134 Sentence generation section< / eos> < / bos> < / eos> < / bos>

Claims

1. an acquisition unit that acquires a target caption, which is a sentence that explains the content of an input target image, for the input target image using a machine learning model; a calculation unit that calculates an evaluation value that evaluates the accuracy of the target caption and a sentence length value that indicates the length of the target caption; a model generation unit that generates the machine learning model that has undergone reinforcement learning by setting a reward value based on the evaluation value and the sentence length value; An information processing device comprising:

2. The acquisition unit obtaining a search caption, the target caption generated by the search method; The calculation unit Calculating a search evaluation value, which is the evaluation value of the searched caption, and a search sentence length value, which is the sentence length value of the searched caption; The model generation unit generating the machine learning model that has undergone reinforcement learning by setting the reward value based on the search evaluation value and the search sentence length value; The information processing device according to claim 1 .

3. The calculation unit calculating the evaluation value based on the similarity between the correct caption and the target caption; The information processing device according to claim 1 .

4. The calculation unit calculating the evaluation value, which is a Consensus-based Image Description Evaluation (CIDEr) score of the target caption; The information processing device according to claim 1 .

5. The calculation unit calculating the sentence length value based on the length of a sequence of tokens included in the target caption; The information processing device according to claim 1 .

6. The image processing system further includes a text generation unit that generates, from a processing target image, an image caption corresponding to the processing target image, using the machine learning model generated by the model generation unit. The information processing device according to claim 1 .

7. an acquisition step of acquiring a target caption, which is a sentence explaining the content of an input target image, for the input target image using a machine learning model; a calculation step of calculating an evaluation value for evaluating the accuracy of the target caption and a sentence length value indicating the length of the target caption; a model generation step of generating the machine learning model that has undergone reinforcement learning by setting a reward value based on the evaluation value and the sentence length value; An information processing program that causes a computer to execute the above.

Images