Information processing system, information processing method, and program

Abstract

The present invention uses an image generation model to generate a second image in which the recognition accuracy of an object included in a first image is improved. An object included in the second image is subjected to recognition processing. Information obtained by the recognition processing of the object included in the second image is input to train the image generation model.

Description

Information processing system, information processing method, and program

[0001] The present disclosure relates to an information processing system, an information processing method, and a program.

[0002] Techniques that use learning models to recognize objects contained in images are widely used, and in this technical field, there is a demand for improved accuracy in recognizing objects contained in images.

[0003] Patent Literature 1 discloses a technique for evaluating a recognition unit that recognizes an object in an input image. The technique disclosed in Patent Literature 1 discloses generating an evaluation unit that evaluates the recognition unit based on the recognition result of the recognition unit for the input image and the recognition result for a low-quality image.

[0004] International Publication No. 2021 / 241260

[0005] In the technology for improving the accuracy of recognizing objects contained in images, there is a demand for technology that generates images that can recognize objects with higher accuracy in the recognition process of objects contained in input images.

[0006] An object of this disclosure is to provide an information processing system, an information processing method, and a program that solve the above-mentioned problems.

[0007] According to a first aspect of this disclosure, an information processing system includes an image generation means that uses an image generation model to generate a second image that improves the recognition accuracy of an object included in a first image, and a recognition means that performs recognition processing on the object included in the second image, and the image generation means inputs information obtained in the recognition processing of the object included in the second image to train the image generation model.

[0008] According to a second aspect of this disclosure, an information processing method generates a second image using an image generation model that improves the recognition accuracy of an object included in a first image, performs a recognition process on the object included in the second image, and inputs information obtained from the recognition process on the object included in the second image to train the image generation model.

[0009] According to a third aspect of this disclosure, the program causes a computer of an information processing system to function as an image generation means that uses an image generation model to generate a second image that improves the recognition accuracy of an object included in a first image, and a recognition means that performs recognition processing on the object included in the second image, and the image generation means inputs information obtained in the recognition processing of the object included in the second image to learn the image generation model.

[0010] FIG. 1 is a first diagram showing the configuration of an information processing system according to an embodiment of the present disclosure. FIG. 2 is a diagram showing the processing flow of an information processing system according to an embodiment of the present disclosure. FIG. 3 is a second diagram showing the configuration of an information processing system according to an embodiment of the present disclosure. FIG. 4 is a fourth diagram showing the public operation of an information processing system according to an embodiment of the present disclosure. FIG. 5 is a hardware configuration diagram of an information processing system according to an embodiment of the present disclosure. FIG. 6 is a diagram showing the configuration of an information processing system according to another embodiment of the present disclosure. FIG. 7 is a diagram showing the processing flow of an information processing system according to another embodiment of the present disclosure.

[0011] First Embodiment FIG. 1 is a first diagram showing the configuration of an information processing system according to an embodiment of the present disclosure.

[0012] The information processing system 100 executes, for example, an image processing program, thereby enabling the information processing system 100 to perform the functions of an image processing unit 11, an image generation unit 12, a recognition unit 13, and a loss processing unit 14.

[0013] The image processing unit 11 generates a first image 52 and a third image 54 using the original image 51. The first image 52 may be called a dark image, and the third image 54 may be called a bright image. In other words, the third image 54 is an image with a higher brightness than the first image 52. Note that the brightness of the third image 54 is higher than that of the first image 52 when the brightness value of the entire image of the third image 54 is higher than that of the first image 52, when the brightness value of a predetermined region (eye region or iris region) of the third image 54 is higher than that of the first image 52, or when the brightness value of a predetermined pixel of the third image 54 is higher than that of the first image 52. Alternatively, the first image 52 may be defined as an image with a lower brightness than that of the third image 54. In this case, the brightness of the first image 52 is lower than that of the third image 54 when the brightness value of the entire image of the first image 52 is lower than that of the third image 54, when the brightness value of a predetermined region (eye region or iris region) of the first image 52 is lower than that of the third image 54, or when the brightness value of a predetermined pixel of the first image 52 is lower than that of the third image 54.

[0014] The image generation unit 12 generates the second image 53 using the first image 52. Specifically, the image generation unit 12 generates the second image 53 using the image generation model, which improves the recognition accuracy of the object by the recognition model 31 that recognizes the object included in the first image 52.

[0015] The recognition unit 13 uses the recognition model 31 to perform recognition processing on an object included in the second image 53 or the third image 54. Here, the recognition unit 13 performs recognition processing by inputting the second image 53, which is an image generated by the image generation unit 12 and which improves the recognition accuracy of the object by the recognition model 31, and therefore can perform recognition processing with high accuracy on the object included in the second image 53. The recognition model 31 outputs a score that is the result of the recognition processing.

[0016] The recognition unit 13 performs the functions of a first similarity loss calculation unit 32 and a second similarity loss calculation unit 33. The first similarity loss calculation unit 32 outputs a value of a similarity index (image similarity loss) between the pixel values ​​of each pixel of the first image 52 and the second image 53. The second similarity loss calculation unit 33 outputs a value of a similarity index (intermediate feature similarity loss) between the intermediate feature obtained in the recognition process of the first image (dark image) 52 and the intermediate feature obtained in the recognition process of the third image (bright image) 54.

[0017] The loss processing unit 14 specifies a loss value using one or more of the score calculated for the second image 53 among the scores output by the recognition unit 13, the intermediate feature similarity loss value between the second image 53 and the third image 54, and the image similarity loss value between the second image 53 and the third image 54, and outputs the loss value to the image generation unit 12. The image generation unit 12 uses the loss value to retrain (optimize) the image generation model used by the image generation unit 12 to generate the second image 53. Note that retraining (optimizing) the image generation model means repeatedly learning by machine learning the relationship between the input and output so that the output for the image generation model is appropriate. In this embodiment, the relationship between the input and output for the image generation model is repeatedly learned by machine learning so that the recognition accuracy when the recognition model 31 recognizes an object included in the second image 53, which is the output for the input of the image generation model, is improved.

[0018] The information processing system 100 that exhibits such functions uses an image generation model to generate a second image 53 that improves the recognition accuracy of an object included in the image using a recognition model that recognizes the object, uses the recognition model to perform recognition processing on the object included in the second image 53, and re-trains the image generation model by inputting information obtained in the recognition processing on the object included in the second image 53. This type of processing makes it possible to provide an information processing system 100 that can generate an image that can recognize the object with higher accuracy in the recognition processing on the object included in the input image.

[0019] FIG. 2 is a diagram showing a processing flow of an information processing system according to an embodiment of the present disclosure.

[0020] Next, the image processing flow will be explained step by step.

[0021] First, the image processing unit 11 inputs an original image 51 (step S101). The original image 51 may be an image input from a photographing device connected to the information processing system 100 for communication, or may be an image input to the information processing system 100 by a user. The photographing device may generate still images or moving images. When a moving image is input from outside, the information processing system 100 may acquire one of the images constituting the moving image as the original image 51.

[0022] The image processing unit 11 generates a first image (dark image) 52 and a third image 54 from the original image 51 (step S102). Specifically, the image processing unit 11 splits the signal of the original image 51 into two using a beam splitter and passes one of the signals through a neutral density filter (ND filter) 111 to generate the first image (dark image) 52. This processing by the image processing unit 11 enables the generation of the first image 52 and the third image 54 with a simple configuration. The beam splitter is a device that splits the input optical signal of the original image 51 into two optical signals and outputs them. The first image 52 may be an image in which the brightness values ​​of pixels in some areas are lower than a predetermined threshold. The image processing unit 11 outputs the other signal as is to the recognition unit 13 as the third image 54. The processing by the image processing unit 11 is an example of a process in which the original image 51 is passed through a neutral density filter to generate the first image 52 and the original image 51 is used as the third image 54. The first image 52 does not have to be a dark image. It is sufficient that the luminance of each pixel of the third image 54 is higher than the luminance of each corresponding pixel of the first image 52. In addition to the case where the luminance value of the entire image of the third image 54 is higher than that of the first image 52, there may be any of the cases where the luminance value of a predetermined region (eye region or iris region) of the third image 54 is higher than the luminance value of each pixel of the corresponding region of the first image 52, or where the luminance value of predetermined pixels (e.g., more than half of the pixels) of the third image 54 is higher than the luminance value of the corresponding pixel of the first image 52.

[0023] As described above, the first image 52 may be defined as an image having a lower brightness than the third image 54. In this case, the brightness of the first image 52 may be lower than the brightness of the third image 54 if the brightness value of the entire first image 52 is lower than the brightness value of the third image 54, if the brightness value of a predetermined region (eye region or iris region) of the first image 52 is lower than the brightness value of the corresponding region of the third image 54, or if the brightness value of a predetermined pixel of the first image 52 is lower than the brightness value of the corresponding pixel of the third image 54.

[0024] The image generation unit 12 inputs a first image (dark image) 52. The image generation unit 12 inputs the first image 52 into an image generation model to generate a second image 53, and as a result, outputs the second image 53 (step S103). The image generation model used by the image generation unit 12 to generate the second image 53 is a learning model that has been trained (optimized) to improve the recognition accuracy of the recognition model 31 of the recognition unit 13 for the object in the image. Therefore, when the recognition unit 13 performs recognition processing of the object included in the second image 53 using the recognition model 31, the object recognition accuracy can be improved compared to when the first image 52 is used. Here, it is desirable that the second image 53 be an image that makes it easy for the recognition model 31 to recognize the object, even if the first image 52 is a dark image, and that makes it easy for a person to visually recognize the object in the second image 53. The image generation model may be, for example, a convolution process, an image processing algorithm including multiple activation functions and convolution processes, a UNet or a diffusion model, a VAE (Variational Autoencoder), a GAN (Generative Adversarial Network), a mapping process for pixel luminance values, etc. The mapping process for image luminance values ​​is a process of emphasizing specific luminance by applying a processing function, generally called a tone curve, to image luminance values, which is a function that represents the correspondence of correction values ​​to image luminance values.

[0025] The recognition unit 13 acquires the second image 53 output by the image generation unit 12 and inputs it to the recognition model 31. The recognition model 31 inputs the first image 52 and outputs a score loss 55 (score loss of the second image 53) indicating the degree to which the object in the image was recognized in the recognition process (step S104). The score loss 55 is, for example, the magnitude of deviation (loss) between the predicted value output by the recognition model 31 and the correct value.

[0026] The recognition unit 13 also receives a third image (bright image) 54 from the image processing unit 11. The first similarity loss calculation unit 32 of the recognition unit 13 calculates the value of the similarity index (image similarity loss 56) between the input second image 53 and third image 54 (step S105). The image similarity loss 56 is the loss of similarity between the two images, the second image 53 and the third image 54, and is calculated based on image similarity calculated using SSIM (structural similarity), PSNR (Peak signal-to-noise ratio), L2 Loss, L1 Loss, KL / JS (Kullback-Leibler / Jensen-Shannon) Divergence, VGG Loss, Adversarial Loss, etc. Known techniques may be used to calculate the loss of similarity between the two images indicated by the image similarity loss 56. The first similarity loss calculation unit 32 outputs the image similarity loss 56 to the loss processing unit 14 .

[0027] Furthermore, in the process of calculating the score loss 55 using the second image 53 described above, the recognition unit 13 outputs the intermediate feature calculated by the recognition model 31 to the second similarity loss calculation unit 33. The recognition unit 13 also acquires the second image 53, inputs it to the recognition model 31, and outputs the intermediate feature calculated by the recognition model 31 to the second similarity loss calculation unit 33. The second similarity loss calculation unit 33 calculates a similarity loss (intermediate feature similarity loss 57) between the intermediate feature calculated in the recognition process of the second image 53 and the intermediate feature calculated in the recognition process of the third image 54 (step S106). The intermediate feature similarity loss 57 may be an L1 loss or an L2 loss between the intermediate feature calculated in the recognition process of the second image 53 and the intermediate feature calculated in the recognition process of the third image 54. The intermediate feature similarity loss 57 may be calculated by using another known technique to calculate the similarity loss between the intermediate feature calculated in the recognition process of the second image 53 and the intermediate feature calculated in the recognition process of the third image 54. The second similarity loss calculation unit 33 outputs the intermediate feature similarity loss 57 to the loss processing unit 14.

[0028] The loss processing unit 14 acquires three losses: a score loss 55, an image similarity loss 56, and an intermediate feature similarity loss 57. The loss processing unit 14 uses one or more of the score loss 55, the image similarity loss 56, and the intermediate feature similarity loss 57 to identify a loss value to be fed back to the image generation unit 12 and outputs the identified loss value to the image generation unit 12 (step S107). The image generation unit 12 optimizes the image generation model to minimize the loss using the loss value fed back from the loss processing unit 14 using a backpropagation method or the like (step S108). In optimization training, the image generation unit 12 performs backpropagation from any of the acquired losses to perform optimization by updating the parameters of the image generation model. At this time, the image generation unit 12 may perform optimization using a method for updating the parameters of the image generation model based on gradient information obtained from the loss, such as stochastic gradient descent (SGD) or Adam.

[0029] The information processing system 100 may repeat the processes from step S101 to step S108 to optimize the image generation model.

[0030] FIG. 3 is a second diagram showing the configuration of an information processing system according to an embodiment of the present disclosure.

[0031] The information processing system shown in Figure 3 illustrates an example of recognition processing after the image generation model of the image generation unit 12 has been optimized by the processing shown in Figure 1. By using this optimized image generation model, the image generation unit 12 trains the image generation model to minimize the loss indicated by the score, which is the result of the recognition processing of the recognition model. This allows the image generation unit 12 to generate, from the first image 52, a second image 53 that allows the recognition model 31 to improve the object recognition accuracy. Furthermore, the image generation unit 12 trains the image generation model to minimize the similarity loss between the second image and the first image and the similarity loss between intermediate features. This allows the recognition model 31 to improve the object recognition accuracy and generates a second image 53 that is easy for people to recognize the object. Generating a second image 53 that is easy for people to recognize the object allows people to easily check for abnormalities in the object in the image and to check other image conditions.

[0032] Second Embodiment The loss processing unit 14 outputs the score loss 55 to the image generation unit 12. The image generation unit 12 uses the score loss 55 to optimize the image generation model to minimize the loss. In this case, in the example of FIG. 3 , the loss processing unit 14 outputs the score loss 55, which is the recognition result of the recognition model 31, to the image generation unit 12. In other words, the information processing system 100 may be configured so that the third image 54 is not input to the recognition unit 13. The optimization of the image generation model by the image generation unit 12 is the same as in the first embodiment. As a result, the image generation unit 12 can generate a second image 53 that allows the recognition model 31 to improve the recognition accuracy of the target.

[0033] Third Embodiment The loss processing unit 14 outputs the image similarity loss 56 to the image generation unit 12. The image generation unit 12 uses the image similarity loss 56 to optimize the image generation model to minimize the loss. This allows the image generation unit 12 to generate a second image 53 that allows the recognition model 31 to improve the object recognition accuracy and that makes it easy for people to recognize the object.

[0034] Fourth Embodiment The loss processing unit 14 outputs the intermediate feature similarity loss 57 to the image generation unit 12. The image generation unit 12 uses the intermediate feature similarity loss 57 to optimize the image generation model to minimize the loss. This allows the image generation unit 12 to generate a second image 53 that enables the recognition model 31 to improve the recognition accuracy of the target.

[0035] Fifth Embodiment The loss processing unit 14 outputs a first integrated loss calculated using the score loss 55 and the image similarity loss 56 to the image generation unit 12. The image generation unit 12 uses the first integrated loss to optimize the image generation model to minimize the loss. The first integrated loss may be a value obtained by multiplying the score loss 55 and the image similarity loss 56 by weights and then adding them together. This allows the image generation unit 12 to generate a second image 53 that allows the recognition model 31 to improve the object recognition accuracy and that makes it easy for people to recognize the object.

[0036] Sixth Embodiment The loss processing unit 14 outputs a second integrated loss calculated using the score loss 55 and the intermediate feature similarity loss 57 to the image generation unit 12. The image generation unit 12 uses the second integrated loss to optimize the image generation model to minimize the loss. The second integrated loss may be a value obtained by multiplying the score loss 55 and the intermediate feature similarity loss 57 by weights and adding them together. This allows the image generation unit 12 to generate a second image 53 that enables the recognition model 31 to improve the recognition accuracy of the target.

[0037] Seventh Embodiment The loss processing unit 14 outputs a third integrated loss calculated using the score loss 55, the image similarity loss 56, and the intermediate feature similarity loss 57 to the image generation unit 12. The image generation unit 12 uses the third integrated loss to optimize the image generation model to minimize the loss. The third integrated loss may be a value obtained by multiplying the score loss 55, the image similarity loss 56, and the intermediate feature similarity loss 57 by weights and then adding them together. This allows the image generation unit 12 to generate a second image 53 that allows the recognition model 31 to improve the object recognition accuracy and that makes it easy for people to recognize the object.

[0038] Eighth Embodiment The loss processing unit 14 outputs a fourth integrated loss calculated using the image similarity loss 56 and the intermediate feature similarity loss 57 to the image generation unit 12. The image generation unit 12 uses the fourth integrated loss to optimize the image generation model to minimize the loss. The fourth integrated loss may be a value obtained by multiplying the image similarity loss 56 and the intermediate feature similarity loss 57 by weights and then adding them together. This allows the image generation unit 12 to generate a second image 53 that allows the recognition model 31 to improve the object recognition accuracy and that makes it easy for people to recognize the object.

[0039] Ninth Embodiment FIG. 4 is a third diagram showing the configuration of an information processing system according to an embodiment of the present disclosure.

[0040] In the information processing system shown in FIG. 4, the processing of the image processing unit 11 that generates a first image 52 and a third image 54 based on an original image 51 is different from the processing described in FIG.

[0041] 4, the image processing unit 11 identifies the original image as a first image 52, and performs luminance correction on the first image 52 to generate a third image 54 in which the luminance values ​​of each pixel are higher than those of the first image 52. This process is an example of a process in which the original image is luminance-corrected to generate a third image 54 with increased luminance, and the original image is used as the first image 52. According to such processing by the image processing unit 11, the first image 52 and the third image 54 can be generated with another simple configuration.

[0042] Tenth Embodiment FIG. 5 is a fourth diagram showing the configuration of an information processing system according to an embodiment of the present disclosure.

[0043] In the information processing system shown in FIG. 5, the processing of the image processing unit 11 that generates a first image 52 and a third image 54 based on an original image 51 is different from the processing described with reference to FIGS.

[0044] 5, the image processing unit 11 may separate the original image 51 into a third image 54 composed of pixels with pixel values ​​equal to or greater than a threshold, and a first image 52 composed of pixels with pixel values ​​less than the threshold. This process is an example of a process using the original image and a predetermined threshold to generate a first image 52 composed of pixels with pixel values ​​less than the threshold and a third image 54 composed of pixels with pixel values ​​equal to or greater than the threshold. According to such processing by the image processing unit 11, the first image 52 and the third image 54 can be generated with another simple configuration.

[0045] FIG. 6 is a hardware configuration diagram of an information processing system according to an embodiment of the present disclosure.

[0046] 6, the information processing system 100 is a computer including various pieces of hardware such as a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, other storage devices 104, and a communication module 105. The information processing system 100 may perform each of the above functions using a single computer, or may perform each of the above functions using a plurality of computers.

[0047] Eleventh Embodiment FIG. 7 is a diagram showing another aspect of an information processing system.

[0048] FIG. 8 is a diagram showing a processing flow according to another aspect of the information processing system.

[0049] The information processing system 100 includes an image generating unit 71 and a recognizing unit 72 .

[0050] The image generation means 71 generates the second image 53 using an image generation model, which improves the recognition accuracy of the object included in the first image 52 (step S801).

[0051] The recognition means 72 performs a recognition process on the object included in the second image 53 (step S802).

[0052] The image generating means 71 inputs information obtained by the recognition process of the object included in the second image 53 and trains (optimizes) the image generation model (step S803).

[0053] Although the information processing system 100 of this disclosure has been described above, this disclosure is not limited to the above-described embodiment. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of this disclosure within the scope of this disclosure.

[0054] Note that part or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0055] (Supplementary Note 1) An information processing system comprising: an image generation means that uses an image generation model to generate a second image that improves the recognition accuracy of an object included in a first image; and a recognition means that performs recognition processing on the object included in the second image, wherein the image generation means inputs information obtained in the recognition processing of the object included in the second image to train the image generation model.

[0056] (Supplementary Note 2) The information processing system described in Supplementary Note 1, wherein the image generation means generates the second image using the image generation model, which improves the recognition accuracy of the object included in the first image by the recognition model that recognizes the object; the recognition means recognizes the object included in the second image using the recognition model; and the image generation means inputs information obtained in the recognition process of the object included in the second image to train the image generation model.

[0057] (Supplementary Note 3) The information processing system according to Supplementary Note 2, wherein the image generation means inputs a recognition score, which is information obtained in the recognition process of the object included in the second image, to train the image generation model.

[0058] (Supplementary Note 4) The information processing system according to Supplementary Note 2 or Supplementary Note 3, wherein the image generation means inputs a similarity index between intermediate features obtained by the recognition process of the first image, which is information obtained by the recognition process, and intermediate features obtained by the recognition process of a third image whose original image matches the first image and whose brightness is higher than that of the first image, to train the image generation model.

[0059] (Supplementary Note 5) The information processing system according to Supplementary Note 4, wherein the image generation means inputs a similarity index between the first image and a third image whose original image matches the first image and is brighter than the first image, the similarity index being information obtained by the recognition process, to learn the image generation model.

[0060] (Supplementary Note 6) The information processing system according to Supplementary Note 5, wherein the first image is generated by passing an original image through a neutral density filter, and the original image is used as the third image.

[0061] (Supplementary Note 7) The information processing system according to Supplementary Note 5, wherein the third image is generated by performing luminance correction on an original image to increase luminance, and the original image is used as the first image.

[0062] (Appendix 8) An information processing system as described in Appendix 5, which uses an original image and a predetermined threshold to generate the first image composed of pixels with pixel values ​​less than the threshold, and the third image composed of pixels with pixel values ​​equal to or greater than the threshold.

[0063] (Supplementary Note 9) An information processing method comprising: generating a second image using an image generation model, which improves the recognition accuracy of an object included in a first image; performing a recognition process on the object included in the second image; and inputting information obtained from the recognition process on the object included in the second image to train the image generation model.

[0064] (Supplementary Note 10) An information processing method according to Supplementary Note 9, comprising: generating the second image using the image generation model, which improves the recognition accuracy of the object included in the first image by a recognition model that recognizes the object; recognizing and processing the object included in the second image using the recognition model; and inputting information obtained in the recognition processing of the object included in the second image to train the image generation model.

[0065] (Supplementary Note 11) The information processing method according to Supplementary Note 10, wherein the image generation model is trained by inputting a recognition score, which is information obtained in the recognition process of the object included in the second image.

[0066] (Supplementary Note 12) The information processing method according to Supplementary Note 10 or Supplementary Note 11, wherein the image generation model is trained by inputting a similarity index between intermediate features obtained by the recognition process of the first image, which is information obtained by the recognition process, and intermediate features obtained by the recognition process of a third image whose original image matches the first image and whose brightness is higher than that of the first image.

[0067] (Supplementary Note 13) The information processing method according to Supplementary Note 12, wherein the image generation model is trained by inputting a similarity index between the first image and a third image whose original image matches the first image and whose brightness is higher than that of the first image, the similarity index being information obtained by the recognition process.

[0068] (Supplementary Note 14) The information processing method according to Supplementary Note 13, wherein the first image is generated by passing an original image through a neutral density filter, and the original image is used as the third image.

[0069] (Supplementary Note 15) The information processing method according to Supplementary Note 13, wherein the third image is generated by performing luminance correction on an original image to increase luminance, and the original image is used as the first image.

[0070] (Supplementary Note 16) The information processing method according to Supplementary Note 13, which uses an original image and a predetermined threshold to generate the first image composed of pixels with pixel values ​​less than the threshold, and the third image composed of pixels with pixel values ​​equal to or greater than the threshold.

[0071] (Supplementary Note 17) A program that causes a computer of an information processing system to function as: an image generation means that uses an image generation model to generate a second image that improves the recognition accuracy of an object included in a first image; and a recognition means that performs recognition processing on the object included in the second image, wherein the image generation means inputs information obtained in the recognition processing of the object included in the second image to learn the image generation model.

[0072] (Supplementary Note 18) The information processing system described in Supplementary Note 17, wherein the image generation means generates the second image using the image generation model, which improves the recognition accuracy of the object included in the first image by the recognition model that recognizes the object; the recognition means recognizes the object included in the second image using the recognition model; and the image generation means inputs information obtained in the recognition processing of the object included in the second image to train the image generation model.

[0073] (Supplementary Note 19) The information processing system according to Supplementary Note 18, wherein the image generation means inputs a recognition score, which is information obtained in the recognition process of the object included in the second image, to train the image generation model.

[0074] (Supplementary Note 20) The information processing system according to Supplementary Note 18 or Supplementary Note 19, wherein the image generation means inputs a similarity index between intermediate features obtained by the recognition process of the first image, which is information obtained by the recognition process, and intermediate features obtained by the recognition process of a third image whose original image matches the first image and whose brightness is higher than that of the first image, to train the image generation model.

[0075] (Supplementary Note 21) The information processing system according to Supplementary Note 20, wherein the image generation means inputs a similarity index between the first image and a third image, which is information obtained by the recognition process and has a higher brightness than the first image and whose original image matches the first image, to learn the image generation model.

[0076] (Supplementary Note 22) The information processing system according to Supplementary Note 21, wherein the first image is generated by passing an original image through a neutral density filter, and the original image is used as the third image.

[0077] (Supplementary Note 23) The information processing system according to Supplementary Note 21, wherein the third image is generated by performing luminance correction on an original image to increase luminance, and the original image is used as the first image.

[0078] (Appendix 24) The information processing system described in Appendix 21 uses an original image and a predetermined threshold to generate the first image composed of pixels with pixel values ​​less than the threshold, and the third image composed of pixels with pixel values ​​equal to or greater than the threshold.

[0079] (Supplementary Note 25) A storage medium that stores the program according to Supplementary Note 17.

[0080] REFERENCE SIGNS LIST 11: Image processing unit 12: Image generation unit (image generation means) 13: Recognition unit (recognition means) 14: Loss processing unit 31: Recognition model 32: First similarity loss calculation unit 33: Second similarity loss calculation unit

Claims

1. An information processing system comprising: an image generation means for generating a second image using an image generation model to improve the recognition accuracy of an object included in a first image; and a recognition means for performing recognition processing of the object included in the second image, wherein the image generation means inputs information obtained by the recognition processing of the object included in the second image to train the image generation model.

2. An information processing system, wherein the image generation means uses the image generation model to generate the second image in which the recognition accuracy of the object included in the first image by the recognition model is improved, the recognition means uses the recognition model to recognize the object included in the second image, and the image generation means inputs information obtained by the recognition processing of the object included in the second image to train the image generation model.

3. The information processing system according to claim 2, wherein the image generation means inputs a recognition score, which is information obtained in the recognition process of the object contained in the second image, to learn the image generation model.

4. An information processing system as described in claim 2 or claim 3, wherein the image generation means learns the image generation model by inputting a similarity index between intermediate features obtained in the recognition processing of the first image, which is information obtained in the recognition processing, and intermediate features obtained in the recognition processing of a third image, which has a higher brightness than the first image and whose original image matches the first image.

5. The information processing system according to claim 4, wherein the image generation means inputs similarity indices between the first image and a third image, which is information obtained by the recognition process and has a higher brightness than the first image and whose original image matches the first image, to learn the image generation model.

6. An information processing system according to claim 5, wherein the first image is generated by passing an original image through a neutral density filter, and the original image is used as the third image.

7. An information processing system according to claim 5, further comprising: generating the third image by performing luminance correction on an original image to enhance luminance; and using the original image as the first image.

8. An information processing system as described in claim 5, which uses an original image and a predetermined threshold to generate a first image composed of pixels with pixel values ​​less than the threshold, and a third image composed of pixels with pixel values ​​equal to or greater than the threshold.

9. An information processing method comprising: generating a second image using an image generation model in which the recognition accuracy of an object included in a first image is improved; recognizing and processing the object included in the second image; and inputting information obtained from the recognition processing of the object included in the second image to train the image generation model.

10. A program that causes a computer of an information processing system to function as an image generation means that uses an image generation model to generate a second image in which the recognition accuracy of an object included in a first image is improved, and a recognition means that recognizes and processes the object included in the second image, wherein the image generation means learns the image generation model by inputting information obtained by the recognition processing of the object included in the second image.

Images