Information processing device, information processing method, and control program

Abstract

To improve the reproducibility of a simulation image while reducing the burden on a user. [Solution] An information processing device (100) that performs adversarial imitation learning using a generator and a classifier, comprising: an acquisition unit (11) that acquires a reference image captured of an area including an object; a generation unit (12) that performs processing as a generator and generates a generated image according to a policy model that is updated according to a reward function; an extraction unit (14) that inputs the reference image to an autoencoder to extract latent variables corresponding to the reference image and inputs the generated image to the autoencoder to extract latent variables corresponding to the generated image; a calculation unit (15) that inputs each extracted latent variable to a classifier that identifies whether the latent variable corresponds to the reference image or the generated image, and updates the reward function according to the output result of the classifier; and a learning unit (17) that updates the policy model according to the updated reward function.

Description

[Technical Field]

[0001] One aspect of the present invention relates to an information processing device, an information processing method, and a control program. [Background technology]

[0002] Conventionally, there is known a technique for generating a simulated image of a school of living organisms or the like using computer graphics. Patent Document 1 discloses an information processing method for generating a simulated image of, for example, an aquaculture farm and estimating the number of fish in the school of fish based on parameter information used in the simulation. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-55140 Summary of the Invention [Problem to be solved by the invention]

[0004] In the conventional techniques described above, for example, when generating a simulated image of an aquaculture farm, the user must directly or selectively specify parameters such as the swimming style of fish and the spacing between fish depending on the fish species, etc. The images that can be reproduced by this method are limited by the specified parameters, and the workload required for the user to specify the parameters is large, leaving room for improvement.

[0005] One aspect of the present invention has been made in consideration of the above-mentioned problems, and aims to improve the reproducibility of simulation images while reducing the burden on the user. [Means for solving the problem]

[0006] In order to solve the above-described problems, an information processing device according to one aspect of the present invention is an information processing device that performs adversarial imitation learning using a generator and a classifier, and generates a generated image of an object using computer graphics, and includes: an acquisition unit that acquires a reference image obtained by capturing an area including the object; a generation unit that performs processing as the generator, and generates the generated image according to a policy model that is updated according to a reward function; an extraction unit that inputs the reference image to an autoencoder to extract latent variables corresponding to the reference image and inputs the generated image to the autoencoder to extract latent variables corresponding to the generated image; a calculation unit that inputs each extracted latent variable to the classifier that identifies whether the latent variable corresponds to the reference image or the generated image, and updates the reward function according to an output result of the classifier; and a learning unit that updates the policy model according to the updated reward function.

[0007] In order to solve the above-mentioned problems, an information processing method according to one aspect of the present invention is an information processing method executed by an information processing device that performs adversarial imitation learning using a generator and a classifier and generates a generated image of an object using computer graphics, the information processing method including: an acquisition step of acquiring a reference image of an area including the object; a generation step of performing processing as the generator, in which the generated image is generated according to a policy model that is updated according to a reward function; an extraction step of inputting the reference image to an autoencoder to extract latent variables corresponding to the reference image and inputting the generated image to the autoencoder to extract latent variables corresponding to the generated image; a calculation step of inputting each extracted latent variable to the classifier that identifies whether the latent variable corresponds to the reference image or the generated image, and updating the reward function according to an output result of the classifier; and a learning step of updating the policy model according to the updated reward function.

[0008] The information processing device according to each aspect of the present invention may be realized by a computer. In this case, the control program of the information processing device that causes the computer to operate as each part (software element) of the information processing device to realize the information processing device on the computer, and the computer-readable recording medium on which the control program is recorded, also fall within the scope of the present invention.

[0009] The control program may use various machine learning techniques in the process of causing a computer to operate as each part of the control block or in other processes. In this case, the program using the machine learning technique may run on the information processing device or on another device (for example, an edge computer or a cloud server). [Effects of the Invention]

[0010] According to one aspect of the present invention, it is possible to improve the reproducibility of a simulation image while reducing the burden on the user. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing device. [Figure 2] 1 is an example of a reference image and a generated image. [Figure 3] FIG. 1 is an example of a conceptual diagram illustrating the structure and processing of a masked video autoencoder. [Figure 4] 10 is an example of a flowchart showing a processing flow of an information processing method executed by an information processing device. [Figure 5] FIG. 1 is an example of a diagram for explaining an example of processing performed by an information processing device. [Figure 6] FIG. 1 is an example of a diagram for explaining an example of processing performed by an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, one embodiment of the present invention will be described in detail.

[0013] [1. Configuration example of information processing device] FIG. 1 is a diagram illustrating an example of a configuration of an information processing device 100 according to the present disclosure.

[0014] The information processing device 100 is a device that generates images and is capable of generating images that are more similar to a reference image to be imitated by updating an image generation policy (policy model) using Adversarial Imitation Learning (or Generative Adversarial Imitation Learning (GAIL)). In this embodiment, unless otherwise specified, the image is a relatively short video clip (moving image) containing an object. Furthermore, when specifying each still image contained in this video clip, the still image may be referred to as a "frame." In the following description, the object is mainly a fish and the image contains multiple fishes, but the object is not limited to this and may be a bird, animal, person, etc.

[0015] Adversarial imitation learning is a type of machine learning that uses a generator and a discriminator that compete with each other. The generator imitates a reference image, such as a photographed image, to generate an image that cannot be detected by the discriminator, and the discriminator performs processing to distinguish whether an input image is the real reference image. The discriminator can also evaluate how similar the image generated by the generator (simulated image) is to the reference image.

[0016] The policy can also be considered a function that defines the behavior of each object in a given state during image generation. The behavior also includes the shape and state of the object. The information processing device 100 improves reproducibility when generating an image of a group of objects by adjusting the policy. The policy is updated according to a reward function that defines the reward obtained for each action of the object, so that the output value of the reward function increases. In other words, the information processing device 100 can be considered to perform inverse reinforcement learning, estimating a reward function that makes the behavior of the object based on the image generation policy similar to the known behavior of the object contained in the reference image.

[0017] The information processing device 100 includes a control unit 1, a storage unit 2, an input unit 3, and a display unit (output unit) 4.

[0018] The control unit 1 is a control device such as a CPU or GPU that controls the entire information processing device 100, and also operates as an acquisition unit 11, a generation unit 12, a processing unit 13, an extraction unit 14, a calculation unit 15, a classification unit 16, and a learning unit 17.

[0019] The acquisition unit 11 acquires a reference image that is an actual image of an area including an object from the storage unit 2 or an external device (not shown). For example, the acquisition unit 11 acquires a reference image that is an image of a fish farm that includes a plurality of fish. Each image included in the image group 21 in FIG. 2 is an example of a reference image. Each image shows a school of fish performing an action such as "circling."

[0020] The generation unit 12 is a component that performs processing as a generator, and generates generated images using computer graphics in accordance with the image generation policy stored in the storage unit 2. For example, the generation unit 12 generates generated images including multiple fish. Each image included in the image set 22 in FIG. 2 is an example of a generated image.

[0021] The processing unit 13 performs a segmentation process and a masking process, which will be described later, as predetermined processing on the input image.

[0022] The extraction unit 14 extracts latent variables corresponding to an image by inputting the image to a masked video autoencoder (MVAE) that has been sufficiently trained in advance by self-supervised learning. The latent variables are a type of feature.

[0023] Here, a masked video autoencoder is an autoencoder that inputs an image in which some regions are masked and hidden to the encoder, and outputs a decoded image in which the masked regions are predicted. The masked video autoencoder reconstructs a decoded image so as to minimize the output of a loss function including, for example, reconstruction loss and KL divergence loss. Note that the loss function is not limited to the above configuration as long as the value converges. For example, the Wasserstein norm may be used instead of the KL divergence loss, or a weighted sum of the reconstruction loss and the KL divergence loss may be used. Since the image input to the masked video autoencoder is a video clip containing multiple frames, prediction is performed so that the masked portions are consistent in the temporal direction. Furthermore, the masked video autoencoder reduces the dimensionality of the input data and maps it to a latent space as a result or during encoding, deriving latent variables that are not directly observed from the input data. Typically, a latent variable is a vector containing one or more elements.

[0024] Fig. 3 is a conceptual diagram showing an example of the structure and processing of a masked video autoencoder. In the example of Fig. 3, a reference image that has been subjected to segmentation processing and masking processing (described later) is input to an encoder using a Vision Transformer (ViT) or the like, and a decoded image, i.e., a reconstructed video clip O^ is output from a decoder. t The z in Figure 3 is output. t Or z in Figure 5 etc. described below t Allindicates an example of a latent variable obtained as a result or process of encoding by the encoder 53 and decoding by the decoder 54. The subscript t indicates a certain point in time or a time step. j denotes the mean of the latent variable, and σ j indicates the standard deviation of the latent variable. The subscript j is a number indicating the component of the dimension number J of the latent variable, and the value of the dimension number J is 100, for example.

[0025] FIG. 3 also illustrates the structure of a neural network 55 of a decoder included in the masked video autoencoder. In the neural network 55, a "Linear Layer" is a layer that performs linear transformation on input data and is also called a linear layer or a fully connected layer. A "ConvT3d (Convolutional Transpose 3D)" is a layer that performs processing such as changing the resolution of input data and is also called a deconvolution layer or a 3D convolution transpose layer. A "BatchNormal3D (Batch Normalization 3D)" is a layer that normalizes input data. A "ReLU (Rectified Linear Unit)" is a layer that applies an activation function that clips input pixel values. A "Sigmoid" is a layer that applies a sigmoid function that is particularly used in the output layer of a neural network.

[0026] Note that the configuration in which the information processing device 100 uses a masked video autoencoder is not essential, and the present disclosure also includes a configuration in which an autoencoder other than a masked video autoencoder is used.

[0027] The calculation unit 15 inputs the classification target to a classifier and calculates the output of the classifier. For example, the calculation unit 15 inputs latent variables corresponding to each of the reference image and the generated image to a classifier that identifies which of the reference image and the generated image the latent variable corresponds to, and calculates the output of the classifier. In addition, the calculation unit 15 calculates various rewards and a new reward function (described below) according to the output result of the classifier, and performs processing to update the reward function.

[0028] The classification unit 16 performs a clustering process to classify (cluster) the input data into a plurality of clusters. For example, the classification unit 16 classifies the latent variables extracted by the extraction unit 14 into a plurality of clusters. The classification unit 16 may perform the clustering process using the K-means method, or may determine the optimal number of clusters using the Elbow method.

[0029] The learning unit 17 performs a process of updating the image generation policy in accordance with the updated reward function. The learning unit 17 also performs a process of training the masked video autoencoder.

[0030] The control unit 1 also controls the screen display by the display unit 4.

[0031] The storage unit 2 is a storage device such as a memory that at least temporarily stores various information. For example, the storage unit 2 stores a reference image, an image generation policy, a reward function, and a parameter set that defines a masked video autoencoder and a classifier.

[0032] The input unit 3 is a member realized as a button, a keyboard, etc., and receives input instructions to the information processing device 100 from a user.

[0033] The display unit 4 has a display that displays a screen. The input unit 3 and the display unit 4 may be configured as an integrated touch panel.

[0034] The above describes an example of the configuration of the information processing device 100. Additionally, each unit included in the information processing device 100 has a function to execute the processes described below.

[0035] [2. Processing example of information processing device] Next, a description will be given of the processing flow of the information processing method executed by the information processing device 100. Fig. 4 is an example of a flowchart showing the processing flow. Figs. 5 and 6 are examples of diagrams for explaining processing examples of the information processing device 100. The processing shown in the flowchart in Fig. 4 is started when, for example, a predetermined input operation is performed by the user on the input unit 3.

[0036] In S1 (step S1), the acquisition unit 11 of the information processing device 100 acquires a reference image that is an actual image of a fish farm containing a plurality of fish. The reference image 41 in Fig. 5 is an example of this.

[0037] In S2, the generation unit 12 generates a generated image including a plurality of fishes based on the image generation policy. The generated image 31 in FIG. 5 is an example of this. t indicates the state of the target fish, and a t is, s t and the next action determined based on the policy. In other words, the policy t From a t It can also be expressed as a function π that determines

[0038] The control unit 1 may display the reference image and the generated image on the display unit 4. The same applies to the images that have been subjected to predetermined processing in the processes from S3 onwards.

[0039] In S3, the processing unit 13 performs a segmentation process on each of the reference image and the generated image to identify and separate areas of objects, such as fish, within the image. This segmentation process includes, following the segmentation process, an exclusion process that removes background areas that do not represent fish and color information of areas that represent fish. In the process of S3, the processing unit 13 may use, for example, the Segment Anything Model (SAM) method. Image 32 in FIG. 5 is an example of an image obtained by performing the process of S3 on generated image 31, and image 42 is an example of an image obtained by performing the process of S3 on reference image 41. The processes of S3 and the following S4 are performed on each frame included in the video clip.

[0040] In S4, the processing unit 13 performs a masking process on each of the reference image and the generated image that have been processed in S3, masking a portion of patches, which are regions obtained by dividing the image into several square pixels. For example, the processing unit 13 may randomly mask patches that account for approximately 50% of the entire image. Masking may also include a process of deleting pixel values ​​of pixels included in the patches and a process of filling pixels with a specific color, such as black. Generated image 33 in FIG. 5 is an example of an image obtained by performing the processes up to S4 on generated image 31, and reference image 43 is an example of an image obtained by performing the processes up to S4 on reference image 41. In generated image 33 and reference image 43, H indicates the height of the image, W indicates the width of the image, and T indicates the number of frames of the video clip (window size).

[0041] Note that the present disclosure also includes a case where at least one of the segmentation process and the masking process is not performed on the reference image and the generated image.

[0042] In S5, the extraction unit 14 inputs each of the reference image and the generated image processed in S4 into a masked video autoencoder that has been sufficiently trained in advance, and extracts latent variables corresponding to each image. In this case, for example, the latent variables corresponding to the reference image (or the latent variables of the reference image) correspond to the latent variables extracted from the reference image processed in S3 and S4. Also, z in FIG. t All shows an example of a latent variable. The learning unit 17 may also perform a process of further learning the masked video autoencoder by comparing a decoded image output from the decoder of the masked video autoencoder, in which the masked portion is predicted, with the image before the masking process is performed.

[0043] Note that graph 57 in FIG. 6 represents the latent variables corresponding to each image, and also represents the concept of imitation learning by information processing device 100, in which the latent variables corresponding to the generated image are made to approach the latent variables corresponding to the reference image.

[0044] In S6, the calculation unit 15 inputs the latent variables corresponding to the reference image and the generated image extracted in the process of S5 to the classifier. As a result, the calculation unit 15 calculates outputs corresponding to "Index" and "Probability" in FIG. 6. "Index" indicates the classification result, i.e., whether the input latent variable corresponds to the reference image or the generated image. In a broad sense, the value of Index is a label indicating which class the input data is classified into. "Probability" indicates the probability that the input latent variable is a latent variable corresponding to the reference image and the probability that it is a latent variable corresponding to the generated image. In a broad sense, Probability indicates the respective probabilities that the input data will be classified into each class. The value of Index corresponds to the class with the higher probability indicated by Probability.

[0045] In S7, the classification unit 16 performs a clustering process to classify the latent variables corresponding to the reference images extracted in the process of S5. The classification unit 16 also performs an SSE (Sum of Squared Errors) check to see if the clustering process is appropriate, and if the result is inappropriate, may adjust the parameters and perform the clustering process again.

[0046] In S8, the calculation unit 15 calculates a first reward r, which is a style reward, based on the output of the classifier in the process of S6 and the result of the clustering process in the process of S7. s (r t s The first reward is calculated as the second reward r B (r t B (also referred to as "third reward r") H (r t H ) along with the reward function r t The calculation unit 15 calculates the first reward r by, for example, the following formulas 1 and 2: s may be calculated.

number

number

[0047] In Equation 1, D(z t ,z t+1 ) indicates the classification result of the classifier.

[0048] W i FG is the number of latent variables belonging to the i-th cluster, N S i the total number of latent variables Σ i N S i As shown in Equation 1 and Equation 3 below, W i FG The larger the value of r sand the reward function r t In this way, a larger reward weight is assigned to a cluster to which more latent variables extracted from the reference image belong. In other words, when the classification unit 16 clusters the latent variables corresponding to the reference image, the calculation unit 15 assigns a value W weighted according to the number of latent variables in each cluster. i FG The reward function after update is calculated as a reward function including the coefficients.

[0049] Graph 58 in FIG. 6 illustrates the results of clustering performed by the classification unit 16. Each bar graph on the left side of graph 58 indicates the same latent variable corresponding to the reference image, and bars of the same color indicate the cluster to which the latent variable corresponding to that bar belongs. In the example of graph 58, the latent variables are classified into six clusters, G1 to G6. For example, the three latent variables corresponding to the three bar graphs on the left side are latent variables classified into cluster G6. The height of each bar graph indicates the number of the same latent variables corresponding to that bar graph. In the example of graph 58, it is shown that of the total number of latent variables corresponding to the reference image, the number of latent variables belonging to cluster G3 is particularly large. As mentioned above, the reward function r t is a weighted value W according to the number of latent variables for each cluster. i FG This means that clusters corresponding to latent variables that appear more frequently tend to be assigned larger reward weights.

[0050] In S9, the calculation unit 15 calculates a second reward r B Here, the following documents (1) and (2) are cited as documents that describe the calculation of rewards based on biological rules. The calculation unit 15 calculates the second reward r in accordance with the descriptions in documents (1) and (2). B may be calculated. (1) Yuko Ishiwaka, Xiao Zeng, Shun Ogawa, Donovan Westwater, Tadayuki Tone, and Masaki Nakada. 2022. DeepFoids: Adaptive Bio-Inspired Fish Simulation with Deep Reinforcement Learning. In Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35. Curran Associates, Inc., 18377-18389. https: / / proceedings.neurips.cc / paper_files / paper / 2022 / file / 74fa9e6bc36aa567fe7cf002b733a30d-Paper-Conference.pdf (2) Yuko Ishiwaka, Xiao S. Zeng, Michael Lee Eastman, Sho Kakazu, Sarah Gross, Ryosuke Mizutani, and Masaki Nakada. 2021. Foids: bio-inspired fish simulation for synthetic generating datasets. ACM Trans. Graph. 40, 6, Article 207 (dec 2021), 15 pages. https: / / doi.org / 10.1145 / 3478513.3480520 The second reward, r, is a biological rule-based reward. B helps stabilize the learning process of imitation learning and reproduce patterns influenced by the biological environment. B The value of is the fish's state at a certain time s t and Action a t , and the next time step, i.e., the state s after a predetermined unit time t+1 Function value r according to B (st ,a t ,s t+1 ) As also shown in FIG. 6, the biological rules may include boundary penalties, energy costs, temperature preferences, and light intensity preferences.

[0051] The boundary penalty is a second reward r when the fish collides with a boundary such as the cage wall or the water surface. B This boundary penalty is a predetermined value that is subtracted from r. This boundary penalty comes from the fact that fish usually swim to avoid hitting walls, etc. In contrast to the boundary penalty, when an individual fish maintains a certain distance or more from its neighbors and maintains its alignment, r B A predetermined value of incentive may be set to be added to the above.

[0052] The energy cost is calculated by the energy expenditure due to the rotation or speed adjustment of the fish's body. B is a predetermined penalty that is subtracted from

[0053] Temperature preference is a set point for temperature change, and light intensity preference is a set point for light change. Temperature change affects the vertical swimming pattern of fish, and light change affects the speed of fish. In other words, temperature preference and light intensity preference are regulated by boundary penalties or energy costs. B This affects the value of

[0054] In S10, the calculation unit 15 calculates a third reward r which is an action-specific reward (high-level task reward). H Calculate the third reward r H is a reward given when multiple objects are in a specific state as a school or when they perform a specific behavior as a school. As shown in FIG. 6, it is a reward given when a school of fish turns clockwise or counterclockwise, lines up, gathers, etc. Therefore, the third reward r H Indirectly contributes to the generation unit 12 reproducing various animations of schools of fish. For example, r H The value of is the fish's state at a certain time s tand Action a t , and the state s at the next time step t+1 , and the policy goal g given as a control signal. t Function value r according to H (s t ,a t ,s t+1, g t ) is given as

[0055] Also, the third reward r H The reward for turning clockwise or counterclockwise is cir , the reward for alignment r ali , and the reward for the set r agg Plus rewards for tracking dom and r sub , feeding reward r feed , and the reward for attracting coh may be included.

[0056] For example, the reward r for turning cir The formula for determining the reward r may include the product of a normal vector in the world coordinate system and a vector from each fish's position toward the center of the cage, and the desired speed of the turn. ali The formula for r can include the angle between each fish and its neighbors. agg contributes to directing each fish towards the center of the school. agg The formula for determining the reward r may include a weight value according to the degree of aggregation of the fish. dom is the reward given when a dominant fish chases another subordinate fish. dom The equation for r can include the relative position of the dominant fish to the subordinate fish. The subordinate fish receives a reward r for moving away from the dominant fish. sub The reward for feeding is r feed is the reward given when a fish comes into contact with an object tagged as bait. feed The formula for determining the distance between the fish and the bait may include a formula for determining whether the distance between the fish and the bait is 0 or a similar value. cohis the reward given when a fish attracts another fish. coh The formula for determining the average position of surrounding fish may be included.

[0057] In S11, the calculation unit 15 calculates a new reward function r according to the first to third rewards. t The calculation unit 15 calculates the reward function r by using, for example, the following formula 3. t The values ​​of the first to third rewards may be scaled to fall within a predetermined range, such as a range of 0 to 1.

number

[0058] The reward function r in Eq. t is the reward r given when multiple objects are in a specific state as a group of objects or when they perform a specific behavior as a group of objects. H It is expressed by a polynomial including terms corresponding to the coefficient W s , W B , and W H are changeable predetermined values, and may be set to, for example, 0.4, 0.1, and 0.5, respectively.

[0059] In S12, the learning unit 17 updates the image generation policy in accordance with the reward function updated in the process of S12 so that the output value of the reward function becomes higher. Note that the image generation policy is based on the fish state s t and the aforementioned goal g t , action (a t |s t ,g t )=N(μ(s t ,g t ),Σ π ) onto a Gaussian distribution, where μ(s t ,g t ) is the mean value of the input, and Σ π is a diagonal covariance matrix of fixed values.

[0060] The above describes the processing flow of the information processing method executed by the information processing device 100. In summary, the information processing method is an information processing method executed by the information processing device 100 that performs adversarial imitation learning using a generator and a classifier and generates a generated image of an object using computer graphics, and includes an acquisition step, a generation step, a processing step, an extraction step, a classification step, a calculation step, and a learning step.

[0061] As described with reference to FIG. 4 and other figures, the information processing device 100 acquires a reference image capturing an area including an object in an acquisition step S1. In a generation step S2, the device performs processing as a generator, generating a generated image according to a policy (policy model) updated according to a reward function. In processing steps S3 and S4, segmentation processing and masking processing are performed on the reference image and the generated image, masking a partial area of ​​each image. In an extraction step S5, the reference image is input to an autoencoder to extract latent variables corresponding to the reference image, and the generated image is input to the autoencoder to extract latent variables corresponding to the generated image. In a classification step S7, the latent variables are clustered into multiple clusters. In calculation steps S6 and S8 to S11, each latent variable is input to a classifier that identifies whether the latent variable corresponds to the reference image or the generated image, and the reward function is updated according to the output result of the classifier. In a learning step S12, the policy is updated according to the updated reward function.

[0062] According to the processing of this example, when generating a generated image of a fish farm, for example, it is possible to generate an image that is more similar to the reference image without the user having to directly or selectively specify parameters such as the swimming style of the fish or the spacing between the fish depending on the fish species, etc. This reduces the burden on the user and improves the reproducibility of the generated image.

[0063] [3. Software implementation example] The functions of the information processing device 100 (hereinafter referred to as the "device") can be realized by a program that causes a computer to function as the device, and a program that causes a computer to function as each control block of the device (particularly each part included in the control unit 1).

[0064] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program, thereby realizing the functions described in each of the above embodiments.

[0065] The program may be non-transitory and may be recorded on one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

[0066] Furthermore, some or all of the functions of the control blocks can be realized by logic circuits. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of the control blocks can also be realized by, for example, a quantum computer.

[0067] Furthermore, each process described in each of the above embodiments may be executed by AI (Artificial Intelligence). In this case, the AI ​​may run on the control device or on another device (for example, an edge computer or a cloud server).

[0068] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0069] 1. Control section 2 Storage section 3 Input section 4 Display section (output section) 11 Acquisition Department 12 Generation part 13 Processing Department 14 Extraction part 15 Calculation section 16 Classification Department 17 Learning Department 51 Masked Video Autoencoder, Autoencoder 100 Information processing device

Claims

1. An information processing device that performs adversarial imitation learning using a generator and a classifier, and generates a generated image of an object using computer graphics, an acquisition unit that acquires a reference image obtained by capturing an area including the object; a generation unit that performs processing as the generator and generates the generated image according to a policy model that is updated according to a reward function; an extraction unit that inputs the reference image to an autoencoder to extract latent variables corresponding to the reference image, and inputs the generated image to the autoencoder to extract latent variables corresponding to the generated image; a calculation unit that inputs each extracted latent variable to the classifier that identifies whether the latent variable corresponds to a reference image or a generated image, and updates a reward function according to an output result of the classifier; a learning unit that updates the policy model in response to the updated reward function; An information processing device comprising:

2. A classification unit that clusters the latent variables into a plurality of clusters, The calculation unit When the classification unit clusters the latent variables corresponding to the reference images, a reward function is calculated that includes, as a coefficient, a value weighted according to the number of the latent variables for each cluster. The information processing device according to claim 1 .

3. a processing unit that performs a masking process on each of the reference image and the generated image to mask a partial area of ​​the image, The extraction unit The reference image subjected to the masking process and the generated image are input to a masked video autoencoder.

3. The information processing device according to claim 1 or 2.

4. The reward function is: The plurality of objects are represented by a polynomial including a term corresponding to a reward to be given when the objects are in a specific state as a group or when the objects perform a specific behavior as a group.

3. The information processing device according to claim 1 or 2.

5. The information processing device according to claim 1 or 2, wherein the object is a fish, and the area is a fish farm.

6. An information processing method executed by an information processing device that performs adversarial imitation learning using a generator and a classifier and generates a generated image of an object using computer graphics, an acquisition step of acquiring a reference image obtained by capturing an area including the object; a generating step of performing processing as the generator, in which the generated image is generated according to a policy model that is updated according to a reward function; an extraction step of inputting the reference image to an autoencoder to extract latent variables corresponding to the reference image, and inputting the generated image to the autoencoder to extract latent variables corresponding to the generated image; a calculation step of inputting each extracted latent variable to the classifier that identifies whether the latent variable corresponds to a reference image or a generated image, and updating a reward function according to an output result of the classifier; a learning step of updating the policy model in response to the updated reward function; An information processing method including:

7. A control program for causing a computer to function as the information processing device according to claim 1, the control program causing a computer to function as the acquisition unit, the generation unit, the extraction unit, the calculation unit, and the learning unit.

Images