Learning device, parameter adjustment method, and recording medium

Abstract

To provide a learning device and the like capable of preventing recognition accuracy in object recognition from degrading.SOLUTION: A learning device includes a learning model for a still image. The learning model comprises mask generation means of generating a first object mask capable of recognizing an area in which an object exists in a still image, for each individual object. A first parameter including at least one parameter used for processing of generating the first object mask is adjusted based on a first loss indicating a difference of the first object mask with respect to a second object mask capable of recognizing an area in which an object exists in a moving image including the still image, for each individual object.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to machine learning technology. [Background Art]

[0002] In the field of machine learning, machine learning models trained to acquire object representations for each individual object included in unsupervised images have been proposed in recent years.

[0003] Specifically, for example, Non-Patent Document 1 proposes a machine learning model trained to acquire object representations for each individual object included in a still image.

[0004] Further, object representations obtained by a machine learning model trained as described above can be utilized, for example, in processing related to object recognition. [Prior Art Literature] [Non-Patent Literature]

[0005] [Non-Patent Document 1] Francesco Locatello, et.al, "Object-Centric Learning with Slot Attention", [online], October 14, 2020, arXiv, [retrieved October 28, 2022], Internet <URL:https: / / arxiv.org / pdf / 2006.15055.pdf> [Summary of the Invention] [Problem to be Solved by the Invention]

[0006] However, according to the technology disclosed in Non-Patent Document 1, for example, in a case where an object having a complex texture is included in a still image, there is a problem that it is difficult to recognize the boundary of the object, which causes a reduction in the acquisition accuracy of the object representation.

[0007] In other words, the technology disclosed in Non-Patent Document 1 presents the aforementioned problem of reduced recognition accuracy in object recognition.

[0008] One objective of this disclosure is to provide a learning device that can prevent a decrease in recognition accuracy in object recognition. [Means for solving the problem]

[0009] In one aspect of this disclosure, the learning device has a learning model for still images. The learning model comprises a mask generation means that generates a first object mask capable of identifying regions in a still image where objects exist for each individual object. A first parameter, which includes at least one parameter used in the process of generating the first object mask, is adjusted based on a first loss that indicates the difference between the first object mask and a second object mask capable of identifying regions in a moving image, including the still image, where objects exist for each individual object.

[0010] In other aspects of this disclosure, the parameter tuning method is applied to a learning model for still images that generates a first object mask capable of identifying regions in a still image where objects exist on an individual object-by-object basis. The first parameter, which includes at least one parameter used in the process of generating the first object mask, is tuned based on a first loss that represents the difference between the first object mask and a second object mask capable of identifying regions in a moving image, including the still image, where objects exist on an individual object-by-object basis.

[0011] In yet another aspect of this disclosure, the recording medium records a program that causes a computer to perform processing on a learning model for still images that generates a first object mask that identifies regions in a still image where objects exist for each individual object. The first parameter, which includes at least one parameter used in the processing for generating the first object mask, is adjusted based on a first loss that indicates the difference between the first object mask and a second object mask that identifies regions in a moving image including the still image where objects exist for each individual object. [Effects of the Invention]

[0012] According to this disclosure, it is possible to prevent a decrease in recognition accuracy in object recognition. [Brief explanation of the drawing]

[0013] [Figure 1] A block diagram showing the hardware configuration of the learning device according to the first embodiment. [Figure 2] A block diagram showing the functional configuration of the learning device according to the first embodiment. [Figure 3] A diagram illustrating the overview of the process involved in generating mask images. [Figure 4] A block diagram illustrating the configuration of an information processing device used to distill the knowledge to be inherited in a learning device according to the first embodiment. [Figure 5] A flowchart showing an example of the processing performed in the learning device according to the first embodiment. [Figure 6] A block diagram illustrating a modified example of the first embodiment. [Figure 7] A diagram illustrating an example of the process involved in generating a composite mask image. [Figure 8] A block diagram showing the functional configuration of the learning device according to the second embodiment. [Modes for carrying out the invention]

[0014] Preferred embodiments of this disclosure will be described below with reference to the drawings.

[0015] [Hardware configuration] Figure 1 is a block diagram showing the hardware configuration of a learning device according to the first embodiment. As shown in Figure 1, the learning device 100 includes an interface (IF) 111, a processor 112, a memory 113, a recording medium 114, and a database (DB) 115.

[0016] IF 111 inputs and outputs data to and from an external device. For example, a still image including an object captured by a camera or the like is input to the learning device 100 via IF 111.

[0017] The processor 112 is a computer such as a CPU (Central Processing Unit), and controls the entire learning device 100 by executing a program prepared in advance. Specifically, the processor 112 performs, for example, processing for acquiring an object representation for each individual object included in a still image.

[0018] The memory 113 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The memory 113 is also used as a working memory while the processor 112 executes various types of processing.

[0019] The recording medium 114 is a non-volatile, non-transitory recording medium such as a disk-shaped recording medium or a semiconductor memory, and is configured to be detachable from the learning device 100. The recording medium 114 stores various programs executed by the processor 112. When the learning device 100 executes various types of processing, the program recorded in the recording medium 114 is loaded into the memory 113 and executed by the processor 112.

[0020] The DB 115 stores, for example, information input via the IF 111, processing results obtained by processing of the processor 112, and the like.

[0021] [Functional Configuration] Figure 2 is a block diagram showing the functional configuration of the learning device according to the first embodiment. As shown in Figure 2, the learning device 100 includes the learning model 10.

[0022] The learning model 10 is configured as a machine learning model for still images that has been trained to acquire object representations for each individual object contained in an unsupervised still image. The learning model 10 also has a configuration equivalent to a world model for still images, such as the model disclosed in Francesco Locatello, et al., "Object-Centric Learning with Slot Attention". Furthermore, as shown in Figure 2, the learning model 10 includes a feature extraction unit 11, a representation acquisition unit 12, and a mask generation unit 13.

[0023] The feature extraction unit 11 functions as a feature extraction means. The feature extraction unit 11 also extracts a feature quantity SGF that represents the features of the entire still image SG input from outside the learning device 100, and outputs the extracted feature quantity SGF to the representation acquisition unit 12.

[0024] The representation acquisition unit 12 functions as a means of acquiring representations. The representation acquisition unit 12 calculates representation vectors SGV corresponding to each individual object contained in the still image SG based on the feature quantities SGF extracted by the feature extraction unit 11. The representation vectors SGV contain the same number of vectors as the number of objects in the still image SG. The representation acquisition unit 12 also acquires the calculation results of the representation vectors SGV as object representations SGR for each individual object in the still image SG, and outputs the acquired object representations SGR to the mask generation unit 13.

[0025] The mask generation unit 13 functions as a mask generation means. The mask generation unit 13 also estimates the object region SGA, which is the region in the still image SG where an object exists, based on the object representation SGR obtained by the representation acquisition unit 12. The mask generation unit 13 also generates an object mask SGM that can identify each individual object region SGA obtained by the aforementioned estimation. The object mask SGM is generated as a mask image in which each individual object region SGA is filled with a different color. The object mask SGM also contains the same number of mask images as the number of objects in the still image SG. Specifically, the mask generation unit 13 generates mask images MGA and MGB as mask images corresponding to the still image SG, for example, as shown in Figure 3. Figure 3 is a diagram showing an overview of the process related to the generation of mask images. Mask image MGA contains mask MA which indicates the object region corresponding to object BA in the still image SG. Mask image MGB contains mask MB which indicates the object region corresponding to object BB in the still image SG. In Figure 3, for illustrative purposes, masks MA and MB are shown not as regions filled with different colors, but as regions with different hatching patterns. Specifically, the mask generation unit 13 generates an object mask SGM that can identify each individual object from the object region SGA, which is the region in the still image SG where an object exists, based on the object representation SGR obtained by the representation acquisition unit 12. The mask generation unit 13 also outputs the object mask SGM and the object representation SGR to the outside of the learning device 100.

[0026] [Knowledge Distillation] In this embodiment, for example, knowledge distilled by an information processing device 200 as shown in Figure 4 is inherited by the learning model 10. The specific configuration of the information processing device 200 that enables such knowledge distillation will be described below. Figure 4 is a block diagram illustrating the configuration of an information processing device used to distill the knowledge to be inherited in the learning device according to the first embodiment.

[0027] The information processing device 200 has the same hardware configuration as the learning device 100. Furthermore, the information processing device 200 is configured to be detachable from the learning device 100. As shown in Figure 4, the information processing device 200 includes a learning model 20, loss calculation units 31, 32, and 33, and parameter adjustment units 41, 42, and 43.

[0028] The learning model 20 is configured as a machine learning model for video that has been trained to acquire object representations for each individual object contained in unsupervised video. The learning model 20 also has a configuration equivalent to a world model for video, such as the model disclosed in Thomas Kipf, et al., "Conditional Object-Centric Learning from Video". As shown in Figure 4, the learning model 20 also includes a feature extraction unit 21, a representation acquisition unit 22, a mask generation unit 23, and a motion estimation unit 24.

[0029] The feature extraction unit 21 extracts a feature quantity DGF that represents the features of the entire video DG input from outside the information processing device 200, and outputs the extracted feature quantity DGF to the representation acquisition unit 22.

[0030] The representation acquisition unit 22 calculates representation vectors DGV corresponding to individual objects present in the moving image DG based on the feature quantities DGF extracted by the feature extraction unit 21. The representation vectors DGV contain the same number of vectors as the number of objects in the moving image DG. The representation acquisition unit 22 also acquires the calculation results of the representation vectors DGV as object representations DGR for each individual object in the moving image DG, and outputs the acquired object representations DGR to the mask generation unit 23 and the motion estimation unit 24.

[0031] The mask generation unit 23 estimates the object region DGA, which is the region in the moving image DG where an object exists, based on the object representation DGR obtained by the representation acquisition unit 22. The mask generation unit 23 also generates an object mask DGM that can identify each individual object region DGA obtained by the aforementioned estimation. The object mask DGM is generated, for example, as a mask image in which each object region DGA is filled with a different color. The object mask DGM also contains the same number of mask images as the number of objects in the moving image DG. Specifically, the mask generation unit 23 generates a mask image similar to the example shown in Figure 3 for each of the multiple time-series images included in the moving image DG by performing the same processing as the mask generation unit 13. In other words, the mask generation unit 23 generates an object mask DGM that can identify each individual object in the object region DGA, which is the region in the moving image DG where an object exists, based on the object representation DGR obtained by the representation acquisition unit 22.

[0032] The motion estimation unit 24 estimates the motion of individual objects included in the moving image DG by calculating optical flow based on the object representation DGR obtained by the representation acquisition unit 22. The motion estimation unit 24 also acquires the estimation results obtained from the above estimation as motion data DGU.

[0033] The loss calculation unit 31 receives the feature quantity SGF extracted by the feature extraction unit 11 and the feature quantity DGF extracted by the feature extraction unit 21 as input. The loss calculation unit 31 also calculates a loss LF, which represents the difference between the feature quantity SGF and the feature quantity DGF, and outputs the calculated loss LF to the parameter adjustment unit 41. Note that the aforementioned difference may be interpreted as error or deviation.

[0034] The loss calculation unit 32 receives the object representation SGR extracted by the representation acquisition unit 12 and the object representation DGR obtained by the representation acquisition unit 22 as input. The loss calculation unit 32 also calculates a loss LR, which represents the difference between the object representation SGR and the object representation DGR, and outputs the calculated loss LR to the parameter adjustment unit 42. Specifically, the loss calculation unit 32 obtains n sets of vectors by associating similar vectors between the n vectors contained in the representation vector DGV of the object representation DGR and the n vectors contained in the representation vector SGV of the object representation SGR, for example, by performing processing using the Hungarian algorithm and cosine similarity. The loss calculation unit 32 then calculates the difference in each of the n sets of vectors obtained as described above as the loss LR. Note that the above difference may be read as error or deviation.

[0035] The loss calculation unit 33 receives the object mask SGM generated by the mask generation unit 13 and the object mask DGM generated by the mask generation unit 23 as inputs. The loss calculation unit 33 also calculates a loss LM, which represents the difference between the object mask SGM and the object mask DGM, and outputs the calculated loss LM to the parameter adjustment unit 43. Specifically, the loss calculation unit 33 obtains n sets of mask images by associating similar mask images from the p mask images contained in the object mask DGM and the p mask images contained in the object mask SGM, for example, by performing processing using the Hungarian algorithm and cosine similarity. The loss calculation unit 33 then calculates the difference in each of the n sets of mask images obtained as described above as the loss LM. Note that the above difference may be read as error or deviation.

[0036] The parameter adjustment unit 41 adjusts at least one parameter used in the process of extracting the feature quantity SGF by the feature extraction unit 11, based on the loss LF calculated by the loss calculation unit 31.

[0037] The parameter adjustment unit 42 adjusts at least one parameter used in the process of acquiring the object representation SGR by the representation acquisition unit 12, based on the loss LR calculated by the loss calculation unit 32.

[0038] The parameter adjustment unit 43 adjusts at least one parameter used in the process of generating the object mask SGM by the mask generation unit 13, based on the loss LM calculated by the loss calculation unit 33.

[0039] With the information processing device 200 having the configuration described above, the loss calculation units 31, 32, and 33 and the parameter adjustment units 41, 42, and 43 perform processing related to knowledge distillation, thereby enabling the learning model 10 to inherit the characteristics (advantages) of the learning model 20. In other words, the learning device 100 can set the parameters related to the processing of each part of the learning model 10 (feature extraction unit 11, representation acquisition unit 12, and mask generation unit 13) so that the characteristics (advantages) of the learning model 20 corresponding to the knowledge distilled by the information processing device 200 are inherited.

[0040] When performing knowledge distillation using the information processing device 200 having the configuration described above, for example, a moving image DG may be input to the learning model 20, and still images SG obtained by dividing the moving image DG into individual frames may be sequentially input to the learning model 10.

[0041] Furthermore, when performing knowledge distillation using the information processing device 200 having the configuration described above, it is desirable that the parameter adjustments by the parameter adjustment units 41, 42, and 43 be repeated until the loss LF, loss LR, and loss LM all become 0. In other words, it is desirable that at least one parameter used in the process related to the extraction of feature quantities SGF, at least one parameter used in the process related to the acquisition of object representations SGR, and at least one parameter used in the process related to the generation of object masks SGM be adjusted so that the loss LF, loss LR, and loss LM all become 0. According to the parameter adjustments described above, the robustness to textures, which is a characteristic (advantage) of the learning model 20, can be inherited by the learning model 10. In addition, according to the parameter adjustments described above, for example, even if an object with a complex texture is included in a still image, the learning model 10 can appropriately recognize the boundary of the object, thereby improving the accuracy of object representation acquisition by the learning model 10.

[0042] [Processing flow] Next, the processing flow performed in the learning device according to the first embodiment will be described. Figure 5 is a flowchart showing an example of the processing performed in the learning device according to the first embodiment.

[0043] The learning device 100 sets the parameters related to the processing of each part of the learning model 10 (feature extraction unit 11, representation acquisition unit 12, and mask generation unit 13) so that the characteristics (advantages) of the learning model 20 corresponding to the knowledge distilled by the information processing device 200 are inherited (step S11). In step S11, it is sufficient to perform the processing corresponding to the knowledge distillation described above.

[0044] After setting parameters in step S11, the learning device 100 extracts feature quantities that represent the features of the entire still image input from an external source (step S12). The processing from step S12 onward corresponds to the process of obtaining object representations and object masks corresponding to individual objects contained in the still image that is input to the learning device 100 after the knowledge distillation described above has been performed.

[0045] Based on the features extracted in step S12, the learning device 100 calculates representation vectors corresponding to each individual object in the still image processed in step S12. The learning device 100 also obtains the results of the above representation vector calculation as object representations for each individual object in the still image processed in step S12 (step S13).

[0046] Based on the object representation obtained in step S13, the learning device 100 estimates the object region, which is the area in the still image processed in step S12 where an object exists. The learning device 100 also generates an object mask that can identify each individual object region obtained through the aforementioned estimation (step S14).

[0047] The learning device 100 outputs the object representation acquired in step S13 and the object mask generated in step S14 to the outside (step S15).

[0048] As described above, according to this embodiment, it is possible to obtain object representations for each individual object contained in a still image by using a learning model 10 that inherits the characteristics (advantages) of the learning model 20 obtained by the knowledge distillation process. Therefore, according to this embodiment, it is possible to prevent a decrease in recognition accuracy that occurs when performing object recognition based on object representations.

[0049] In this embodiment, the learning device 100 only needs to have parameters related to at least one of the processes among the feature extraction unit 11, the representation acquisition unit 12, and the mask generation unit 13 adjusted by knowledge distillation using the information processing device 200. In such cases, it is desirable that the parameters be adjusted so that all losses to be calculated become zero.

[0050] Furthermore, according to this embodiment, parameter adjustments for each part of the learning model 10 may be performed by distilling knowledge obtained from a series of processes, including, for example, processes performed in each part of the learning model 20 and one or more processes performed based on the object mask DGM and motion data DGU obtained by the learning model 20. The processes performed based on the object mask DGM and motion data DGU may include, for example, a process to extract the features of individual objects contained in the video DG.

[0051] Furthermore, this embodiment can also be applied to an information processing system having, for example, a learning device 100 and an information processing device 200.

[0052] [Differentiation] The following describes modifications of the above embodiment. For simplicity, detailed explanations of the parts to which the previously described processes can be applied will be omitted as appropriate.

[0053] This embodiment is not limited to being applicable to a learning device 100 and information processing device 200 having the configuration shown in Figure 4, but can also be applied to a learning device 100A and information processing device 200A having the configuration shown in Figure 6, for example. Figure 6 is a block diagram illustrating a modified example of the first embodiment.

[0054] Learning device 100A has the same hardware configuration as learning device 100. Learning device 100A also includes learning model 10A.

[0055] The learning model 10A is configured as a machine learning model for still images that has been trained to generate object masks capable of identifying the object regions of each individual object contained in an unsupervised still image. The learning model 10A also has a configuration equivalent to an instance segmentation model. Furthermore, as shown in Figure 6, the learning model 10A has a mask generation unit 13A.

[0056] The mask generation unit 13A generates object masks SGN that can identify each individual object in the object region SGA, which is the area where an object exists in the still image SG input from outside the learning device 100A, and outputs the generated object masks SGN to the outside of the learning device 100A. The object masks SGN are generated, for example, as mask images in which each object region SGA is filled with a different color. In addition, the object masks SGN contain the same number of mask images as the number of objects in the still image SG. Specifically, the mask generation unit 13A can generate mask images similar to those exemplified in Figure 3 by performing the same processing as the mask generation unit 13.

[0057] The information processing device 200A has the same hardware configuration as the learning device 100. Furthermore, the information processing device 200A is configured to be detachable from the learning device 100A. As shown in Figure 6, the information processing device 200A includes a learning model 20, a loss calculation unit 33A, and a parameter adjustment unit 43A.

[0058] The loss calculation unit 33A receives the object mask SGN generated by the mask generation unit 13A and the object mask DGM generated by the mask generation unit 23 as inputs. The loss calculation unit 33A also calculates a loss LN, which represents the difference between the object mask SGN and the object mask DGM, and outputs the calculated loss LN to the parameter adjustment unit 43A. Specifically, the loss calculation unit 33A calculates the loss LN as the difference between the composite mask image obtained by combining the highly reliable masks from each mask image included in the object mask DGM and the composite mask image obtained by combining the highly reliable masks from each mask image included in the object mask SGN. The aforementioned reliability can be expressed as a value that is relatively large in one mask representing the foremost object region and relatively small in other masks that include object regions hidden by the foremost object region. The aforementioned difference may also be interpreted as error or misalignment. Furthermore, as illustrated in Figure 3, if there are no overlapping regions (pixels) between the mask MA of the mask image MGA and the mask MB of the mask image MGB, the loss calculation unit 33A may calculate the loss LN based on a composite mask image obtained by directly combining the mask images MGA and MGB. Also, if there are overlapping regions (pixels) between the mask MC of the mask image MGC and the mask MD of the mask image MGD, as shown in Figure 7, the loss calculation unit 33A may calculate the loss LN based on a composite mask image CMG having a composite mask CM that is synthesized to show the foremost object region in the mask images MGC and MGD. The composite mask CM in Figure 7 shows the synthesis result when the confidence level of mask MC is relatively low and the confidence level of mask MD is relatively high in the region where mask MC and mask MD overlap. Figure 7 is a diagram showing an example of the process related to the generation of a composite mask image.

[0059] The parameter adjustment unit 43A adjusts at least one parameter used in the process of generating the object mask SGN by the mask generation unit 13A, based on the loss LN calculated by the loss calculation unit 33A.

[0060] With the information processing device 200A having the configuration described above, the loss calculation unit 33A and the parameter adjustment unit 43A perform processing related to knowledge distillation, thereby enabling the characteristics (advantages) of the learning model 20 to be inherited by the learning model 10A. In other words, the learning device 100A can set the parameters related to the processing of the mask generation unit 13A in the learning model 10A so that the characteristics (advantages) of the learning model 20 corresponding to the knowledge distilled by the information processing device 200A are inherited.

[0061] When performing knowledge distillation using the information processing device 200A having the configuration described above, it is desirable that the parameter adjustment by the parameter adjustment unit 43A be repeated until the loss LN becomes 0. In other words, it is desirable that at least one parameter used in the process related to the generation of the object mask SGN be adjusted so that the loss LN becomes 0. By performing the parameter adjustment as described above, the robustness to textures, which is a characteristic (advantage) of the learning model 20, can be inherited by the learning model 10A.

[0062] As described above, this modified version uses a learning model 10A that inherits the characteristics (advantages) of the learning model 20 obtained through the knowledge distillation process, and can acquire an object mask capable of identifying the object region of each individual object contained in a still image. Therefore, this modified version can prevent the decrease in recognition accuracy that occurs when performing object recognition based on the object mask.

[0063] Furthermore, knowledge distillation using the information processing device 200A can be applied not only when the learning model 10A has a configuration corresponding to an instance segmentation model, but also, for example, when the learning model 10A has a configuration corresponding to either a panoptic segmentation model or an amodal segmentation model.

[0064] If the learning model 10A has a configuration corresponding to a panoptic segmentation model, the object mask SGN and the background mask SGH, which shows the area other than the object region in the still image SG, should be output from the mask generation unit 13A. Also, if the learning model 10A has a configuration corresponding to a panoptic segmentation model, the object mask DGM and the background mask DGH, which shows the area other than the object region in the moving image DG, should be output from the mask generation unit 23. Also, if the learning model 10A has a configuration corresponding to a panoptic segmentation model, the loss calculation unit 33A should calculate the loss LN, which shows the difference between the object mask SGN and the object mask DGM, and the loss LH, which shows the difference between the background mask SGH and the background mask DGH. The aforementioned difference may also be read as error or deviation. Also, if the learning model 10A has a configuration corresponding to a panoptic segmentation model, the parameter adjustment based on the losses LN and LH should be performed by the parameter adjustment unit 43A. In such cases, it is desirable that the parameter adjustment by the parameter adjustment unit 43A be repeated until both loss LN and loss LH become 0. In other words, if the learning model 10A has a configuration corresponding to a panoptic segmentation model, it is desirable that at least one parameter used in the process of generating the object mask SGN and background mask SGH be adjusted so that both loss LN and loss LH become 0.

[0065] If the learning model 10A has a configuration equivalent to an amodal segmentation model, the loss LZ, which represents the difference between the object mask SGN and the object mask DGM, should be calculated by the loss calculation unit 33A. The loss LZ should be calculated in the same way as the loss LM calculated by the loss calculation unit 33. The aforementioned difference may also be interpreted as error or deviation. Furthermore, if the learning model 10A has a configuration equivalent to an amodal segmentation model, the parameter adjustment based on the loss LZ should be performed by the parameter adjustment unit 43A. In such cases, it is desirable that the parameter adjustment by the parameter adjustment unit 43A be repeated until the loss LZ becomes 0. In other words, if the learning model 10A has a configuration equivalent to an amodal segmentation model, it is desirable that at least one parameter used in the process related to the generation of the object mask SGN be adjusted so that the loss LZ becomes 0.

[0066] <Second Embodiment> Figure 8 is a block diagram showing the functional configuration of the learning device according to the second embodiment.

[0067] The learning device 500 according to this embodiment has the same hardware configuration as the learning device 100. The learning device 500 also has a learning model 510 for still images. The learning model 510 also includes a mask generation means 520.

[0068] The mask generation means 520 generates a first object mask that allows for the identification of individual objects in the region where an object exists in the still image.

[0069] The first parameter, which includes at least one parameter used in the process for generating the first object mask, is adjusted based on a first loss that indicates the difference between the first object mask and a second object mask that can identify the region in which an object exists in a moving image including the still image for each individual object.

[0070] According to this embodiment, it is possible to prevent a decrease in recognition accuracy in object recognition.

[0071] Some or all of the above embodiments may also be described as follows, but are not limited to the following:

[0072] (Note 1) A learning device having a learning model for still images, The learning model comprises a mask generation means that generates a first object mask capable of identifying the region in which an object exists in a still image for each individual object, A learning device in which a first parameter, including at least one parameter used in the process of generating the first object mask, is adjusted based on a first loss that indicates the difference between the first object mask and a second object mask capable of identifying regions where objects exist for each individual object in a moving image including a still image.

[0073] (Note 2) The first parameter is adjusted in the learning device of Appendix 1 such that the first loss is 0.

[0074] (Note 3) The learning model further includes a feature extraction means for extracting a first feature quantity that represents the features of the entire still image, A learning device as described in Appendix 1, wherein a second parameter, which includes at least one parameter used in the process of extracting the first feature, is adjusted based on a second loss that indicates the difference between the first feature and a second feature representing the features of the entire video image.

[0075] (Note 4) The learning device of Appendix 3, wherein the first and second parameters are adjusted such that both the first and second losses are zero.

[0076] (Note 5) The learning model further includes representation acquisition means for acquiring a first object representation for each individual object in the still image, A learning device as described in Appendix 1, wherein a third parameter, including at least one parameter used in the process of acquiring the first object representation, is adjusted based on a third loss indicating the difference between the first object representation and a second object representation for each individual object in the video.

[0077] (Note 6) The learning device of Appendix 5, wherein the first and third parameters are adjusted so that both the first and third losses are zero.

[0078] (Note 7) The learning model includes a feature extraction means for extracting a first feature quantity that represents the features of the entire still image, a representation acquisition means for acquiring a first object representation for each individual object in the still image based on the first feature quantity, and a mask generation means for generating a first object mask based on the first object representation. A second parameter, which includes at least one parameter used in the process for extracting the first feature, is adjusted based on a second loss that represents the difference between the first feature and a second feature representing the features of the entire video image. A learning device as described in Appendix 1, wherein a third parameter, including at least one parameter used in the process of acquiring the first object representation, is adjusted based on a third loss indicating the difference between the first object representation and a second object representation for each individual object in the video.

[0079] (Note 8) The learning device of Appendix 7, wherein the first parameter, the second parameter, and the third parameter are adjusted so that the first loss, the second loss, and the third loss are all zero.

[0080] (Note 9) A parameter tuning method applied to a learning model for still images that generates a first object mask capable of identifying the region in which an object exists in a still image for each individual object, A parameter adjustment method for adjusting a first parameter, which includes at least one parameter used in the process for generating the first object mask, based on a first loss that indicates the difference between the first object mask and a second object mask that can identify the region where an object exists in a moving image including a still image for each individual object.

[0081] (Note 10) A recording medium that stores a program that causes a computer to perform processing on a learning model for still images, which generates a first object mask that can identify the region in which an object exists in a still image for each individual object, A recording medium that records a program causing the computer to perform a process of adjusting a first parameter, which includes at least one parameter used in the process of generating the first object mask, based on a first loss that indicates the difference between the first object mask and a second object mask that can identify the region in which an object exists for each individual object in a moving image including the still image.

[0082] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the above embodiments and examples. Various modifications to the structure and details of the present disclosure are possible, as can be understood by those skilled in the art within the scope of the present disclosure. [Explanation of Symbols]

[0083] 10 Learning Models 11 Feature Extraction Unit 12 Expression acquisition part 13 Mask generation unit 100 Learning Devices

Claims

1. A learning device having a learning model for still images, The learning model comprises a mask generation means that generates a first object mask capable of identifying the region in which an object exists in a still image for each individual object, A learning device in which a first parameter, including at least one parameter used in the process for generating the first object mask, is adjusted based on a first loss that indicates the difference between the first object mask and a second object mask capable of identifying regions where objects exist for each individual object in a moving image including a still image.

2. The learning device according to claim 1, wherein the first parameter is adjusted so that the first loss is zero.

3. The learning model further includes a feature extraction means for extracting a first feature quantity that represents the features of the entire still image, The learning device according to claim 1, wherein a second parameter, which includes at least one parameter used in the process for extracting the first feature, is adjusted based on a second loss that indicates the difference between the first feature and a second feature representing the features of the entire video image.

4. The learning device according to claim 3, wherein the first parameter and the second parameter are adjusted such that both the first loss and the second loss are zero.

5. The learning model further includes representation acquisition means for acquiring a first object representation for each individual object in the still image, The learning device according to claim 1, wherein a third parameter, which includes at least one parameter used in the process of acquiring the first object representation, is adjusted based on a third loss indicating the difference between the first object representation and a second object representation for each individual object in the moving image.

6. The learning device according to claim 5, wherein the first parameter and the third parameter are adjusted such that both the first loss and the third loss are zero.

7. The learning model includes a feature extraction means for extracting a first feature quantity that represents the features of the entire still image, a representation acquisition means for acquiring a first object representation for each individual object in the still image based on the first feature quantity, and a mask generation means for generating a first object mask based on the first object representation. The second parameter, which includes at least one parameter used in the process for extracting the first feature, is adjusted based on a second loss that represents the difference between the first feature and a second feature representing the features of the entire video image. The learning device according to claim 1, wherein a third parameter, which includes at least one parameter used in the process of acquiring the first object representation, is adjusted based on a third loss indicating the difference between the first object representation and a second object representation for each individual object in the moving image.

8. The learning device according to claim 7, wherein the first parameter, the second parameter, and the third parameter are adjusted such that the first loss, the second loss, and the third loss are all zero.

9. A parameter tuning method applied to a learning model for still images that generates a first object mask capable of identifying the region in which an object exists for each individual object in a still image, A parameter adjustment method for adjusting a first parameter, which includes at least one parameter used in the process for generating the first object mask, based on a first loss that indicates the difference between the first object mask and a second object mask that can identify the region where an object exists in a moving image including a still image for each individual object.

10. A recording medium that stores a program that causes a computer to perform processing on a learning model for still images, which generates a first object mask that can identify the region in which an object exists in a still image for each individual object, A recording medium that records a program causing the computer to perform a process of adjusting a first parameter, which includes at least one parameter used in the process of generating the first object mask, based on a first loss that indicates the difference between the first object mask and a second object mask that can identify the region in which an object exists for each individual object in a moving image including the still image.

Images