Model generation method and model generation system
The model generation method improves generalization performance of video extraction models by focusing on input sentence content through a multi-sentence training approach, ensuring specific loss relationships, resulting in a model capable of accurately extracting matching sections from diverse input sentences.
Patent Information
- Application Number
- JP2023083333
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-19
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2043-05-19
AI Technical Summary
Conventional machine learning models for video extraction tasks lack sufficient generalization performance for various input sentences.
A model generation method that includes inputting a plurality of sentences, including base and sub-sentences, to a video extraction model, calculating losses based on matching intervals, and performing machine learning to ensure the second loss is smaller than sub-losses, thereby focusing on the content of the input sentence.
The method enables the generation of a video extraction model with high generalization performance for various input sentences by enhancing the semantic relationship between the input sentence and the estimated matching section.
Smart Images

Figure 2024166908000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to generating machine learning models. [Background technology]
[0002] In recent years, techniques have been proposed for generating machine learning models that enable appropriate processing of various tasks.
[0003] For example, Patent Document 1 discloses a technique for generating a machine learning model for processing a task of inputting data of a plurality of different modalities and outputting data of a modality different from the input.
[0004] Other documents that demonstrate the technical level of this technical field include the following Patent Documents 2 and 3. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] International Publication No. 2021 / 182199 [Patent Document 2] Patent Publication No. 2022-072444 [Patent Document 3] Patent Publication No. 2021-189892 Summary of the Invention [Problem to be solved by the invention]
[0006] One task is to extract segments from a video that match a sentence as a query. Conventional machine learning models for processing such tasks (hereafter simply called "video extraction models") have not been able to achieve sufficient generalization performance for various query sentences.
[0007] One objective of the present disclosure is to provide a technology that enables generation of an image extraction model with high generalization performance for various input sentences. [Means for solving the problem]
[0008] A first aspect of the present disclosure relates to a model generation method for generating a video extraction model that extracts a matching section that matches the content of an input sentence from a video.
[0009] A model generation method according to a first aspect is executed by a computer and includes inputting a plurality of sentences to a video extraction model and extracting a plurality of matching intervals for each of the plurality of sentences from a training video. The plurality of sentences include a base sentence and at least one sub-sentence shorter than the base sentence. The at least one sub-sentence includes at least one of a positive example sub-sentence including a word included in the base sentence and not including a noise word unrelated to the base sentence and a negative example sub-sentence including at least a noise word. The plurality of matching intervals include a base matching interval for the base sentence and at least one sub-matching interval for the at least one sub-sentence. The model generation method further includes calculating a first loss by processing a learning task to reconstruct a base sentence based on features of the training video corresponding to the correct answer section, calculating a second loss by processing a learning task to reconstruct the base sentence based on features of the training video corresponding to the base matching section, calculating at least one sub-loss by processing a learning task to reconstruct the base sentence based on features of the training video corresponding to the at least one sub-matching section, and performing machine learning of the video extraction model such that the first loss is smaller than the second loss and the second loss is smaller than at least one sub-loss.
[0010] A second aspect of the present disclosure relates to a model generation system that generates a video extraction model that extracts a matching section that matches the content of an input sentence from a video.
[0011] A model generation system according to a second aspect includes one or more processors, the one or more processors being configured to execute a process of inputting a plurality of sentences to a video extraction model and extracting a plurality of matching sections for each of the plurality of sentences from a training video. The plurality of sentences include a base sentence and at least one sub-sentence shorter than the base sentence. The at least one sub-sentence includes at least one of a positive example sub-sentence including a word included in the base sentence and not including a noise word unrelated to the base sentence, and a negative example sub-sentence including at least a noise word. The plurality of matching sections include a base matching section for the base sentence and at least one sub-matching section for the at least one sub-sentence. The one or more processors are further configured to execute the following processes: calculating a first loss by processing a learning task to reconstruct a base sentence based on features of the training video corresponding to the correct answer section; calculating a second loss by processing a learning task to reconstruct the base sentence based on features of the training video corresponding to the base matching section; calculating at least one sub-loss by processing a learning task to reconstruct the base sentence based on features of the training video corresponding to the at least one sub-matching section; and performing machine learning of the video extraction model such that the first loss is smaller than the second loss and the second loss is smaller than at least one sub-loss. Effect of the Invention
[0012] According to the present disclosure, the learning of the video extraction model is performed so as to focus on the content of a part of the input sentence, thereby making it possible to generate a video extraction model with high generalization performance for various input sentences. [Brief description of the drawings]
[0013] [Figure 1] FIG. 2 is a diagram for explaining an overview of the function of an image extraction model according to the present embodiment. [Diagram 2] FIG. 13 is a diagram illustrating an example of a configuration of an image extraction model at the time of inference. [Diagram 3] 4 is a flowchart showing a model generating method according to the present embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a process executed in the model generation method according to the present embodiment. [Diagram 5] FIG. 4 is a diagram illustrating an example of a process executed in the model generation method according to the present embodiment. [Figure 6] FIG. 2 is a diagram illustrating an example of a configuration of an image extraction model during learning. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0014] Hereinafter, the present embodiment will be described with reference to the drawings.
[0015] 1.Video Extraction Model The model generation method according to this embodiment generates a video extraction model that uses a sentence as a query and extracts sections from a video that match the content of the sentence (hereinafter, referred to as "matching sections").
[0016] 1(A) and 1(B) are diagrams for explaining an outline of the function of an image extraction model 1 according to this embodiment.
[0017] The image extraction model 1 according to this embodiment functions by processing executed by a computer 100. The computer 100 includes one or more processors 110 (hereinafter simply referred to as "processor 110") and one or more storage devices 120 (hereinafter simply referred to as "storage device 120"). The processor 110 executes various processes. The storage device 120 is connected to the processor 110 and stores various information required for the processor 110 to execute the processes. The processor 110 is configured, for example, by a central processing unit (CPU) including an arithmetic unit, a register, and the like. The storage device 120 is configured by a recording medium such as a read only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), or a solid state drive (SSD).
[0018] The video extraction model 1 is stored in the storage device 120. The processor 110 reads the video extraction model 1 from the storage device 120 and executes processing, thereby realizing the functions of the video extraction model 1. The video extraction model 1 may be realized as a computer program. In particular, the video extraction model 1 may be stored in a computer-readable recording medium.
[0019] The video extraction model 1 receives as input a query sentence 10 (hereinafter referred to as the "input sentence 10") and a video 20 from which a matching section is to be extracted. The video extraction model 1 then extracts a matching section for the input sentence 10 from the video 20. In FIG. 1(A), the matching section is represented as [SP, EP]. SP is the start point of the matching section, and EP is the end point of the matching section. The video extraction model 1 may be configured to output data of the video 20 corresponding to the matching section.
[0020] Fig. 1(B) shows an example of a matching section for the input sentence 10 shown in Fig. 1(A). In Fig. 1(B), it can be seen that the video 20 corresponding to the matching section matches the content of the input sentence 10. The video extraction model 1 aims to achieve such a function during inference.
[0021] 2 is a diagram showing an example of the configuration of the video extraction model 1 at the time of inference. The video extraction model 1 includes a sentence feature extraction unit 210, a video feature extraction unit 220, and a matching section estimation unit 230.
[0022] The sentence feature extraction unit 210 is composed of a text encoder 211 and a text transformer 212. The text encoder 211 outputs a distributed representation for each word included in the input sentence 10. The text transformer 212 is a Transformer model that receives the output of the text encoder 211 as input. The output of the text transformer 212 becomes the output of the sentence feature extraction unit 210.
[0023] The video feature extraction unit 220 is composed of a video encoder 221 and a vision transformer 222. The video encoder 221 outputs a feature amount for each frame included in the video 20. The vision transformer 222 is a Transformer model that receives the output of the video encoder 221 as an input. The output of the vision transformer 222 becomes the output of the video feature extraction unit 220.
[0024] The matching section estimation unit 230 estimates a matching section from the output of the sentence feature extraction unit 210 and the output of the video feature extraction unit 220. The matching section estimation unit 230 can be configured, for example, with a Transformer model and a fully connected layer. In this case, the Transformer model calculates mutual attention between the output of the sentence feature extraction unit 210 and the output of the video feature extraction unit 220. Then, the fully connected layer calculates a matching section for the input sentence 10 from the feature amount based on the mutual attention. The fully connected layer may be configured to output the degree of match between the content of the input sentence 10 and each frame of the video 20. In this case, the matching section is represented, for example, by a normalized distribution for each frame of the video 20. At this time, the SP and EP can be given from the center and width of the distribution.
[0025] 2. Model generation method The model generation method according to the present embodiment generates a video extraction model 1 by implementing machine learning. The model generation method according to the present embodiment is executed by a computer 100. The storage device 120 may be configured to store a computer program (hereinafter referred to as a "model generation program") for causing the processor 110 to execute each process in the model generation method according to the present embodiment. In this case, the model generation method and model generation system according to the present embodiment are realized by the processor 110 reading and executing the model generation program.
[0026] The model generation method according to this embodiment will be described below with reference to Fig. 3. Fig. 3 is a flowchart showing the model generation method according to this embodiment.
[0027] In step S100, the processor 110 acquires training data. The training data is composed of a combination of a training sentence (hereinafter referred to as a "base sentence"), a training video (hereinafter referred to as a "training video"), and a correct answer section that provides a correct answer for a matching section for the base sentence. The computer 100 may be configured to acquire the training data via a user interface or communication, and store the acquired training data in the storage device 120. The training data may include multiple data combinations of the base sentence, the training video, and the correct answer section. The following description focuses on one combination.
[0028] Next, in step S110, the processor 110 generates a sub-sentence shorter than the base sentence. In this embodiment, the sub-sentence is generated to include both a positive example sub-sentence that includes a word included in the base sentence (hereinafter referred to as an "extracted word") but does not include a word unrelated to the base sentence (hereinafter referred to as a "noise word"), and a negative example sub-sentence that includes at least a noise word. In particular, in this embodiment, the negative example sub-sentence is generated to include a first negative example sub-sentence that includes both an extracted word and a noise word, and a second negative example sub-sentence that does not include an extracted word but includes a noise word. However, the sub-sentence may be generated to include at least one of the positive example sub-sentence and the negative example sub-sentence. The sub-sentence can be considered to be a sentence obtained by breaking down a part of the content of the base sentence 11.
[0029] Here, the noise words may be, for example, words included in separate sentences acquired as training data, or may be managed as data stored in the storage device 120.
[0030] The processor 110 can execute a process of generating a sub-sentence as follows. FIG. 4(A) is a diagram for explaining an example of a process of generating a sub-sentence 12. In the example shown in FIG. 4(A), the sub-sentence 12 is generated from a base sentence 11 and a fill-in sentence 15 in which some words are missing. The fill-in sentence 15 is composed of a template section in which a specific word is set, and a fill-in section indicating the position of the missing word. In addition, the part of speech of the missing word is specified in the fill-in section. FIG. 4(A) shows three fill-in sentences 15. For example, in "The person is [Verb] [Noun].", the template section is "The", "person", and "is", and the fill-in section is [Verb] and [Noun]. The fill-in sentence 15 may be configured to be learnable. For example, the words set in the template section may be expressed by learnable tokens. This allows an optimal template section to be configured through learning.
[0031] First, the processor 110 performs part-of-speech tagging on each extracted word in the base sentence 11. An example of an extracted word 14 tagged with a part-of-speech tag is shown in FIG.
[0032] The processor 110 then generates the subsentence 12 by completing the fill-in sentence 15 using the extracted words 14 or noise words corresponding to the parts of speech defined in the fill-in section. Specifically, the processor 110 can generate a positive example subsentence by completing the fill-in sentence 15 using only the extracted words 14. The processor 110 can also generate a first negative example subsentence by completing the fill-in sentence 15 using both the extracted words 14 and the noise words. The processor 110 can also generate a second negative example subsentence by completing the fill-in sentence 15 using only the noise words.
[0033] FIG. 4(A) shows an example of a subsentence 12 generated for "The person is [Verb] [Noun]." when the extracted words 14 are "cut" and "dog's hair" and the noise words are "add" and "onion." The extracted words 14 corresponding to the parts of speech defined in the fill-in section or the noise words may be selected randomly. The processor 110 may also generate a plurality of positive example subsentences, a first negative example subsentence, or a second negative example subsentence.
[0034] 3 again. Next, in step S120, the processor 110 inputs the base sentence 11 and each of the generated sub-sentences 12 to the video extraction model 1, and extracts a plurality of matching sections for each sentence from the training video. That is, in this embodiment, the plurality of matching sections include a base matching section estimated for the base sentence 11, a positive example sub-matching section estimated for the positive example sub-sentence, a first negative example sub-matching section estimated for the first negative example sub-sentence, and a second negative example sub-matching section estimated for the second negative example sub-sentence.
[0035] Next, in step S130, the processor 110 calculates the loss for the correct answer interval based on the base matching interval. Specifically, the processor 110 calculates a regression loss L according to the difference between the base matching interval and the correct answer interval. reg Calculate.
[0036] Next, in step S140, the processor 110 generates a learning task to reconstruct the base sentence 11. For example, the processor 110 generates a learning task to reconstruct masked words by masking some words from the base sentence 11. FIG. 4B shows an example of the generated learning task 16. In FIG. 4B, the learning task 16 is generated by masking "blue" in the base sentence 11. In this case, the number of words to be masked may be multiple. For example, the learning task 16 may be generated by masking 30% of the words randomly in the base sentence 11. Since the learning task 16 is generated from the base sentence 11, it is a self-supervised learning task.
[0037] Next, in step S150, the processor 110 processes the learning task 16 based on the feature of the training video corresponding to each matching section, thereby calculating a loss for each matching section. The feature of the training video corresponding to the matching section is calculated, for example, by weighting the feature of the training video according to the matching section. When the matching section is calculated by a distribution for each frame, the weighting can be performed by multiplying the feature of each frame of the training video by the value of the corresponding distribution. Alternatively, the feature of the training video corresponding to the matching section is the feature of the training video included in the matching section. The loss is given according to the error of the reconstructed base sentence 11. For example, when processing the learning task 16 of reconstructing masked words, the loss can be calculated by the cross-entropy error. That is, in this case, the loss indicates the inference error of the masked words.
[0038] FIG. 5 is a diagram showing an example of the configuration of the process related to step S150. In FIG. gt , EP gt ],[SP q , EP q ], and [SP p , EP p5, the first negative example sub-matching interval and the second negative example sub-matching interval are omitted.
[0039] The feature amounts of the training video 21 acquired by the video encoder 221 are weighted according to each matching section. As a result, feature amounts of the training video 21 corresponding to each of the correct answer section, the base matching section, the positive example sub-matching section, the first negative example sub-matching section, and the second negative example sub-matching section are acquired.
[0040] The task processing unit 241 processes the learning task 16 based on the feature amount of the training video 21 corresponding to each matching section. The configuration of the task processing unit 241 may be a suitable configuration depending on the content of the learning task 16. For example, when processing the learning task 16 of reconstructing masked words, the task processing unit 241 can be configured with a Transformer model and a fully connected layer. In this case, the Transformer model calculates mutual attention between the feature amount of the training video 21 corresponding to the matching section and each word included in the masked base sentence 11. Then, the fully connected layer calculates an inference result of the masked word from the feature amount based on the mutual attention.
[0041] The loss calculation unit 242 calculates a loss from each processing result of the task processing unit 241. Specifically, a loss (hereinafter referred to as the "first loss") L is calculated from the processing result of the learning task 16 based on the feature amount of the training video 21 corresponding to the correct answer section. gt Similarly, a loss (hereinafter referred to as the “second loss”) L is calculated from the processing result of the learning task 16 based on the feature amount of the training video 21 corresponding to the base matching section. q Similarly, a loss (hereinafter referred to as a “positive example sub-loss”) L is calculated from the processing result of the learning task 16 based on the feature amount of the training video 21 corresponding to the positive example sub-matching section. pSimilarly, a loss (hereinafter referred to as the “first negative example sub-loss”) L c Similarly, a loss (hereinafter referred to as the “second negative example sub-loss”) L is calculated from the processing result of the learning task 16 based on the feature amount of the training video 21 corresponding to the second negative example sub-matching section. o is calculated.
[0042] Referring again to FIG. 3, next, in step S160, the processor 110 performs machine learning of the video extraction model 1 based on each calculated loss. Specifically, the processor 110 performs machine learning so as to reduce the loss function L of the following formula. L is L reg ,L rec , and L rank It is composed of the linear sum of the three loss functions. In other words, reducing L reg ,L rec , and L rank This means making each smaller. reg ,λ rec , and λ rank are hyperparameters that define the contribution of each loss function to L.
[0043]
number
[0044] L reg As mentioned above, L is the regression loss according to the difference between the base matching interval and the correct answer interval. That is, L reg Implementing machine learning to reduce means supervised learning regarding the correct answer interval.
[0045] L rec is constructed using the first loss, the second loss, or the positive example sub-loss. For example, L recis the sum of the first loss, the second loss, and the positive example sub-loss. The loss calculated by processing the learning task 16 is one of the indices measuring the semantic relevance between the training video 21 of the matching section and the base sentence 11. This is because it is considered that the higher the semantic relevance between the training video 21 of the matching section and the base sentence 11, the higher the accuracy of the base sentence 11 reconstructed from the training video 21 of the matching section. Therefore, L rec By performing machine learning to reduce , the video extraction model 1 can be trained to increase the semantic relevance between the input sentence 10 and the estimated matching section.
[0046] L rank is composed of losses calculated by processing the learning task 16. In particular, L rank is configured to determine the magnitude of each loss. rank can be constructed as follows: m 0 ,m 1 ,m 2 ,m 3 is a constant that gives a certain margin.
[0047]
number
[0048] The first term makes it possible to train video extraction model 1 so that the first loss is smaller than the second loss. The second term makes it possible to train video extraction model 1 so that the second loss is smaller than the positive example sub-loss. The third term makes it possible to train video extraction model 1 so that the positive example sub-loss is smaller than the first negative example sub-loss. The fourth term makes it possible to train video extraction model 1 so that the first negative example sub-loss is smaller than the second negative example sub-loss.
[0049] L rank Since L contains each sub-loss for each sub-matching interval, rank Reducing L means increasing the semantic relevance of the estimated matching interval to the content of a part of the base sentence 11. rank can hierarchically assign the degree of relevance of the training video 21 to the base sentence 11 of the correct answer section, the base matching section, and each sub-matching section. This makes it possible to train the video extraction model 1 so that the magnitude relationship of the semantic relevance of each matching section becomes appropriate. Therefore, L rank By performing machine learning to reduce , the image extraction model 1 can be trained to pay attention to the content of a portion of the input sentence 10.
[0050] Furthermore, L rank In the above, the third or fourth term can be omitted. In this case, the computational cost in the learning can be improved by trading off the estimation accuracy of the matching section. In this case, the second term L p may be changed depending on the sub-sentence to be generated. For example, when generating only the first negative example sub-sentence, the second term L p L c It would be fine to say so.
[0051] As described above, the processor 110 performs machine learning. The machine learning may be performed by the backpropagation method. Fig. 6 is a diagram showing an example of the configuration of the video extraction model 1 during learning.
[0052] The sub-sentence generation unit 201 generates a sub-sentence from the base sentence 11. The learning task generation unit 202 generates a learning task 16 from the base sentence 11. The task processing execution unit 240 calculates each loss by processing the learning task 16 based on the feature amount of the training video 21 corresponding to each matching section. The loss function calculation unit 250 calculates the loss function L.
[0053] 3.Effects As described above, according to this embodiment, the video extraction model 1 can be trained so as to increase the semantic relevance between the input sentence 10 and the estimated matching section. Furthermore, the video extraction model 1 can be trained so as to increase the semantic relevance even when focusing on the content of a portion of the input sentence 10. This makes it possible to generate a video extraction model 1 with high generalization performance for various input sentences 10. [Explanation of symbols]
[0054] 1 Video extraction model, 10 Input sentence, 11 Base sentence, 12 Subsentences, 16 Learning Tasks, 20 Videos, 21 Training Videos, 100 computer, 110 processor, 120 storage device
Claims
1. A model generation method for generating a video extraction model that extracts a matching section that matches a content of an input sentence from a video, comprising: The method for generating a model is executed by a computer, The model generation method includes inputting a plurality of sentences to the video extraction model and extracting a plurality of matching segments for each of the plurality of sentences from a training video; The plurality of sentences includes a base sentence and at least one sub-sentence that is shorter than the base sentence; The at least one sub-sentence comprises: a positive example sub-sentence that includes a word included in the base sentence and does not include a noise word unrelated to the base sentence; a negative example sub-sentence including at least the noise word; At least one of the plurality of matching intervals include a base matching interval for the base sentence and at least one sub-matching interval for the at least one sub-sentence; The model generation method further comprises: Calculating a first loss by processing a learning task of reconstructing the base sentence based on the feature amount of the training video corresponding to the correct answer section; Calculating a second loss by processing the learning task based on the feature amount of the training video corresponding to the base matching section; Calculating at least one sub-loss by processing the learning task based on the features of the training video corresponding to the at least one sub-matching section; performing machine learning of the video extraction model such that the first loss is smaller than the second loss and the second loss is smaller than the at least one sub-loss; Includes Model generation method.
2. 2. The model generation method of claim 1, the at least one sub-sentence includes both the positive example sub-sentence and the negative example sub-sentence; The at least one sub-matching section includes a positive example sub-matching section for the positive example sub-sentence and a negative example sub-matching section for the negative example sub-sentence; The at least one sub-loss includes a positive example sub-loss calculated by processing the learning task based on features of the training video corresponding to the positive example sub-matching section, and a negative example sub-loss calculated by processing the learning task based on features of the training video corresponding to the negative example sub-matching section, The machine learning is further performed such that the positive example sub-loss is smaller than the negative example sub-loss. Model generation method.
3. 3. A model generation method according to claim 2, comprising: The negative example sub-sentence is a first negative example sub-sentence including both a word included in the base sentence and the noise word; a second negative example sub-sentence that does not include a word included in the base sentence and includes the noise word; Including, the negative example sub-matching section includes a first negative example sub-matching section for the first negative example sub-sentence and a second negative example sub-matching section for the second negative example sub-sentence; The negative example sub-loss includes a first negative example sub-loss calculated by processing the learning task based on a feature of the training video corresponding to the first negative example sub-matching section, and a second negative example sub-loss calculated by processing the learning task based on a feature of the training video corresponding to the second negative example sub-matching section, The machine learning is further performed such that the first negative example sub-loss is smaller than the second negative example sub-loss. Model generation method.
4. A model generation method according to any one of claims 1 to 3, comprising: The learning task is generated by masking some words from the base sentence, and the task is to infer the masked words. Model generation method.
5. A model generation system for generating a video extraction model that extracts a matching section that matches the content of an input sentence from a video, comprising: One or more processors; The one or more processors are configured to execute a process of inputting a plurality of sentences to the video extraction model and extracting a plurality of matching sections for each of the plurality of sentences from a training video; The plurality of sentences includes a base sentence and at least one sub-sentence that is shorter than the base sentence; The at least one sub-sentence comprises: a positive example sub-sentence that includes a word included in the base sentence and does not include a noise word unrelated to the base sentence; a negative example sub-sentence including at least the noise word; At least one of the plurality of matching intervals include a base matching interval for the base sentence and at least one sub-matching interval for the at least one sub-sentence; The one or more processors further include: A process of calculating a first loss by processing a learning task of reconstructing the base sentence based on the feature amount of the training video corresponding to the correct answer section; calculating a second loss by processing the learning task based on the feature amount of the training video corresponding to the base matching section; calculating at least one sub-loss by processing the learning task based on the feature of the training video corresponding to the at least one sub-matching section; performing machine learning of the video extraction model such that the first loss is smaller than the second loss and the second loss is smaller than the at least one sub-loss; It is configured to run Model generation system.
Citation Information
Patent Citations
Device and program for extracting video-image
JP2008141621A
Metal-organic framework (MOFs) nanoparticles coated with a fusion protein of Glutathione S-transferase and peptides targeting diseased cells and use thereof
KR102968421B1
Method and system for retrieving video segment by a sementic query
US20230083476A1
Device for forming action interval estimation model, method for forming action interval estimation model and program for forming action interval estimation model
JP2021189892A
Behavior recognition learning device, behavior recognition device, behavior recognition learning method, behavior recognition method, and behavior recognition learning program
JP2022072444A
