Information processing device, generation method, and generation program

Abstract

To improve accuracy of a video.SOLUTION: An information processing device stores a plurality of image data items. The information processing device classifies the plurality of image data items into a plurality of groups based on feature quantities of the plurality of image data items, and determines, for each group, whether or not the group is to be discarded, based on aggregation conditions of the feature quantities of the plurality of image data items belonging to the group. The information processing device generates teacher data using the plurality of image data items belonging to the remaining groups obtained by discarding the group to be discarded from the plurality of groups.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device and the like. [Background technology]

[0002] In recent years, the number of opportunities to watch promotional videos among video content has increased due to the spread of smartphones and tablet terminals, etc. For this reason, commercial stores and the like are creating promotional videos to promote their products (Public Relations), etc.

[0003] Here, when there are multiple stores to be promoted, it is a heavy burden for users to manually create videos (promotional videos), so video creation devices that automatically create videos are used.

[0004] Fig. 15 is a diagram for explaining a conventional motion generation device. As shown in Fig. 15, this motion generation device 10 has a still image DB (Data Base) 11, a model generation unit 12, a material analysis unit 13, and an image insertion unit 14.

[0005] The video generating device 10 uses web scraping or the like to collect multiple still image data related to the promotional video, and stores it in the still image DB 11. The video generating device 10 also extracts multiple still image data from video data posted from a user's terminal device or the like, and stores it in the still image DB 11. The video generating device 10 generates teacher data by assigning correct answer labels to the multiple still image data stored in the still image DB 11.

[0006] The model generation unit 12 executes machine learning using the teacher data stored in the still image DB 11 as input to generate a learning model. For example, by inputting still image data to the learning model, a classification class of the still image data is output. The classification class corresponds to the name of an object included in the still image data, and is used as a tag for the still image data. The model generation unit 12 outputs the learned learning model to the material analysis unit 13.

[0007] The material analysis unit 13 acquires video data posted from a user's terminal device, etc., and inputs a plurality of still image data included in the video data into a learning model in order to identify tags (classification classes) of the still image data. The material analysis unit 13 outputs the still image data with the tags to the image insertion unit 14.

[0008] The image insertion unit 14 generates promotional video data based on a template and a plurality of still image data to which tags have been added. The template defines the duration of the video and the type of tag for the still image data to be placed during that duration. The image insertion unit 14 places the still image data of the corresponding tag according to the template. For example, the image insertion unit 14 places a plurality of still image data to which the tag "store clerk" has been added during a time period corresponding to the tag "store clerk," thereby generating a video of the store clerk during that time period. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] JP 2017-111731 A [Patent Document 2] JP 2017-173098 A Summary of the Invention [Problem to be solved by the invention]

[0010] In conventional technology, still image data to be used as training data is collected using web scraping, etc., but the collected still image data may contain multiple copies of the same still image data or may contain still image data that is not suitable as training data.

[0011] If such still image data is included in the teacher data and a learning model is trained, over-learning or under-learning occurs, resulting in poor accuracy of the learning model. Also, if tagging is performed using such a learning model, appropriate tags cannot be assigned to the still image data, resulting in poor accuracy of the video.

[0012] It is possible for users to check each still image data collected using web scraping, etc., and remove unnecessary still image data, but this is not realistic because the amount of still image data to be checked is enormous.

[0013] According to one aspect, the present invention has an object to provide an information processing device, a generation method, and a generation program capable of improving the accuracy of moving images. [Means for solving the problem]

[0014] In a first proposal, the information processing device has a storage unit, a determination unit, and a generation unit. The storage unit stores a plurality of image data. The determination unit classifies the plurality of image data into a plurality of groups based on the feature amounts of the plurality of image data, and for each group, determines whether or not the group is to be discarded based on the degree of aggregation of the feature amounts of the plurality of image data belonging to the group. The generation unit generates teacher data using a plurality of image data belonging to the remaining groups after discarding the groups to be discarded from the plurality of groups. Effect of the Invention

[0015] The accuracy of the video can be improved. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram for explaining the process of the information processing device according to the present embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a system according to the present embodiment. [Diagram 3] FIG. 3 is a functional block diagram illustrating a configuration of an information processing device according to the present embodiment. [Figure 4] FIG. 4 is a diagram showing an example of the data structure of the still image DB. [Diagram 5] FIG. 5 is a diagram illustrating an example of a data structure of the teacher data table. [Figure 6] FIG. 6 is a diagram showing an example of the data structure of the posted video DB. [Figure 7] FIG. 7 is a diagram illustrating an example of a data structure of the analysis result table. [Figure 8] FIG. 8 is a diagram illustrating an example of the data structure of the template. [Figure 9] FIG. 9 is a diagram for explaining the process of the determination unit. [Figure 10] FIG. 10 is a flowchart illustrating a processing procedure of the information processing device according to the present embodiment. [Figure 11] FIG. 11 is a flowchart of the discarding process according to the present embodiment. [Figure 12] FIG. 12 is a diagram for explaining other processes of the information processing device. [Figure 13] FIG. 13 is a flowchart showing other processing procedures of the information processing device. [Figure 14] FIG. 14 is a diagram illustrating an example of a hardware configuration of a computer that realizes the same functions as the information processing apparatus of the embodiment. [Figure 15] FIG. 15 is a diagram for explaining a conventional motion generating device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, an information processing device, a generating method, and a generating program disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments. EXAMPLES

[0018] 1 is a diagram for explaining the process of the information processing device according to the present embodiment. For example, the process of the information processing device will be explained in the order of steps S10 to S12.

[0019] The process of step S10 executed by the information processing device will be described below. The information processing device executes web scraping using keywords and collects a plurality of still image data 20.

[0020] The process of step S11 executed by the information processing device will be described. The information processing device executes clustering (unsupervised classification) on the multiple still image data 20 based on the feature amount of the still image data. As a result, the multiple still image data 20 are classified into multiple groups. Here, it is assumed that the multiple still image data 20 are classified into groups Gr1 to Gr8. In the following description, any one of the multiple groups Gr1 to Gr8 will be referred to as "group Grn."

[0021] The process of step S12 executed by the information processing device will be described below. The information processing device selects a group Grn, and determines whether or not to discard the group Grn based on the degree of aggregation of the feature amount space of a plurality of still image data included in the selected group Grn.

[0022] When a plurality of still image data included in group Grn are distributed over a wide range or concentrated in a narrow range, the information processing device determines to discard group Grn.

[0023] The information processing device stores all still image data included in group Grn determined not to be discarded in the still image DB 141. The information processing device discards all still image data included in group Grn determined to be discarded.

[0024] The information processing device repeatedly executes the above process for the other groups. The information processing device generates training data using the plurality of still image data remaining in the still image DB 141, and executes machine learning of the learning model using the training data.

[0025] As described above, the information processing device according to the present embodiment clusters a plurality of still image data, and for each group, determines whether to discard the group based on the degree of aggregation of the feature amount space of the still image data. For example, if the still image data included in the group is distributed over a wide range, it can be said that the still image data included in the group is unnecessary still image data. Also, if the still image data included in the group is distributed over a narrow range, it can be said that the still image data included in the group is duplicated still image data.

[0026] Therefore, the information processing device can remove unnecessary and duplicated still image data by discarding still image data of a group based on the degree of aggregation of the feature amount space of the still image data from the still image DB 141. The information processing device generates teacher data using the still image data remaining in the still image DB 141 and executes machine learning of the learning model, thereby improving the accuracy of the learning model and improving the accuracy of the video using the learning model.

[0027] Fig. 2 is a diagram showing an example of a system according to the present embodiment. As shown in Fig. 2, the system includes a terminal device 30 and an information processing device 100. The terminal device 30 and the information processing device 100 are connected to each other via a network 50. Although Fig. 2 shows only the terminal device 30, the system may include other terminal devices.

[0028] The terminal device 30 is a terminal device operated by a user, and corresponds to a PC (Personal Computer), a notebook PC, a tablet terminal, a smartphone, etc. The user operates the terminal device 30 to transmit the posted video data that he / she has shot to the information processing device 100 and make a request to create a promotional video. The terminal device 30 may transmit a plurality of types of posted video data to the information processing device 100.

[0029] The information processing device 100 generates a learning model by executing the processes described in steps S10 to S12 in Fig. 1. When the information processing device 100 receives a request to create a promotional video from the terminal device 30, the information processing device 100 creates a promotional video using video data received from the terminal device 30.

[0030] 3 is a functional block diagram showing a configuration of an information processing device according to the present embodiment. As shown in FIG. 3, the information processing device 100 includes a communication unit 110, an input unit 120, a display unit 130, a storage unit 140, and a control unit 150.

[0031] The communication unit 110 transmits and receives information to and from the terminal device 30 and other external devices via the network 50. For example, the communication unit 110 is realized by a network interface card (NIC) or the like.

[0032] The input unit 120 is realized using input devices such as a keyboard and a mouse, and inputs various instruction information to the control unit 150 in response to an input operation by an operator. For example, an administrator of the information processing device 100 operates the input unit 120 to input various keywords to be used in web scraping.

[0033] The display unit 130 is a display device that displays information output from the control unit 150. The display unit 130 corresponds to an organic EL (Electro Luminescence) display, a liquid crystal display, a touch panel, or the like.

[0034] The storage unit 140 has a still image DB 141, a teacher data table 142, a learning model 143, a posted video DB 144, an analysis result table 145, templates 146, and video data 147. The storage unit 140 corresponds to a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as an HDD (Hard Disk Drive).

[0035] The still image DB 141 is a DB that stores still image data collected by web scraping or the like. FIG. 4 is a diagram showing an example of a data structure of the still image DB. As shown in FIG. 4, the still image DB 141 associates a keyword, identification information, and still image data. The keyword is a keyword specified when performing web scraping. The identification information is information that identifies the still image data. The still image data is still image data collected by web scraping.

[0036] 4 shows a case where still image data of identification information IM1-1 to IM1-n1 is collected by specifying a keyword "KW01 (for example, miso ramen)." It shows a case where still image data of identification information IM2-1 to IM2-n2 is collected by specifying a keyword "KW01 (for example, ramen bowl)."

[0037] The teacher data table 142 is a table that stores a teacher data set for executing machine learning of the learning model 143. FIG. 5 is a diagram showing an example of a data structure of the teacher data table. As shown in FIG. 5, the teacher data table 142 associates classification classes with still image data. The classification classes correspond to correct labels when executing machine learning. The still image data is still image data that has not been discarded by the process described in FIG. 1, among the still image data stored in the still image DB 141.

[0038] The learning model 143 is a learning model such as a Neural Network (NN) that is machine-trained based on a teacher data set stored in the teacher data table 142. By inputting still image data to the learning model 143, a classification class to which the input still image data belongs is output.

[0039] The posted video DB 144 is a DB that stores posted video data (time-series still image data) transmitted from the terminal device 30. FIG. 6 is a diagram showing an example of the data structure of the posted video DB. As shown in FIG. 6, the posted video DB 144 associates posted video identification information with posted video data. The posted video identification information is information that uniquely identifies the posted video data. The posted video data has time-series still image data.

[0040] The analysis result table 145 is a table that holds the analysis results of a plurality of still image data stored in the posted video DB 144. FIG. 7 is a diagram showing an example of the data structure of the analysis result table. As shown in FIG. 7, the analysis result table 145 associates posted video identification information, time, still image data, and tags. The posted video identification information is information that uniquely identifies the posted video data. The time is the time of the still image data. For example, if the start time of the posted video data is time t 1 The still image data is still image data included in the posted video data. The tag corresponds to a classification class output from the learning model 143 when the still image data is input to the learning model 143.

[0041] In template 146, a scenario is defined by a plurality of scenes, and video data 147 is generated based on the scenario defined in template 146. Fig. 8 is a diagram showing an example of a data structure of a template. In the example shown in Fig. 8, a store scene, a ramen scene, and a store clerk scene are set in this order in template 146.

[0042] For example, the start time of the store scene is "T1" and the end time is "T2", indicating that the tag corresponding to the store scene is "classification class C1". The start time of the ramen scene is "T2" and the end time is "T3", indicating that the tag corresponding to the ramen scene is "classification class C2". The start time of the store clerk scene is "T3" and the end time is "T4", indicating that the tag corresponding to the store scene is "classification class C3".

[0043] Returning to the explanation of Fig. 3, the video data 147 is video data generated in accordance with the scenario of the template 146.

[0044] The control unit 150 includes a collection unit 151, a determination unit 152, a generation unit 153, a learning unit 154, a reception unit 155, a material analysis unit 156, and an image insertion unit 157. The control unit 150 can be realized by a central processing unit (CPU) or a micro processing unit (MPU). The control unit 150 can also be realized by hardwired logic such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0045] The collection unit 151 receives a keyword from the input unit 120, and executes web scraping using the received keyword to collect still image data corresponding to the keyword from an external device on the network. The collection unit 151 associates the collected still image data with the keyword and stores it in the still image DB 141. The collection unit 151 assigns unique identification information to each still image data in the still image DB.

[0046] The administrator may set a plurality of keywords in the collection unit 151 in advance, and the collection unit 151 may execute web scraping using the keywords set in advance.

[0047] The determination unit 152 classifies still image data registered in the still image DB 141 into a plurality of groups based on the feature amount of the still image data. For each group, the determination unit 152 determines whether or not to discard the group based on the degree of aggregation of the feature amounts of a plurality of image data belonging to the group. For example, the feature amount of the still image data may be a feature amount of luminance, brightness, saturation, SIFT, HOG, or the like, or may be a feature amount of another image.

[0048] Fig. 9 is a diagram for explaining the process of the determination unit. For convenience of explanation, Fig. 9 uses a two-dimensional feature amount space 40 for explanation, but is not limited to this. The feature amount space 40 is a space with the horizontal axis representing the first feature amount and the vertical axis representing the second feature amount. Plots based on the feature amounts of still image data in the still image DB 141 are indicated by black circles.

[0049] The determination unit 152 classifies the still image data into a plurality of groups by clustering based on the feature amounts of the still image data in the still image DB 141. In the example shown in Fig. 9, the still image data are classified into groups Gr1, Gr2, and Gr3. The determination unit 152 may perform the clustering using k-means or the like.

[0050] The determination unit 152 selects a group Grn, and determines whether or not to discard the group Grn based on the degree of aggregation of the feature amount space 40 of the multiple still image data included in the selected group Grn. The determination unit 152 calculates the degree of aggregation based on the distance between the center of gravity of the group Grn and each feature amount of the group Grn. In Fig. 9, the centers of gravity of the groups Gr1 to Gr3 are indicated by white circles.

[0051] For example, the determination unit 152 calculates the degree of aggregation S based on the formula (1). Here, as an example, the feature amount space is assumed to be two-dimensional. The coordinates of the center of gravity are (X, Y), and the coordinates of the i-th still image data of the group Grn are (x, Y). i ,y i ) The number of still image data included in group Grn is assumed to be n.

[0052]

number

[0053] The determination unit 152 also calculates a first threshold value Th1 and a second threshold value Th2. The determination unit 152 calculates the distance between the coordinates of the center of gravity and the coordinates of each still image data of the group Grn, and specifies the maximum value of the calculated distances. The determination unit 152 sets the value obtained by dividing the specified maximum value by Na as the first threshold value Th1. Na is a preset value, and may be 2, 3, 4, or another numerical value.

[0054] The determination unit 152 calculates the distance between the coordinates of the center of gravity and the coordinates of each still image data of the group Grn, and specifies the minimum value of the calculated distances. The determination unit 152 sets the value obtained by multiplying the specified minimum value by Nb as the first threshold value Th2. Nb is a preset value, and may be 2, 3, 4, or another value.

[0055] When the degree of aggregation S is equal to or greater than the first threshold Th1, the determination unit 152 determines that the still image data belonging to the group Grn are widely dispersed and determines to discard the group Grn. For example, since the degree of aggregation S of the group Gr1 in FIG. 9 is equal to or greater than the first threshold Th1, the determination unit 152 determines to discard the group Gr1.

[0056] When the degree of aggregation S is less than the second threshold Th2, the determination unit 152 determines that the still image data belonging to the group Grn are concentrated in a small range and determines to delete the group Grn. For example, since the degree of aggregation S of the group Gr3 in Fig. 9 is less than the first threshold Th2, the determination unit 152 determines to discard the group Gr3.

[0057] On the other hand, the aggregation degree S of the group Gr2 is less than the first threshold value Th1 and is equal to or greater than the second threshold value Th2, so that the determination unit 152 determines not to discard the group Gr2.

[0058] The determination unit 152 repeatedly executes the above process for each classified group, determines which groups are to be discarded and which groups are not to be discarded, and outputs the determination result to the generation unit 153. For example, the determination result output by the determination unit 152 to the generation unit 153 includes identification information of the still image data included in the group to be discarded.

[0059] The generation unit 153 generates the teacher data table 142 based on the determination result of the determination unit 152 and the still image DB 141. The generation unit 153 deletes the still image data included in the group to be discarded from the still image DB 141 based on the determination result of the determination unit 152, and acquires pairs of the still image data remaining in the still image DB 141 and the keywords of this still image data.

[0060] The generating unit 153 converts the keywords into general-purpose classification classes (tags) using a tag conversion table (not shown). The tag conversion table is a table that associates keywords with classification classes corresponding to the keywords. The generating unit 153 associates the still image data with the converted classification classes and registers them in the teacher data table 142.

[0061] The generating unit 153 repeatedly executes the above process for the pairs of still image data and keywords remaining in the still image DB 141.

[0062] The learning unit 154 executes learning (machine learning by backpropagation) of the learning model 143 using the teacher data set stored in the teacher data table 142 as an input. For example, the learning unit 154 adjusts the parameters of the learning model 143 so that the output when the still image data (input data) in the teacher data table 142 is input to the learning model 143 approaches the corresponding classification class.

[0063] The reception unit 155 receives a request to create a video from the terminal device 30. The reception unit 155 receives posted video data from the terminal device 30, and stores the received posted video data in the posted video DB 144. When storing the posted video data in the posted video DB 144, the reception unit 155 assigns unique posted video identification information to the posted video data.

[0064] The material analysis unit 156 determines the tag (classification class) of the still image data by inputting the time-series still image data included in the posted video data stored in the posted video DB 144 into the learning model 143. The material analysis unit 156 associates the posted video identification information, the time of the still image data (or the frame number), the still image data, and the tag, and stores them in the analysis result table 145.

[0065] The material analysis unit 156 repeatedly executes the above process on the time-series still image data included in the posted video data stored in the posted video DB 144.

[0066] The image insertion unit 157 generates video data based on the analysis result table 145 and the template 146. For example, the image insertion unit 157 identifies the tag, start time, and end time of each scenario in the template 146. The image insertion unit 157 calculates the scenario time from the start time to the end time. The image insertion unit 157 refers to the analysis result table 145, and acquires time-series still image data for the scenario time from the still image data corresponding to the scenario tag, and sets the acquired time-series still image data in the video data 147.

[0067] The image insertion unit 157 repeatedly executes the above process for each scenario of the template 146 to generate the video data 147.

[0068] Next, an example of a processing procedure of the information processing device 100 according to this embodiment will be described. Fig. 10 is a flowchart showing the processing procedure of the information processing device according to this embodiment. As shown in Fig. 10, the collection unit 151 of the information processing device 100 collects still image data and stores it in the still image DB 141 (step S101).

[0069] The determination unit 152 of the information processing device 100 performs clustering based on the feature amounts of the still image data in the still image DB 141, and classifies the data into a plurality of groups (step S102). The determination unit 152 and the generation unit 153 of the information processing device 100 perform a discard process (step S103).

[0070] The generation unit 153 generates a teacher data set based on the still image data remaining in the still image DB, and stores the teacher data set in the teacher data table (step S104). The learning unit 154 of the information processing device 100 learns the learning model 143 based on the teacher data table 142 (step S105).

[0071] Next, an example of the procedure of the discarding process shown in step S103 of Fig. 10 will be described. Fig. 11 is a flowchart showing the procedure of the discarding process according to this embodiment. As shown in Fig. 11, the determination unit 152 of the information processing device 100 selects an unselected group (step S201).

[0072] The determination unit 152 calculates the center of gravity of the feature amounts in the group (step S202). The determination unit 152 calculates the distance between the feature amount of each still image data and the center of gravity in the feature amount space (step S203). The determination unit 152 identifies the maximum distance, the minimum distance, and the degree of accumulation (step S204).

[0073] The determination unit 152 determines whether the degree of accumulation is 1 / Na times or more of the maximum value (step S205). If the degree of accumulation is 1 / Na times or more of the maximum value (step S205, Yes), the generation unit 153 of the information processing device discards the still image data of the selected group from the still image DB 141 (step S207) and proceeds to step S208.

[0074] If the accumulation degree is not 1 / Na times the maximum value or more (step S205, No), the determination unit 152 determines whether the accumulation degree is less than Nb times the minimum value (step S206). If the accumulation degree is less than Nb times the minimum value (step S206, Yes), the determination unit 152 proceeds to step S207.

[0075] On the other hand, if the accumulation level is not less than Nb times the minimum value (step S206, No), the determination unit 152 proceeds to step S208. If all groups have not been selected (step S208, No), the determination unit 152 proceeds to step S201. If all groups have been selected (step S208, Yes), the determination unit 152 ends the discarding process.

[0076] Next, the effect of the information processing device 100 according to the present embodiment will be described. The information processing device 100 clusters a plurality of still image data, and for each group, determines whether or not to discard the group based on the degree of aggregation of the feature amount space of the still image data. For example, if the still image data included in the group is distributed over a wide range, it can be said that the still image data included in the group is unnecessary still image data. Also, if the still image data included in the group is distributed over a narrow range, it can be said that the still image data included in the group is duplicated still image data.

[0077] Therefore, the information processing device 100 can remove unnecessary and duplicated still image data by discarding still image data of a group based on the degree of aggregation of the feature amount space of the still image data from the still image DB 141. The information processing device 100 generates teacher data using the still image data remaining in the still image DB 141 and executes machine learning of the learning model, thereby improving the accuracy of the learning model and improving the accuracy of the video using the learning model.

[0078] The information processing device 100 calculates the center of gravity of the feature amounts of the multiple image data belonging to the same group, and calculates the degree of aggregation based on the average value of the difference between the center of gravity and the feature amounts of the multiple image data belonging to the same group. This makes it possible to quantitatively identify the degree of aggregation of the multiple still image data included in each group.

[0079] The information processing device 100 calculates a first threshold value Th1 and a second threshold value Th2, and identifies a group to be discarded by comparing the degree of aggregation with the first threshold value Th1 and the second threshold value Th2. This makes it possible to remove unnecessary still image data and duplicated still image data.

[0080] The information processing device 100 removes unnecessary and duplicated still image data from the still image DB 141, and generates a teacher data table 142 based on the remaining still image data. Then, the information processing device 100 learns a learning model based on the teacher data table 142. This prevents overlearning and underlearning of the learning model 143, and improves the accuracy of the learning model 143. By improving the accuracy of the learning model 143, the accuracy of the video data using the learning model 143 can also be improved.

[0081] Incidentally, the above-mentioned information processing device 100 performs clustering of a plurality of still image data, and determines whether or not to discard a group for each group based on the degree of aggregation of the feature amount space of the still image data, but is not limited to this. Other processes of the information processing device 100 will be described below.

[0082] 12 is a diagram for explaining other processes of the information processing device 100. For example, the other processes of the information processing device 100 will be explained in the order of steps S20 to S22.

[0083] The process of step S20 executed by the information processing device 100 will be described. The information processing device 100 executes web scraping using keywords to collect a plurality of still image data 20a. The information processing device 100 also acquires a plurality of still image data photographed by the user himself. The still image data photographed by the user himself is expressed as a plurality of local image data 20b. The local image data 20b can be said to be still image data that well represents the characteristics of the still image data that the user wants to include in the video data.

[0084] The process of step S21 executed by the information processing device 100 will be described. The determination unit 152 of the information processing device 100 executes clustering (unsupervised classification) on the multiple still image data 20a and multiple local image data 20b based on the feature amount of the still image data. As a result, the multiple still image data 20a and multiple local image data 20b are classified into multiple groups. Here, it is assumed that the multiple still image data 20a and multiple local image data 20b are classified into groups Gr1 to Gr8. In the following description, any one of the multiple groups Gr1 to Gr8 will be referred to as "group Grn."

[0085] The process of step S21 executed by the information processing device 100 will be described. The determination unit 152 of the information processing device 100 selects a group Grn, and determines whether or not to discard the group Grn depending on whether or not the selected group Grn includes local image data. The determination unit 152 selects a group Grn, and discards the Grn if the selected group Grn does not include local image data. On the other hand, the determination unit 152 selects a group Grn, and does not discard the Grn if the selected group Grn includes local image data.

[0086] The information processing device 100 repeatedly executes the above process for the other groups. The information processing device 100 generates a teacher data table 142 using the multiple still image data remaining in the still image DB 141, and executes machine learning of a learning model 143 using the teacher data table 142.

[0087] Here, a description will be given of the procedure of other processes executed by the information processing device 100. Fig. 13 is a flowchart showing the procedure of other processes of the information processing device. The process shown in Fig. 13 is a process that replaces the discard process shown in Fig. 11.

[0088] As shown in FIG. 13, the determination unit 152 of the information processing device 100 selects an unselected group (step S301).

[0089] The determination unit 152 determines whether or not the group includes local image data (step S302). If the group does not include local image data (step S303, No), the determination unit 152 discards the still image data of the selected group from the still image DB 141 (step S304) and proceeds to step S305.

[0090] On the other hand, if the group includes local image data (step S303, Yes), the determination unit 152 determines whether or not all groups have been selected (step S305). If all groups have not been selected (step S305, No), the determination unit 152 proceeds to step S301. If all groups have been selected (step S305, Yes), the determination unit 152 ends the process.

[0091] As described above, according to the information processing device 100, a group Grn is selected, and depending on whether local image data is included in the selected group Grn, it is determined whether to discard the group Grn. This allows the still image data belonging to the same group as the local image data captured by the user to be retained.

[0092] Note that the information processing device 100 according to the present embodiment may determine a group to be discarded by combining the process described in Fig. 11 and the process described in Fig. 13. For example, the information processing device 100 may execute the process shown in Fig. 13 to narrow down a plurality of groups to groups that include local image data, and determine which groups to discard from the narrowed down groups based on the degree of gathering.

[0093] Next, an example of a hardware configuration of a computer that realizes the same functions as the information processing device 100 described in the above embodiment will be described. Fig. 14 is a diagram showing an example of a hardware configuration of a computer that realizes the same functions as the information processing device of the embodiment.

[0094] 14, the computer 200 has a CPU 201 that executes various arithmetic processes, an input device 202 that accepts data input from a user, and a display 203. The computer 200 also has a reading device 204 that reads programs and the like from a storage medium, and a communication device 205 that transmits and receives data to and from the terminal device 30 and other external devices via a wired or wireless network. The computer 200 also has a RAM 206 that temporarily stores various information, and a hard disk device 207. The devices 201 to 207 are connected to a bus 208.

[0095] The hard disk device 207 has a collection program 207a, a determination program 207b, a generation program 207c, a learning program 207d, a reception program 207e, a material analysis program 207f, and an image insertion program 207f. The CPU 201 reads out each of the programs 207a to 207f and develops them in the RAM 206.

[0096] The collection program 207a functions as a collection process 206a. The judgment program 207b functions as a judgment process 206b. The generation program 207c functions as a generation process 206c. The learning program 207d functions as a learning process 206d. The reception program 207e functions as a reception process 206e. The material analysis program 207f functions as a material analysis process 206f. The image insertion program 207g functions as an image insertion process 206g.

[0097] The processing of the collection process 206a corresponds to the processing of the collection unit 151. The processing of the judgment process 206b corresponds to the processing of the judgment unit 152. The processing of the generation process 206c corresponds to the processing of the generation unit 153. The processing of the learning process 206d corresponds to the processing of the learning unit 154. The processing of the reception process 206 corresponds to the processing of the reception unit 155. The processing of the material analysis process 206 corresponds to the processing of the material analysis unit 156. The processing of the image insertion process 206g corresponds to the processing of the image insertion unit 157.

[0098] It should be noted that each of the programs 207a to 207g does not necessarily have to be stored in the hard disk device 207 from the beginning. For example, each of the programs may be stored in a "portable physical medium" such as a flexible disk (FD), CD-ROM, DVD, magneto-optical disk, or IC card that is inserted into the computer 200. Then, the computer 200 may read and execute each of the programs 207a to 207g.

[0099] The following supplementary notes are further disclosed regarding the embodiments including the above examples.

[0100] (Supplementary Note 1) A storage unit for storing a plurality of image data; a determination unit that classifies the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data, and determines for each group whether or not the group is to be discarded based on a degree of aggregation of the feature amounts of the plurality of image data belonging to the group; a generating unit that generates training data using a plurality of image data belonging to remaining groups after discarding groups to be discarded from the plurality of groups; 13. An information processing device comprising:

[0101] (Appendix 2) The information processing device according to appendix 1, wherein the determination unit calculates a center of gravity of features of multiple image data belonging to the same group, and calculates the degree of aggregation based on a difference between the center of gravity and the features of the multiple image data belonging to the same group.

[0102] (Supplementary Note 3) The information processing device according to Supplementary Note 2, characterized in that the determination unit determines that the group is to be discarded when the degree of aggregation is equal to or greater than a first threshold or is less than a second threshold that is smaller than the first threshold.

[0103] (Supplementary Note 4) The information processing device according to Supplementary Note 1, characterized in that the plurality of image data stored in the memory unit includes a pre-specified local image, and the determination unit further executes a process of determining, among each group, a group that does not include the local image as a group to be discarded.

[0104] (Supplementary Note 5) The information processing device according to any one of Supplementary Notes 1 to 4, further comprising a learning unit that executes machine learning of a learning model based on the teacher data.

[0105] (Appendix 6) A computer-implemented generation method, comprising: storing a plurality of image data in a storage device; classifying the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data; For each group, it is determined whether or not the group is to be discarded based on the degree of aggregation of the features of the multiple image data belonging to the group; Generate training data using a plurality of image data belonging to the remaining groups after discarding the group to be discarded from the plurality of groups. A generating method comprising:

[0106] (Appendix 7) The generation method described in Appendix 6, characterized in that the determining process calculates the center of gravity of features of multiple image data belonging to the same group, and calculates the degree of aggregation based on the difference between the center of gravity and the features of the multiple image data belonging to the same group.

[0107] (Appendix 8) The generation method described in Appendix 7, characterized in that the determination process determines that the group is to be discarded if the degree of aggregation is greater than or equal to a first threshold or less than a second threshold that is smaller than the first threshold.

[0108] (Appendix 9) The generating method described in Appendix 6, characterized in that the multiple image data stored in the storage device include a pre-specified local image, and the determining process further includes a process of determining, among each group, a group that does not include the local image as a group to be discarded.

[0109] (Supplementary Note 10) The generation method according to any one of Supplementary Notes 6 to 9, further comprising a process of executing machine learning of a learning model based on the training data.

[0110] (Appendix 11) To the computer, storing a plurality of image data in a storage device; classifying the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data; For each group, it is determined whether or not the group is to be discarded based on the degree of aggregation of the features of the multiple image data belonging to the group; Generate training data using a plurality of image data belonging to the remaining groups after discarding the group to be discarded from the plurality of groups. A generating program for executing a process.

[0111] (Appendix 12) The generation program according to appendix 11, characterized in that the determining process calculates the center of gravity of features of multiple image data belonging to the same group, and calculates the degree of aggregation based on the difference between the center of gravity and the features of the multiple image data belonging to the same group.

[0112] (Appendix 13) The generation program according to appendix 12, characterized in that the determination process determines that the group is to be discarded when the degree of aggregation is greater than or equal to a first threshold or less than a second threshold that is smaller than the first threshold.

[0113] (Appendix 14) The generation program according to appendix 11, characterized in that the plurality of image data stored in the storage device include a pre-specified local image, and the determining process further executes a process of determining, among each group, a group that does not include the local image as a group to be discarded.

[0114] (Supplementary Note 15) The generation program according to any one of Supplementary Notes 11 to 14, further comprising a process of executing machine learning of a learning model based on the teacher data. [Explanation of symbols]

[0115] 100 Information processing device 110 Communications Department 120 Input section 130 Display section 140 Storage section 141 Still image DB 142 Teacher Data Table 143 Learning Model 144 Posted Video DB 145 Analysis Results Table 146 Templates 147 Video Data 150 Control section 151 Collection Department 152 Judgment section 153 Generation part 154 Learning Department 155 Reception 156 Material Analysis Department 157 Image Insertion Section

Claims

1. A storage unit that stores a plurality of image data; a determination unit that classifies the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data, calculates a degree of aggregation for each group based on feature amounts of the plurality of image data belonging to the group, and determines that the group is to be discarded when the degree of aggregation is equal to or greater than a first threshold or less than a second threshold that is smaller than the first threshold; a generating unit that generates training data using a plurality of image data belonging to remaining groups after discarding groups to be discarded from the plurality of groups; 13. An information processing device comprising:

2. The information processing device according to claim 1 , characterized in that the determination unit calculates a center of gravity of features of multiple image data belonging to the same group, and calculates the degree of aggregation based on a difference between the center of gravity and the features of the multiple image data belonging to the same group.

3. 2. The information processing device according to claim 1, wherein the plurality of image data stored in the storage unit includes a pre-specified local image, and the determination unit further executes a process of determining, among each group, a group that does not include the local image as a group to be discarded.

4. 4. The information processing device according to claim 1, further comprising a learning unit that executes machine learning of a learning model based on the teacher data.

5. 1. A computer-implemented generation method comprising: storing a plurality of image data in a storage device; classifying the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data; For each classified group, a degree of aggregation is calculated based on feature amounts of the multiple image data belonging to the group, and if the degree of aggregation is equal to or greater than a first threshold or less than a second threshold that is smaller than the first threshold, the group is determined to be discarded; Generate training data using a plurality of image data belonging to the remaining groups after discarding the group to be discarded from the plurality of groups. A generating method comprising:

6. On the computer, storing a plurality of image data in a storage device; classifying the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data; For each classified group, a degree of aggregation is calculated based on feature amounts of the multiple image data belonging to the group, and if the degree of aggregation is equal to or greater than a first threshold or less than a second threshold that is smaller than the first threshold, the group is determined to be discarded; Generate training data using a plurality of image data belonging to the remaining groups after discarding the group to be discarded from the plurality of groups. A generating program for executing a process.

7. A storage unit for storing a plurality of image data; a determination unit that classifies the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data, calculates a degree of aggregation for each group based on feature amounts of the plurality of image data belonging to the group, and identifies, from among the plurality of groups, a group whose calculated degree of aggregation is less than a first threshold and equal to or greater than a second threshold that is smaller than the first threshold; A generation unit that generates teacher data using a plurality of image data belonging to the identified group; 13. An information processing device comprising:

8. A computer-implemented generation method comprising: storing a plurality of image data in a storage device; classifying the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data; For each classified group, a degree of aggregation is calculated based on the feature amounts of the multiple image data belonging to the group, and a group is identified in which the degree of aggregation is less than a first threshold and is equal to or greater than a second threshold that is smaller than the first threshold; The training data is generated using the plurality of image data belonging to the identified group. A generating method comprising:

9. A computer comprising: storing a plurality of image data in a storage device; classifying the plurality of image data into a plurality of groups based on feature amounts of the plurality of image data; For each classified group, a degree of aggregation is calculated based on the feature amounts of the multiple image data belonging to the group, and a group is identified in which the degree of aggregation is less than a first threshold and is equal to or greater than a second threshold that is smaller than the first threshold; The training data is generated using the plurality of image data belonging to the identified group. A generating program for executing a process.

Images