Product generation device, product generation method, and program

Abstract

Provided is a product generation device comprising: an intermediate product generation unit that generates an intermediate product using a predetermined intermediate product generation model; a property designation unit that designates a correction property for correcting the intermediate product; and a correction product generation unit that generates a correction product in which the intermediate product is corrected on the basis of the designated correction property.

Description

Product generation device, product generation method, and program

[0001] The present invention relates to a product generation device, a product generation method, and a program.

[0002] A component inspection device that generates defective product images using a machine learning model and then inspects components is known (see, for example, Patent Document 1). [Prior Art Literature] [Patent Document] Patent Document 1: JP 2022-108855 A

[0003] Products generated using machine learning models may not fully reflect user intent. General disclosure

[0004] A first aspect of the present invention provides a product generation device comprising: an intermediate product generation unit that generates an intermediate product using a predetermined intermediate product generation model; a property designation unit that designates a modification property for modifying the intermediate product; and a modified product generation unit that generates a modified product by modifying the intermediate product based on the designated modification property.

[0005] In the above product generation device, the modification property may include an entire property for modifying the entire intermediate product.

[0006] In any of the above product generation devices, the modification property may include a part property for modifying a part of the intermediate product.

[0007] In any of the above product generation devices, the modified properties may include destination information regarding a destination to which the modified product is to be provided.

[0008] Any of the above product generation devices may include a modification comparison unit that compares the intermediate product with the modified product. Any of the above product generation devices may include a comparison result output unit that outputs a comparison result between the intermediate product and the modified product.

[0009] Any of the above product generation devices may include a match degree determination unit that determines a match degree between a prompt input to the intermediate product generation model to generate the intermediate product and the generated intermediate product. Any of the above product generation devices may include a modification suggestion unit that suggests the modification property according to the match degree.

[0010] Any of the above product generation devices may include a correction portion designation unit that designates a correction portion of the intermediate product that should be corrected by the corrected product generation unit, and / or designates a non-correction portion that should not be corrected by the corrected product generation unit.

[0011] Any of the above product generation devices may include an unnecessary portion designation unit that designates an unnecessary portion of the intermediate product that should be deleted in the corrected product.

[0012] Any of the above product generation devices may include a modification data acquisition unit that acquires at least one of structured data and unstructured data as modification data for generating the modified product, and the modified product generation unit may generate the modified product based on at least one of the structured data and the unstructured data.

[0013] In any of the above product generation devices, the modified product generation unit may include a determination unit that determines, based on predetermined criteria, whether to reflect an automatic modification item not specified in the modification property in the modified product.In any of the above product generation devices, the modified product generation unit may include an automatic reflection unit that reflects the automatic modification item in the modified product when the determination unit determines to reflect the automatic modification item.

[0014] In any of the above product generation devices, the modified product generation section may include a modification degree adjustment section for adjusting the degree of modification made by the automatic reflection section.

[0015] Any of the above product generation devices may include an initial product acquisition unit that acquires an initial product to be input to the intermediate product generation unit. The intermediate product generation unit may generate the intermediate product including an initial product remaining portion where the initial product remains and a generated portion generated by the intermediate product generation model.

[0016] In any of the above product generation devices, the property designation unit may designate different modification properties for the remaining portion of the initial product and the portion to be generated, and the modified product generation unit may generate the modified product reflecting modifications that differ between the remaining portion of the initial product and the portion to be generated.

[0017] Any of the above product generation devices may include a user input information acquisition unit that acquires user input information input by a user, and the modified product generation unit may generate the modified product based on the modified properties and the user input information.

[0018] In any of the above product generation devices, the user input information may include location designation information regarding locations to be modified by the modified product generation unit and / or locations not to be modified by the modified product generation unit. Any of the above product generation devices may include a modification location adjustment unit that adjusts the locations to be modified and / or the locations not to be modified based on the location designation information.

[0019] Any of the above product generation devices may include a template information acquisition unit that acquires template information that specifies a predetermined template, and the modified product generation unit may generate the modified product based on the modified property and the template information.

[0020] In any of the above product generation devices, the modified product generation unit may include a generation model selection unit for selecting a modified product generation model to be used for generating the modified product based on the modified property.

[0021] In a second aspect of the present invention, there is provided a product generation method including the steps of: a computer generating an intermediate product using a predetermined intermediate product generation model; a computer specifying modification properties for modifying the intermediate product; and a computer generating a modified product by modifying the intermediate product based on the specified modification properties.

[0022] In a third aspect of the present invention, there is provided a program that, when executed by a computer, causes the computer to generate an intermediate product using a predetermined intermediate product generation model, cause the computer to specify modification properties for modifying the intermediate product, and generate a modified product by modifying the intermediate product based on the specified modification properties.

[0023] The above summary of the invention does not list all of the features of the present invention, and subcombinations of these features may also be inventions.

[0024] 1 shows an overview of the configuration of the product generation device 100. 1 shows an example of an operational flowchart of the product generation device 100. 1 shows an example of a method for generating a modified product Pm. 1 shows a specific example of a modified property Prm. 1 shows a modified example of the product generation device 100. 1 shows a modified example of the product generation device 100. 1 shows an example of a method for generating a modified product Pm. 1 shows an example of a method for generating a modified product Pm. 1 shows a modified example of the product generation device 100. 1 shows an example of a method for generating a modified product Pm. 1 shows a modified example of the product generation device 100. 1 shows a modified example of the product generation device 100. 1 shows a diagram for explaining a method for generating a modified product Pm. 1 shows a modified example of the product generation device 100. 1 shows an example of a method for generating a modified product Pm. 1 shows an example of a method for adjusting a modified portion Rm. 1 shows a modified example of the product generation device 100. 1 shows a modified example of the product generation device 100. 1 shows an example of a method for generating a modified product Pm. 1 shows an example of a method for generating a modified product Pm. 22 illustrates an example of a method for generating a modified product Pm. 22 illustrates an example of a computer 2200 in which aspects of the present invention may be embodied in whole or in part.

[0025] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention as claimed. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0026] 1A shows an overview of the configuration of a product generation device 100. The product generation device 100 of this example includes an intermediate product generation unit 20, a property specification unit 30, a modified product generation unit 40, an information output unit 50, and a modification adjustment unit 60. The product generation device 100 may include a storage unit 55.

[0027] The intermediate product generation unit 20 generates an intermediate product Pi using a predetermined intermediate product generation model 25. The intermediate product Pi may be any product generated by a user using the intermediate product generation model 25. The intermediate product Pi may be generated by the user inputting any prompt so that a product according to the user's request is generated. The intermediate product Pi may be generated by inputting the same type of product to the intermediate product generation model 25. The intermediate product generation unit 20 of this example has an intermediate product generation model 25.

[0028] The intermediate product generation model 25 generates the intermediate product Pi. The intermediate product generation model 25 may include at least one of a generative model, a machine learning model including a multimodal model, or a simulation that generates data. The intermediate product generation model 25 may be provided outside the product generation device 100.

[0029] In this specification, a product may be anything that can be generated using a generative model, and may include at least one of images, audio, music, text, three-dimensional drawing data, video, movies, and time-series data.

[0030] The type of prompt for generating the intermediate product Pi may be text, image, sound, music, sentence, three-dimensional drawing data, video, movie, time series data, or other products. The prompt input for generating the intermediate product Pi may be in the form of a dialogue between a user and a large-scale language model (LLM). The type of prompt for generating the intermediate product Pi may be the same as or different from the intermediate product Pi.

[0031] The property specification unit 30 specifies a modified property Prm for modifying the intermediate product Pi. The property specification unit 30 may specify the modified property Prm in accordance with a user instruction. The property specification unit 30 transmits the specified modified property Prm to the modified product generation unit 40.

[0032] The modification property Prm may be any property that can specify the user's intention, such as text, image, sound, music, sentence, three-dimensional drawing data, video, movie, time-series data, or other formats. The modification property Prm may be in the form of an interaction between the user and a large-scale language model. The modification property Prm may be input by the user, or the user may select one proposed by the product generation device 100. Specific examples of the modification property Prm will be described later.

[0033] The modified product generation unit 40 generates a modified product Pm by modifying the intermediate product Pi based on the specified modified property Prm. By generating the modified product Pm based on the modified property Prm, the modified product generation unit 40 can generate a modified product Pm that reflects the user's intention in the intermediate product Pi. The modified product generation unit 40 may generate a plurality of modified products Pm. The modified product generation unit 40 may generate a plurality of modified products Pm from the same modified property Prm, or may generate a plurality of modified products Pm from different modified properties Prm. The modified product generation unit 40 of this example has a modified product generation model 45.

[0034] The modified product generation model 45 generates a modified product Pm from the intermediate product Pi based on the modified property Prm. The modified product generation model 45 may generate the modified product Pm so that information from the modification adjustment unit 60 is reflected. The modified product generation model 45 may include at least one of a generative model, a machine learning model including a multimodal model, or a simulation that generates data. The modified product generation model 45 may be provided outside the product generation device 100.

[0035] The modified product generation model 45 may be the same generation model as the intermediate product generation model 25, or may be a different generation model. The modified product generation model 45 may be a generation model that is trained completely independently of the intermediate product generation model 25. That is, the modified product generation model 45 may be trained using a training data set that is completely different from that of the intermediate product generation model 25. The modified product generation model 45 may also be trained using a training data set that is partially common to that of the intermediate product generation model 25.

[0036] The information output unit 50 outputs the modified product Pm to the outside of the product generation device 100. The information output unit 50 may transmit data of the modified product Pm to display the modified product Pm on a display unit such as a monitor. The information output unit 50 may transmit data of the modified product Pm to the outside of the product generation device 100. The information output unit 50 may transmit the modified product Pm to the storage unit 55.

[0037] When displaying the modified product Pm, the information output unit 50 may display a plurality of modified products Pm side by side. The information output unit 50 may display the modified property Prm and the modified product Pm in association with each other, allowing the user to select a modified product Pm that meets the user's intention. The information output unit 50 may display the intermediate product Pi and the modified product Pm side by side.

[0038] The storage unit 55 may store the modified product Pm generated by the modified product generation unit 40. The storage unit 55 may store the modified product Pm and the modified property Prm used to generate the modified product Pm in association with each other. The storage unit 55 may store the modified product Pm and the intermediate product Pi that was the basis of the modified product Pm in association with each other. The storage unit 55 may store the modified product Pm and the initial product Ps, which will be described later, in association with each other.

[0039] The modification adjustment unit 60 adjusts the modified product Pm generated by the modified product generation unit 40. In one example, the modification adjustment unit 60 may adjust the modified product Pm to be generated by providing additional information different from the modified property Prm to the modified product generation unit 40. A specific example of the modification adjustment unit 60 will be described later.

[0040] The product generation device 100 of this example can generate a modified product Pm that reflects a user's intention in the intermediate product Pi generated by the intermediate product generation model 25. As a result, even if the user's intention is not sufficiently reflected in the intermediate product Pi, the product generation device 100 can provide the user with a modified product Pm that has been modified in accordance with the user's intention.

[0041] The product generation device 100 of this example may display both the modified product Pm and the intermediate product Pi before modification. This allows the user to compare the products before and after modification and easily determine whether the product has been modified to meet the user's intentions. For example, the user can easily confirm that the issues in the intermediate product Pi have been resolved. The product generation device 100 may also display additional information to the modified product Pm. For example, the product generation device 100 displays the modified property Prm as additional information to the modified product Pm.

[0042] 1B shows an example of an operational flowchart of the product generation device 100. In step S100, an intermediate product Pi is generated. In step S102, a modified property Prm is specified. In step S104, a modified product Pm is generated.

[0043] Steps S100 to S104 may be repeated in any order. After generating a plurality of intermediate products Pi in step S100, a modified product Pm may be generated by specifying a modified property Prm for each intermediate product Pi. Alternatively, after generating a modified product Pm according to the intermediate product Pi in step S104, the process may return to step S100 to generate a new intermediate product Pi and specify a different modified property Prm to generate the modified product Pm.

[0044] 1C shows an example of a method for generating a modified product Pm. The intermediate product Pi in this example is an image in which an apple is drawn on a background of style A. The modification property Prm indicates that the style of the background is to be changed to style B. The modified product generation model 45 generates a modified product Pm by modifying the intermediate product Pi based on the modification property Prm. The modified product generation model 45 generates an image in which an apple is drawn on a background of style B as the modified product Pm.

[0045] The product generation device 100 of this example can generate a modified product Pm that conforms to the user's intention while reducing the user's effort by providing the user's intention to modify the modified product generation model 45 and having the modified product generation model 45 make modifications, rather than having the user modify all of the generated intermediate products Pi. The product generation device 100 of this example can save the user the effort of regenerating the generated intermediate products Pi from scratch or having the user manually add or modify the products.

[0046] The product creation device 100 of this example can accurately reflect the user's intentions, thereby reducing the time and man-hours required for product refinement. The product creation device 100 allows the user to freely communicate their intentions, allowing them to freely realize their desired product in a simple manner.

[0047] 2 shows a specific example of the modified property Prm. The modified property Prm may include at least one of the entire property Prw, the partial property Prp, and the destination information Ip.

[0048] The entire property Prw is a modification property Prm for modifying the entire intermediate product Pi. If the intermediate product Pi is an image, the entire intermediate product Pi may refer to the entire area of ​​the image. If the intermediate product Pi is a video, the entire intermediate product Pi may refer to the video from start to finish. In this example, the entire property Prw is a modification property Prm for specifying the background style. For example, the entire property Prw indicates the background of style B that is to be reflected in the modified product Pm.

[0049] The partial property Prp is a modification property Prm for modifying a part of the intermediate product Pi. If the intermediate product Pi is an image, the part of the intermediate product Pi may indicate a partial area of ​​the image. If the intermediate product Pi is a video, the part of the intermediate product Pi may indicate a part of the video, such as one scene of the video. In this example, the partial property Prp is a modification property Prm for specifying an object that is a part of the product. For example, the partial property Prp indicates an apple and a bird object that you want to reflect in the modified product Pm.

[0050] The recipient information Ip is a modified property Prm related to the recipient of the modified product Pm. The recipient information Ip may include the age, gender, place of residence, nationality, affiliation, company name, job title, or preferences of the recipient of the product. The recipient information Ip may include information regarding the emotional expression or joy, anger, sadness, or pleasure desired to be imparted to the recipient. The modified property Prm in this example does not specify the recipient information Ip.

[0051] The product generation device 100 may generate the intermediate product Pi without specifying a destination, and then set the destination information Ip according to the content of the intermediate product Pi. Alternatively, the product generation device 100 may set a predetermined destination when generating the intermediate product Pi, and then change the destination information Ip to a destination according to the content of the intermediate product Pi when generating the modified product Pm.

[0052] Here, the overall property Prw may include quantitative properties. The quantitative properties may include the temporal and spatial size of the product. The quantitative properties may be the pixel size of an image or movie, the number of characters of text, or the duration of audio, music, or movie. The quantitative properties may include the temporal and spatial density of the product. The quantitative properties may be the frame rate of a movie or the sampling frequency of audio as a temporal property. The quantitative properties may be the resolution of an image (DPI) as a spatial property.

[0053] The overall property Prw may include a style property that specifies the genre, style, or musical style of the product. The overall property Prw may specify blurring of the image quality, image size, image aspect ratio, color to monochrome conversion, weather, time, or season.

[0054] The partial property Prp may include a property corresponding to a temporal and spatial portion of the product. The partial property Prp may be an object unit, an instrument track, a scene of a video, or a phrase of lyrics. The partial property Prp may specify that the type of object is converted from a dog to a cat, may specify the number of objects, or may specify the position or size of an object.

[0055] The partial property Prp may include a relationship property indicating a temporal and spatial relationship between each object. The relationship property may indicate the spatial positional relationship between a dog and a cat, the temporal order in which musical instruments appear, the temporal order in which scenes appear, or the order in which sentences appear.

[0056] The part property Prp may include an emphasis part specification property that specifies an emphasis part of the product. The emphasis part specification property may specify that a specific object is to be emphasized, may specify the conclusion of a sentence or the climax of a story, may specify the climax of a piece of music, or may specify the climax of a video.

[0057] The part property Prp may be a contrast property that contrasts contrast concepts and includes them in the modified product Pm to further emphasize the emphasized portion. The contrast property may specify the contrast between light and dark colors in an image, or between vivid and pale colors. For example, placing a bright object on a dark background can make the object stand out. The contrast property may emphasize a particular point by juxtaposing contrasting stories, such as success and failure. The contrast property may specify the contrast between high and low tones, or the contrast between fast and slow speaking to attract the listener's attention.

[0058] 3 shows a modified example of the product generation device 100. The correction adjustment unit 60 of this example has a correction part comparison unit 61, a match degree determination unit 62, and a correction suggestion unit 63. The information output unit 50 of this example has a comparison result output unit 52.

[0059] The correction part comparison unit 61 compares the intermediate product Pi with the corrected product Pm. When the products are images, the correction part comparison unit 61 may compare the entire images, or may compare parts of the images, such as specific objects. The correction part comparison unit 61 may compare the sizes of the images, or may compare the lengths of the videos. By comparing the intermediate product Pi with the corrected product Pm, the correction part comparison unit 61 may inform the user whether the corrected property Prm is reflected in the corrected product Pm as intended by the user.

[0060] The match degree determination unit 62 determines the degree of match between a prompt input to the intermediate product generation model 25 to generate the intermediate product Pi and the generated intermediate product Pi. The match degree determination unit 62 may determine that the degree of match is low when the prompt input to the intermediate product generation model 25 is not sufficiently reflected in the intermediate product Pi, and may determine that the degree of match is high when the prompt input to the intermediate product generation model 25 is sufficiently reflected in the intermediate product Pi.

[0061] The correction suggestion unit 63 proposes a correction property Prm for generating a corrected product Pm in line with the user's intention. The correction suggestion unit 63 may propose the correction property Prm based on the comparison result by the correction portion comparison unit 61. The correction suggestion unit 63 may also propose the correction property Prm based on the degree of match determined by the match degree determination unit 62. The correction suggestion unit 63 of this example suggests a correction property Prm to the user based on the degree of match, thereby enabling the user to recognize issues that the user has not yet grasped. The correction suggestion unit 63 may approach the user's intention by repeatedly proposing the correction property Prm to the user. The correction suggestion unit 63 may acquire user information such as a user ID and reflect the user's preferences in the correction property Prm. The correction suggestion unit 63 may also propose the correction property Prm based on the content of a dialogue between the user and the large-scale language model.

[0062] The revision suggestion unit 63 may present the reason for proposing a revision to the user. This allows the product generation device 100 to deepen the user's understanding. By proposing a document revision policy and the reason for it to the user who is creating the document, the product generation device 100 can provide not only product generation but also an educational function for the user.

[0063] The comparison result output unit 52 outputs the comparison result between the intermediate product Pi and the modified product Pm. In one example, the comparison result output unit 52 displays the comparison result between the intermediate product Pi and the modified product Pm to the user. The comparison result output unit 52 may output the extent to which the user instruction has been improved by the modified property Prm presented by the modification suggestion unit 63. The comparison result output from the comparison result output unit 52 may be stored in the storage unit 55.

[0064] Even if the intermediate product Pi is not generated as intended by the user, the product generation device 100 of this example can support the user in generating a modified product Pm, making it easier to generate a modified product Pm that meets the user's intentions. Furthermore, the product generation device 100 of this example can provide the user with the degree of match when the intermediate product Pi is generated, and can suggest a modification method for generating the product that the user originally wanted to generate when the intermediate product Pi is generated.

[0065] 4A shows a modified example of the product generation device 100. The product generation device 100 of this example includes a correction portion designation unit 64 and an unnecessary portion designation unit 65.

[0066] The modification portion designation unit 64 designates a modification portion Rm of the intermediate product Pi that should be modified by the modified product generation unit 40, and / or designates a non-modification portion Rn that should not be modified by the modified product generation unit 40. The modification portion Rm may be modified in accordance with the modification property Prm. The non-modification portion Rn may remain in the intermediate product Pi without being modified in accordance with the modification property Prm. The modification portion designation unit 64 may designate the modification portion Rm and / or the non-modification portion Rn using a mask. The modification portion Rm and the non-modification portion Rn may be manually designated by the user, or may be automatically selected by detecting the user's intention based on the modification property Prm.

[0067] The unnecessary part designation unit 65 designates an unnecessary part Ru of the intermediate product Pi that should be deleted in the modified product Pm. For example, the unnecessary part designation unit 65 designates an object that should be deleted from the image as the unnecessary part Ru. The unnecessary part designation unit 65 may designate the unnecessary part Ru of the intermediate product Pi using a mask. The unnecessary part Ru may be designated manually by the user, or may be automatically selected by detecting the user's intention based on the modification property Prm.

[0068] The unnecessary part designation unit 65 may designate a foreign object that does not conform to the user's intention as the unnecessary part Ru. The unnecessary part designation unit 65 may read the user's intention from the modified property Prm and automatically designate the unnecessary part Ru. The unnecessary part designation unit 65 may designate a part of the intermediate product Pi that was generated in a manner that deviates from the intent of the initial product Ps that is the basis of the intermediate product Pi as the unnecessary part Ru.

[0069] 4B shows an example of a method for generating a modified product Pm. In this example, a case where the modified product Pm is generated using the modification part designation unit 64 will be described. The intermediate product Pi in this example is an image depicting an apple and a background. The modification part designation unit 64 in this example designates the background as the modification part Rm and the apple as the non-modification part Rn. The modified product generation model 45 modifies the background of the intermediate product Pi to generate a modified product Pm in which the apple is not modified from the intermediate product Pi.

[0070] 4C shows an example of a method for generating a modified product Pm. In this example, a case where the modified product Pm is generated using the unnecessary part designation unit 65 will be described. The intermediate product Pi in this example is an image depicting an apple and a bug. The unnecessary part designation unit 65 in this example designates the bug as the unnecessary part Ru. The modified product generation model 45 deletes the bug from the intermediate product Pi and generates a modified product Pm depicting an apple. The apple and background in the modified product Pm are maintained without being modified from the intermediate product Pi.

[0071] 5A shows a modified example of the product generation device 100. The product generation device 100 of this example differs from the product generation device 100 of FIG. 1A in that it includes a correction data acquisition unit 70. The correction data acquisition unit 70 may be appropriately combined with other modified examples of the product generation device 100.

[0072] The correction data acquisition unit 70 acquires at least one of structured data and unstructured data as correction data Dm for generating the corrected product Pm. The correction data Dm is an example of a prompt for reflecting the user's intention. Specific examples of the correction data Dm will be described later.

[0073] The modified product generation unit 40 may generate a modified product Pm that reflects the user's intention based on the modified data Dm. The modified product generation unit 40 of this example generates the modified product Pm based on at least one of structured data and unstructured data. By using the modified data Dm in addition to the modified property Prm, the modified product generation unit 40 can generate a modified product Pm that more accurately reflects the user's intention.

[0074] 5B shows an example of a method for generating the modified product Pm. In this example, a case where the modified product Pm is generated using the modified data acquisition unit 70 will be described.

[0075] In this example, the intermediate product Pi is an image of an apple. The modified data Dm in this example includes structured data and unstructured data. The modified data Dm may include only either structured data or unstructured data. The unstructured data includes an image of a bird. The structured data includes information for specifying an object. In this example, the structured data specifies the object as a bird, the action as a flying state, the size as 300 pixels, and the color as light blue. The modified product generation model 45 in this example generates a modified product Pm that includes the image of the bird specified in the modified data Dm in the intermediate product Pi. This makes it possible to generate a modified product Pm that includes a bird in addition to an apple, in line with the user's intentions.

[0076] 6 shows a modified example of the product generation device 100. The product generation device 100 of this example differs from the product generation device 100 of FIG. 1A in that the modified product generation unit 40 includes a determination unit 42, an automatic reflection unit 44, and a modification degree adjustment unit 46. The configuration of this example may be appropriately combined with other modified examples of the product generation device 100.

[0077] The determination unit 42 determines whether to reflect automatic correction items not specified in the correction property Prm in the corrected product Pm based on predetermined criteria. The automatic correction items may be items not specified in the correction property Prm. In other words, items specified in the correction property Prm may be corrected based on the correction property Prm, and items not specified in the correction property Prm may be automatically corrected. The predetermined criteria may be based on physical phenomena or human anatomical structure, etc. The predetermined criteria may be the direction of gravity or the number of human fingers. If the corrected product Pm is a piece of music, the predetermined criteria may be the chords or scale of the music. If the corrected product Pm is a video, the predetermined criteria may be the tone of voice of the speaker in the video. If the corrected product Pm is text, the predetermined criteria may be the tone of voice of the text. If the corrected product Pm is an image, the predetermined criteria may be perspective or vanishing point.

[0078] When the determination unit 42 determines to reflect the automatic correction items, the automatic reflection unit 44 reflects the automatic correction items in the corrected product Pm. For example, when the number of human fingers included in the intermediate product Pi is different from normal, the automatic reflection unit 44 may automatically change the number of fingers, determining that the number does not satisfy a predetermined standard.

[0079] The correction degree adjustment unit 46 adjusts the degree of correction by the automatic reflection unit 44. The correction degree adjustment unit 46 may adjust whether or not to correct the intermediate product Pi depending on the degree of deviation from a predetermined standard. For example, in the case where the height of a person is automatically corrected when the height exceeds a predetermined range, the correction degree adjustment unit 46 adjusts the range at which the execution of the automatic correction of the height starts. The correction degree adjustment unit 46 may confirm the preferred degree of correction with the user.

[0080] 7A shows a modified example of the product generation device 100. The product generation device 100 of this example differs from the product generation device 100 of FIG. 1A in that it includes an initial product acquisition unit 10. The initial product acquisition unit 10 may be appropriately combined with other modified product generation devices 100.

[0081] The initial product acquisition unit 10 acquires an initial product Ps to be input to the intermediate product generation unit 20. The initial product Ps is a product to be input to the intermediate product generation model 25 to generate the intermediate product Pi. The initial product Ps may be an original product that is not generated using a machine learning model, or may be a product generated using a machine learning model. The initial product Ps may be generated manually by a user. The initial product Ps may be the same as or different from the product type of the intermediate product Pi. Similarly, the initial product Ps, the intermediate product Pi, and the modified product Pm may be the same as or different from the product type.

[0082] The intermediate product generation unit 20 generates an intermediate product Pi based on the initial product Ps. The intermediate product generation unit 20 may generate the intermediate product Pi including an initial product remaining portion Rr where the initial product Ps remains and a generated portion Rg generated by the intermediate product generation model 25. The initial product remaining portion Rr is an area where the initial product Ps remains, and therefore may include content manually generated by the user.

[0083] The property specification unit 30 specifies different modification properties Prm for the initial product remaining portion Rr and the generation portion Rg. The property specification unit 30 may specify the modification properties Prm so that the initial product remaining portion Rr is maintained. The property specification unit 30 may specify the modification properties Prm so that the initial product remaining portion Rr is not modified. The property specification unit 30 may specify the modification properties Prm so that the user's intention is reflected in the generation portion Rg.

[0084] The modified product generation unit 40 generates a modified product Pm in which different modifications are reflected in the initial product remaining portion Rr and the generated portion Rg. The modified product generation unit 40 may modify the generated portion Rg to a greater extent than the initial product remaining portion Rr. That is, the modified product generation unit 40 may modify the generated portion Rg, which is more likely to not be in line with the user's intention, to a greater extent than the initial product remaining portion Rr, which still retains the user's intention.

[0085] The product generation device 100 of this example can reflect different modifications between the generated range generated in the intermediate product Pi and the manual range included in the initial product Ps, thereby more efficiently reflecting the user's intentions.

[0086] Here, machine learning models that generate products can sometimes generate very high-quality products by specifying arbitrary prompts, but it is not easy to generate a product that meets the user's intentions. If a product that meets the user's intentions cannot be generated, the specified prompts may be changed and the product may be recreated.

[0087] In particular, when the types of products are different between the initial product Ps and the intermediate product Pi, such as when the initial product Ps is text and the intermediate product Pi is an image, an intermediate product Pi that differs from the user's intention is likely to be generated. That is, even when similar text is input, a variety of images may be generated, which may not meet the user's intention. The product generation device 100 of this example can specify the modification property Prm to generate a modified product Pm that meets the user's intention, even when it is not easy to generate a product that meets the user's intention, such as when generating an image using text as a prompt.

[0088] Furthermore, the modified property Prm may be text or unstructured data such as an image. Even if it is difficult to reflect the user's intention in text, inputting unstructured data as the modified property Prm makes it easier to reflect the user's intention in the modified product Pm.

[0089] On the other hand, even when the initial product Ps and the intermediate product Pi are the same type of product, the user's intention may not be accurately reflected, and the intermediate product Pi may be too different from the initial product Ps. Also, the user's intention may not be accurately reflected, and the intermediate product Pi may be barely modified from the initial product Ps. The product generation device 100 of this example can generate a modified product Pm that is more in line with the user's intention by accurately reflecting the modification intention in the modification property Prm.

[0090] 7B is a diagram for explaining a method for generating the modified product Pm. The initial product Ps, the intermediate product Pi, and the modified product Pm are images containing apples.

[0091] The intermediate product Pi has a generation point Rg and an initial product remaining point Rr. In this example, the generation point Rg is the background, and the initial product remaining point Rr is the apple. In the generation point Rg, no elements of the corresponding point in the initial product Ps may remain. On the other hand, in the initial product remaining point Rr, elements of the apple in the initial product Ps remain. The intermediate product Pi may be generated while maintaining the composition of the initial product Ps.

[0092] The modification property Prm may be different for the generation location Rg and the initial product remaining location Rr. The modification property Prm specifies that the background of the generation location Rg should be modified to a beautiful forest. The modification property Prm specifies that the apple of the initial product remaining location Rr should have a lustrous color. The modification property Prm may change the position and size of the apple of the initial product remaining location Rr.

[0093] 8A shows a modified example of the product generation device 100. The product generation device 100 of this example differs from the product generation device 100 of FIG. 1A in that it includes a user input information acquisition unit 80. The user input information acquisition unit 80 may be appropriately combined with other modified examples of the product generation device 100.

[0094] The user input information acquisition unit 80 acquires user input information Iu input by a user. The user input information acquisition unit 80 may acquire the user input information Iu and transmit it to the modified product generation unit 40. The user input information acquisition unit 80 of this example includes a modification part adjustment unit 82.

[0095] The user input information Iu may include location designation information Id relating to a location Rm to be modified by the modified product generation unit 40 and / or a location Rn not to be modified by the modified product generation unit 40. In other words, the location designation information Id is an example of the user input information Iu.

[0096] The location designation information Id may include information manually input by the user using a keyboard, mouse, pen-type input means, or the like. The location designation information Id may be information in which the user manually designates an arbitrary position in the intermediate product Pi. The location designation information Id may include information indicating the correction location Rm in the intermediate product Pi. The user may circle the area they want to designate in red, or may fill it in with a brush. The user may designate the destination of the correction location Rm by combining a red circle and an arrow. The user may designate an area in the intermediate product Pi that does not meet the user's intention as the correction location Rm. The location designation information Id may display candidates for the correction location Rm or the non-correction location Rn and allow the user to select.

[0097] Furthermore, the user input information Iu may include information indicating the details of modifications to the intermediate product Pi. For example, the user input information Iu may include information in which the user manually draws the modification area Rm and the modifications to be reflected in the modification area Rm. The user may draw a rough outline of the composition or pose of the modified product Pm to be generated. The user input information Iu may be input by voice using the user's own voice, or may be input by voice using the sound of an instrument played by the user.

[0098] The modified product generation unit 40 generates a modified product Pm based on the modified property Prm and the user input information Iu. In one example, the modified product generation unit 40 generates a modified product Pm in which the content of the modified property Prm is reflected in the modification location Rm specified by the user input information Iu.

[0099] The correction part adjustment unit 82 adjusts the correction part Rm specified by the user in the user input information Iu to a more appropriate area. The correction part adjustment unit 82 adjusts the correction part Rm and / or the non-correction part Rn based on the part designation information Id. The part designation information Id may be information designating a spatial area such as an image, or may be information designating an arbitrary time point in a video or the like.

[0100] The product generation device 100 of this example can generate a modified product Pm that is more in line with the user's intentions by using the user input information Iu. The product generation device 100 may use the user input information Iu in combination with the modified property Prm and / or other information. The product generation device 100 may use the user input information Iu in combination with the modified property Prm and the modified data Dm.

[0101] 8B shows an example of a method for generating a modified product Pm. User input information Iu is location specification information Id that specifies a partial region of the intermediate product Pi. In this example, the user input information Iu specifies a partial region of the background of the intermediate product Pi, to which the modification property Prm is to be applied. The modification property Prm specifies that the intermediate product Pi should be modified so that a bird can fly in the background. Based on the modification property Prm and the user input information Iu, the modified product generation model 45 generates a modified product Pm in which a bird has been added to the region specified by the location specification information Id.

[0102] 8C shows an example of a method for adjusting the correction portion Rm. In this example, the intermediate product Pi is an image of a cat.

[0103] The user input information Iu selects the area where the cat is located, so as to specify the cat. In this example, the user input information Iu includes a circular area manually input by the user as the location specification information Id. The user may set the user input information Iu by handwriting using a pen-type input device, or may select by manipulating any shape, such as a circle, with a mouse or the like. The location specification information Id is the user's intention to specify the entire cat, but does not completely match the outline of the cat.

[0104] The virtual intermediate product Pi is an intermediate product Pi in which the user input information Iu has been adjusted by the correction part adjustment unit 82. The virtual intermediate product Pi may or may not be displayed to the user. The correction part Rm indicates an area corresponding to the image of the cat selected by the user input information Iu. In this example, the correction part Rm is adjusted to fit the outline of the cat in accordance with the user input information Iu specified as a circle. The non-correction part Rn may be an area that was not selected as the correction part Rm. The product generation device 100 may display the adjusted correction part Rm to the user for confirmation.

[0105] Even if the user does not input the user input information Iu accurately, the product generation device 100 of this example adjusts the content of the user input information Iu to reflect the user's intention, thereby making it possible to generate a modified product Pm that conforms to the user's intention with reduced burden on the user.

[0106] In this example, the correction part adjustment unit 82 adjusts the contours of the image in the spatial direction, but other user input information Iu may also be adjusted. When the intermediate product Pi is audio or video, the correction part adjustment unit 82 may also adjust the part designation information Id in the time series direction. The correction part adjustment unit 82 may adjust the part designation information Id of the video designated by the user to match the division of an editing point. The correction part adjustment unit 82 may adjust the part designation information Id of the music designated by the user in units of measures or divisions of song parts (e.g., verse 1, verse 2, chorus). For video and music, the correction part adjustment unit 82 may adjust the part designation information Id in units of tracks. When the image includes multiple layers, the correction part adjustment unit 82 may adjust the part designation information Id in units of layers.

[0107] 9 shows a modified example of the product generation device 100. The product generation device 100 of this example differs from the product generation device 100 of FIG. 1A in that it includes a generative model selection unit 48. The generative model selection unit 48 may be appropriately combined with other modified examples of the product generation device 100.

[0108] The generation model selection unit 48 selects a modified product generation model 45 to be used for generating the modified product Pm based on the modified property Prm. The generation model selection unit 48 may select an arbitrary modified product generation model 45 from the multiple modified product generation models 45. The generation model selection unit 48 may select the same generation model as the intermediate product generation model 25, or may select a generation model different from the intermediate product generation model 25.

[0109] The generation model selection unit 48 may select the modified product generation model 45 depending on the style of the modified product Pm to be generated. For example, the generation model selection unit 48 may select the modified product generation model 45 depending on whether the modified product Pm to be generated is character-style or realistic-style. The generation model selection unit 48 may select the modified product generation model 45 depending on the type of modified product Pm to be generated. Even if the type of product is the same, the generation model selection unit 48 may change the modified product generation model 45 depending on the overall style.

[0110] 10A shows a modified example of the product generation device 100. The product generation device 100 of this example differs from the product generation device 100 of FIG. 1A in that it includes a template information acquisition unit 90. The template information acquisition unit 90 may be appropriately combined with other modified examples of the product generation device 100.

[0111] The template information acquisition unit 90 acquires template information It that specifies a predetermined template. The template information acquisition unit 90 may transmit the acquired template information It to the modified product generation unit 40.

[0112] The template information It may include a template that describes the relationship between properties of inputs and outputs when generating the modified product Pm based on the intermediate product Pi. For example, the template information It may specify that two selected objects are used as inputs and that these objects are swapped and output.

[0113] The template information It may include a template that specifies how to apply the input modification data Dm to the modified product Pm. For example, the template information It specifies that, when the modification data Dm that specifies a style is input, an image whose background has been changed to match the style of the modification data Dm is output.

[0114] The template information It may include a template that specifies the scope of application of the input correction data Dm to the corrected product Pm. For example, the template information It specifies that the correction data Dm and the correction location Rm are input, and an image in which the correction data Dm is depicted at the correction location Rm is output.

[0115] The template information It may be selected from predetermined types or may be registered by the user. The template information It may include information that changes the combination of season, weather, and time. The template information It may retain the shapes of objects and other elements and change only the season, weather, and time. The template information It may include information that specifies the range of parameters for the size, angle, distortion, and color conversion of a defect portion when generating a defect image. The product generation device 100 can use these settings as a template to generate a corrected product Pm that includes any defect portion.

[0116] The modified product generation unit 40 generates the modified product Pm based on the modified property Prm and the template information It. The modified product generation unit 40 may generate the modified product Pm by applying the modified property Prm based on the template information It. The modified product generation unit 40 may also apply the modified property Prm independently, without based on the template information It.

[0117] The modified product generation unit 40 may generate a plurality of modified products Pm. The modified product generation unit 40 may generate a plurality of modified products Pm based on a list of modified properties Prm, or may generate a plurality of modified products Pm by specifying a plurality of modified properties Prm from a predetermined range. The modified product generation unit 40 may generate a modified product Pm using modified properties Prm specified randomly from a predetermined range.

[0118] The product generation device 100 of this example can more easily generate a modified product Pm that is in line with the user's intention by using the template information acquisition unit 90. The product generation device 100 may display a plurality of modified products Pm and allow the user to select one that is in line with the user's intention.

[0119] The product generation device 100 may include multiple pieces of template information It. The product generation device 100 may select any one of the multiple pieces of template information It to generate a modified product Pm. The product generation device 100 may select any one of the multiple pieces of template information It according to the degree of match and recommend it to the user. The product generation device 100 may selectively display high-priority template information It to the user. The product generation device 100 may display templates in a pull-down list and generate a modified product Pm by selecting an intermediate product Pi or a tag. This reduces the number of times the user enters input prompts, making it possible to more easily generate a modified product Pm that meets the user's intentions.

[0120] 10B shows an example of a method for generating a modified product Pm. The intermediate product Pi is an image of an apple drawn on a background of style A. The modification property Prm indicates that the style of the background is to be changed to style B.

[0121] The template information It indicates that the modified data Dm is to be placed at the position of the user input information Iu. The modified data Dm may include an image of a bird as unstructured data. The structured data of the modified data Dm specifies the object as a bird, the action as flying, the size as 300 pixels, and the color as light blue. The user input information Iu specifies the area in the left corner of the image. The modified product generation model 45 of this example changes the background to style B and generates a modified product Pm in which the modified data Dm is placed at the position of the user input information Iu. The modified data Dm may specify multiple objects.

[0122] 10C shows an example of a method for generating a modified product Pm. The template information It specifies that two selected objects are to be swapped. The user input information Iu selects a dog and a cat as two objects in the intermediate product Pi. The modified product Pm is an image in which the dog and cat in the intermediate product Pi are swapped.

[0123] 10D shows an example of a method for generating a modified product Pm. The template information It specifies that an object specified in the modified data Dm is to appear in a specified location of the intermediate product Pi. The modified data Dm may be unstructured data. In this example, the modified data Dm specifies an image of a dog. The user input information Iu specifies an area to the left of the cat. The modified product Pm is an image in which the dog appears in the area to the left of the cat in the intermediate product Pi.

[0124] 10E shows an example of a method for generating a modified product Pm. The template information It specifies that the background of the intermediate product Pi be changed to the style of the modified data Dm. The modified data Dm may be unstructured data. The modified data Dm in this example specifies a background of style B. The modified product Pm is an image in which the background of the intermediate product Pi has been changed to style B.

[0125] FIG. 10F shows an example of a method for generating a modified product Pm. The template information It specifies that the background of the intermediate product Pi be modified to the style of the modified data Dm. The modified data Dm specifies three backgrounds, namely, the sea, mountains, and rivers, in list format. The modified product generation model 45 of this example generates multiple modified products Pm based on the list of modification data Dm. The modified product generation model 45 of this example generates three modified products Pm1 to Pm3 according to the modification data Dm. The backgrounds of the modified products Pm1 to Pm3 are set to the sea, mountains, and rivers, respectively. The product generation device 100 may display the multiple generated modified products Pm to the user, allowing the user to select a modified product Pm that matches their intentions.

[0126] In this example, the modification data Dm is specified in a list format by specifying multiple data items, but multiple values ​​may be selected randomly by specifying a range of values.In this example, the modification data Dm is specified in a list format, but the modification property Prm may also be specified in a list format.

[0127] 11 shows an example of a method for generating a modified product Pm. The product generation device 100 of this example may include a user input information acquisition unit 80 and a template information acquisition unit 90. The product generation device 100 may also include a modification portion designation unit 64 and an unnecessary portion designation unit 65.

[0128] The initial product Ps is an image containing an apple, and the intermediate product Pi is an image of an apple and a bug on a background of style A.

[0129] The modification property Prm specifies that the background style is to be changed to style B. The template information It indicates that the modification data Dm is to be placed at the position of the user input information Iu. The modification data Dm includes unstructured data and structured data for adding an image of a bird. The user input information Iu specifies the area in the left corner of the image. The unnecessary area Ru specifies the area where an insect is drawn. The unmodified area Rn specifies an apple. The modified product generation model 45 in this example changes the background to style B, places the image of the bird at the position of the user input information Iu, deletes the image of the insect, and generates a modified product Pm in which the apple image is not modified.

[0130] In this way, it is possible to combine multiple modified examples of the product generation device 100. That is, the product generation device 100 may include at least two of the correction adjustment unit 60, the correction data acquisition unit 70, the user input information acquisition unit 80, and the template information acquisition unit 90.

[0131] As described above, the product generation device 100 can set the modification property Prm to generate a modified product Pm that meets the user's intention. The product generation device 100 may be used to generate a training dataset for machine learning. By using a combination of the modified product Pm generated by the product generation device 100 and the initial product Ps as a training dataset, the machine learning model can be trained to generate a product that meets the user's intention. Although defective product images generated by the machine learning model may have unnatural shapes and textures of defective areas, using the modified product Pm generated by the product generation device 100 makes it possible to learn data that meets the user's intention.

[0132] 12 illustrates an example of a computer 2200 in which aspects of the present invention may be embodied, in whole or in part. Programs installed on the computer 2200 may cause the computer 2200 to function as or perform operations associated with an apparatus or one or more sections of the apparatus according to embodiments of the present invention, and / or to perform a process or steps of a process according to embodiments of the present invention. Such programs may be executed by the CPU 2212 to cause the computer 2200 to perform specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0133] A computer 2200 according to this embodiment includes a CPU 2212, a RAM 2214, a graphics controller 2216, and a display device 2218, which are interconnected by a host controller 2210. The computer 2200 also includes input / output units such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card drive, which are connected to the host controller 2210 via an input / output controller 2220. The computer also includes legacy input / output units such as a ROM 2230 and a keyboard 2242, which are connected to the input / output controller 2220 via an input / output chip 2240.

[0134] The CPU 2212 operates according to programs stored in the ROM 2230 and RAM 2214, thereby controlling each unit. The graphics controller 2216 acquires image data generated by the CPU 2212 into a frame buffer or the like provided in the RAM 2214 or into the graphics controller 2216 itself, and causes the image data to be displayed on the display device 2218.

[0135] The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data used by the CPU 2212 in the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides the programs or data to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0136] ROM 2230 stores therein a boot program or the like that is executed by computer 2200 upon activation, and / or programs that depend on the hardware of computer 2200. I / O chip 2240 may also connect various I / O units to I / O controller 2220 via a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0137] The programs are provided by a computer-readable medium such as a DVD-ROM 2201 or an IC card. The programs are read from the computer-readable medium, installed in the hard disk drive 2224, RAM 2214, or ROM 2230, which are also examples of computer-readable media, and executed by the CPU 2212. Information processing described in these programs is read by the computer 2200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by realizing information manipulation or processing in accordance with the use of the computer 2200.

[0138] For example, when communication is performed between computer 2200 and an external device, CPU 2212 may execute a communication program loaded into RAM 2214 and instruct communication interface 2222 to perform communication processing based on the processing described in the communication program. Under the control of CPU 2212, communication interface 2222 reads transmission data stored in a transmission buffer processing area provided in RAM 2214, hard disk drive 2224, DVD-ROM 2201, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes received data received from the network to a reception buffer processing area or the like provided on the recording medium.

[0139] Furthermore, the CPU 2212 may cause all or a necessary portion of a file or database stored on an external recording medium such as the hard disk drive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), an IC card, etc. to be read into the RAM 2214, and may perform various types of processing on the data on the RAM 2214. The CPU 2212 then writes back the processed data to the external recording medium.

[0140] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 2212 may perform various types of processing on data read from the RAM 2214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 2214. The CPU 2212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored on the recording medium, the CPU 2212 may search for an entry that matches a condition specified by the attribute value of the first attribute from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0141] The above-described programs or software modules may be stored in a computer-readable medium on or near the computer 2200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable medium, thereby providing the programs to the computer 2200 via the network.

[0142] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0143] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order.

[0144] 10... initial product acquisition unit, 20... intermediate product generation unit, 25... intermediate product generation model, 30... property specification unit, 40... modified product generation unit, 42... determination unit, 44... automatic reflection unit, 45... modified product generation model, 46... modification degree adjustment unit, 48... generation model selection unit, 50... information output unit, 52... comparison result output unit, 55... storage unit, 60... modification adjustment unit, 61... modification part comparison unit, 62... match degree determination unit, 63... modification proposal unit, 64... modification part specification unit, 65... unnecessary part specification unit, 70... modification data acquisition unit, 80... user User input information acquisition unit, 82...correction part adjustment unit, 90...template information acquisition unit, 100...product generation device, 2200...computer, 2201...DVD-ROM, 2210...host controller, 2212...CPU, 2214...RAM, 2216...graphics controller, 2218...display device, 2220...input / output controller, 2222...communication interface, 2224...hard disk drive, 2226...DVD-ROM drive, 2230...ROM, 2240...input / output chip, 2242...keyboard

Claims

1. An intermediate product generation unit that generates an intermediate product using a predetermined intermediate product generation model, a property specification unit that specifies a correction property for correcting the intermediate product, and a correction product generation unit that generates a corrected product by correcting the intermediate product based on the specified correction property. A product generation device comprising:

2. The product generation device according to claim 1, wherein the correction property includes an overall property for correcting the entire intermediate product.

3. The product generation device according to claim 1, wherein the correction property includes a partial property for correcting a part of the intermediate product.

4. The product generation device according to claim 1, wherein the correction property includes destination information regarding the destination of the corrected product.

5. The product generation device according to claim 1, further comprising a correction location comparison unit that compares the intermediate product and the corrected product, and a comparison result output unit that outputs a comparison result between the intermediate product and the corrected product.

6. The product generation device according to claim 1, further comprising a matching degree determination unit that determines the matching degree between the prompt input to the intermediate product generation model to generate the intermediate product and the generated intermediate product, and a correction proposal unit that proposes the correction property according to the matching degree.

7. The product generation device according to claim 1, further comprising a correction location specification unit that specifies a correction location to be corrected by the correction product generation unit and / or a non-correction location that should not be corrected by the correction product generation unit among the intermediate products.

8. The product generation device according to claim 1, further comprising an unnecessary location specification unit that specifies an unnecessary location to be deleted from the intermediate product in the corrected product.

9. The product generation device according to any one of claims 1 to 8, further comprising a correction data acquisition unit that acquires at least one of structured data or unstructured data as correction data for generating the corrected product, and the correction product generation unit generates the corrected product based on at least one of the structured data or unstructured data.

10. The correction product generation unit includes a determination unit that determines whether to reflect in the correction product automatic correction items not specified by the correction property based on a predetermined criterion, and an automatic reflection unit that reflects the automatic correction items in the correction product when the determination unit determines to reflect the automatic correction items. The product generation device according to any one of claims 1 to 8.

11. The correction product generation unit has a correction degree adjustment unit for adjusting the degree of correction by the automatic reflection unit. The product generation device according to claim 10.

12. An initial product acquisition unit for acquiring an initial product for input to the intermediate product generation unit is provided. The intermediate product generation unit generates the intermediate product including the remaining location of the initial product where the initial product remains and the generated location generated by the intermediate product generation model. The product generation device according to any one of claims 1 to 8.

13. The property specification unit specifies different correction properties for the remaining location of the initial product and the generated location, and the correction product generation unit generates the correction product in which different corrections are reflected for the remaining location of the initial product and the generated location. The product generation device according to claim 12.

14. A user input information acquisition unit for acquiring user input information input by a user is provided. The correction product generation unit generates the correction product based on the correction property and the user input information. The product generation device according to any one of claims 1 to 8.

15. The user input information includes location specification information regarding a correction location to be corrected by the correction product generation unit and / or a non-correction location that should not be corrected by the correction product generation unit. A correction location adjustment unit for adjusting the correction location and / or the non-correction location based on the location specification information is provided. The product generation device according to claim 14.

16. A template information acquisition unit for acquiring template information for specifying a predetermined template is provided. The correction product generation unit generates the correction product based on the correction property and the template information. The product generation device according to any one of claims 1 to 8.

17. The product generation device according to any one of claims 1 to 8, wherein the corrected product generation unit includes a generation model selection unit for selecting a corrected product generation model to be used for generating the corrected product based on the correction property.

18. A product generation method comprising: a step in which a computer generates an intermediate product using a predetermined intermediate product generation model; a step in which the computer designates a correction property for correcting the intermediate product; and a step in which the computer generates a corrected product obtained by correcting the intermediate product based on the designated correction property.

19. A program which, when executed by a computer, causes the computer to generate an intermediate product using a predetermined intermediate product generation model, to designate a correction property for correcting the intermediate product, and to generate a corrected product obtained by correcting the intermediate product based on the designated correction property.

Images