Information processing device, information processing method, and computer-readable recording medium

Abstract

An information processing device 10 comprises: a display control unit 11 that displays, on a screen, text information and a group of objects for forming an image; an input reception unit 12 that receives, on the screen, a selection of an object from the group of objects and a designation of a position of the selected object; and an image generation unit 13 that places the selected object at the designated position on the screen and generates an image composed of the selected object.

Description

Information processing device, information processing method, and computer-readable recording medium

[0001] The present disclosure relates to an information processing device and an information processing method, and further to a computer-readable recording medium on which a program for realizing these is recorded.

[0002] Techniques for generating images based on input text have been proposed. Patent Document 1 discloses an apparatus for generating images using a dictionary. The dictionary holds attribute values ​​of text, corresponding elements, and their graphic display processing. When text is input, the apparatus disclosed in Patent Document 1 uses the dictionary to perform graphic display processing (size adjustment, position adjustment) on the elements corresponding to the input text, thereby generating an image.

[0003] Japanese Patent Application Laid-Open No. 2003-196306

[0004] However, in the device disclosed in Patent Document 1, images are automatically generated according to text, which reduces the burden on the user when generating images, but there is a possibility that images not intended by the user may be generated.

[0005] One example of a goal of the present disclosure is to assist users in generating images according to text.

[0006] In order to achieve the above object, an information processing device according to one aspect of the present disclosure is characterized by comprising: a display control unit that displays text information and a group of objects that constitute an image on a screen; an input receiving unit that receives a selection of an object from the group of objects and a specification of a position of the selected object on the screen; and an image generating unit that places the selected object at a specified position on the screen and generates an image composed of the selected objects.

[0007] In addition, in order to achieve the above object, an information processing method according to one aspect of the present disclosure is characterized by having: a display control step of displaying text information and a group of objects for constituting an image on a screen; an input receiving step of selecting an object from the group of objects and receiving a designation of the position of the selected object on the screen; and an image generation step of placing the selected object at a designated position on the screen and generating an image composed of the selected objects.

[0008] Furthermore, in order to achieve the above object, a computer-readable recording medium according to one aspect of the present disclosure is characterized in that it records a program including instructions for causing a computer to execute the following steps: a display control step for displaying text information and a group of objects for constituting an image on a screen; an input acceptance step for accepting the selection of an object from the group of objects and the specification of the position of the selected object on the screen; and an image generation step for placing the selected object at the specified position on the screen and generating an image composed of the selected objects.

[0009] As described above, according to the present disclosure, it is possible to assist a user in generating an image according to text.

[0010] FIG. 1 is a configuration diagram showing a schematic configuration of a first example of an information processing device. FIG. 2 is a configuration diagram showing a more specific configuration of the example of the information processing device. FIG. 3 is a diagram showing a first example of an operation screen of the information processing device. FIG. 4 is a flow diagram showing the operation of the first example of the information processing device. FIG. 5 is a configuration diagram showing the configuration of a second example of the information processing device. FIG. 6 is a diagram showing a second example of an operation screen of the information processing device. FIG. 7 is a flow diagram showing the operation of the second example of the information processing device. FIG. 8 is a configuration diagram showing the configuration of a third example of the information processing device. FIG. 9 is a diagram showing a third example of the operation screen of the information processing device. FIG. 10 is a flow diagram showing the operation of the third example of the information processing device. FIG. 11 is a block diagram showing an example of a computer that realizes the information processing device.

[0011] First Embodiment In the first embodiment, a first example of an information processing device, an information processing method, and a program will be described below with reference to FIGS.

[0012] [Device Configuration] First, a schematic configuration of a first example of an information processing device will be described with reference to Fig. 1. Fig. 1 is a diagram showing a schematic configuration of the first example of an information processing device.

[0013] 1 is a device for generating an image. As shown in FIG. 1, the information processing device 10 includes a display control unit 11, an input receiving unit 12, and an image generating unit 13.

[0014] The display control unit 11 displays text information and a group of objects that constitute an image on a screen. The input receiving unit 12 receives a selection of an object from the group of displayed objects and a specification of the position of the selected object on the screen. The image generation unit 13 places the selected object at the specified position on the screen and generates an image made up of the selected objects.

[0015] In this way, according to the information processing device 10, an image corresponding to the text is generated simply by the user selecting an object and specifying the position of the object while checking the text information displayed on the screen. In other words, the information processing device 10 can assist the user in generating an image corresponding to the text.

[0016] Next, the configuration and functions of the information processing device 10 will be specifically described with reference to Figures 2 and 3. Figure 2 is a configuration diagram showing a more specific configuration of an example of the information processing device. Figure 3 is a diagram showing a first example of an operation screen of the information processing device.

[0017] 2, the information processing device 10 includes a display control unit 11, an input receiving unit 12, and an image generation unit 13, as well as a voice recognition unit 14, a training data generation unit 15, and a storage unit 16. With this configuration, the information processing device 10 can output text information obtained from voice and generated images as training data for constructing a machine learning model for image generation. This will be described in detail below.

[0018] When voice data is input to the information processing device 10, the storage unit 16 stores the input voice data. The storage unit 16 also stores object information. The object information is image data of each object that constitutes an object group. Furthermore, the storage unit 16 also stores text information generated by the voice recognition processing unit 14, as will be described later.

[0019] The speech recognition unit 14 performs speech recognition processing on the speech data stored in the storage unit 16, thereby generating text data from the speech data. The speech recognition processing unit 14 stores the generated text data in the storage unit 16 as text information to be displayed on the screen. Existing technology is used for the speech recognition processing performed by the speech recognition processing unit 14.

[0020] The display control unit 11 displays a user interface 21 on the screen of the display device 20, as shown in Fig. 3. In the example of Fig. 3, the user interface 21 is composed of an object list 22 showing a group of objects, a voice playback button 23, a text window 24 displaying text information that is the voice recognition result, and an image window 25 displaying an image generated by the image generation unit 13.

[0021] When the user clicks the audio playback button 23 in the user interface 21, the audio recognition unit 14 executes audio recognition processing on the audio data stored in the storage unit 16 to generate text information, and stores the generated text information in the storage unit 16. The display control unit 11 then displays the text information in a text window 24. The display control unit 11 also displays each object in an object list 22 using the object information stored in the storage unit 16.

[0022] Furthermore, when the input receiving unit 12 receives the user's selection of an object and the designation of its position, the image generating unit 13 places the selected object at the designated position in the image window 25. Then, the image generating unit 13 sets the image in the image window 25 in which the object is placed as the generated image.

[0023] Thereafter, the image generation unit 13 outputs the displayed text information and image data of the generated image to the training data generation unit 15. The training data generation unit 15 generates training data for learning from the output text information and image data of the generated image, and outputs the generated training data to the outside. The output training data is used as training data for a machine learning model, specifically, a generative AI, that generates images from text information.

[0024] [Device Operation] Next, the operation of the information processing device will be described with reference to FIG. 4. FIG. 4 is a flow diagram showing the operation of a first example of the information processing device. In the following description, reference will be made to FIGS. 1 to 3 as appropriate. In addition, in the first embodiment, an information processing method is implemented by operating the information processing device 10. Therefore, the description of the information processing method will be replaced by the following description of the operation of the information processing device 10.

[0025] First, as a premise, audio data and object information are stored in the storage unit 16. Also, the display control unit 11 displays a user interface 21 on the screen of the display device 20 (see FIG. 3).

[0026] 4, when the user first clicks the voice playback button 23, the voice recognition unit 14 executes a voice recognition process on the voice data stored in the storage unit 16 to generate text information (step A1). The voice recognition unit 14 also stores the generated text information in the storage unit 16.

[0027] Next, the display control unit 11 displays the text information in the text window 24 and the object information in the object list 22 in the user interface 21 (step A2).

[0028] Next, the input receiving unit 12 receives the user's selection of an object and the designation of its position, and notifies the image generating unit 13 of the selected object and its position (step A3).

[0029] Next, the image generating unit 13 places the object selected in step A3 at the specified position in the image window 25 (step A4).

[0030] Next, the input receiving unit 12 determines whether or not the user has instructed the end of the process (step A5). If the result of the determination in step A5 is that the user has not instructed the end of the process, step A3 is executed again.

[0031] On the other hand, if the result of the determination in step A5 indicates that the user has instructed to end the process, the image generator 13 sets the image in the image window 25 in which the object is placed as the generated image (step A6). The image generator 13 also outputs the displayed text information and the image data of the generated image to the training data generator 15.

[0032] Thereafter, the training data generating unit 15 outputs the output text information and the image data of the generated image to the outside as training data for learning (step A7).

[0033] [Effects of First Embodiment] As described above, according to the first embodiment, it is possible to assist a user in generating images according to text. Furthermore, the first embodiment is particularly useful when creating training data for a generative AI that generates images from text information. In general, building a generative AI requires a huge amount of training data, and creating the training data takes time and effort. In contrast, by using the first embodiment, the time and effort required to create the training data can be reduced.

[0034] In the information processing device 40, the input receiving unit 12 can further receive input of motion information necessary to move an object in the generated image. The input receiving unit 12 also stores the motion information in the storage unit 16.

[0035] The motion information may include coordinates of the start point and the end point when the object moves. In this case, the coordinates of the start point and the end point are specified by the user on the screen via an input device. Alternatively, a path drawn by the user on the screen using the input device may be used as motion information indicating the path of the object's movement.

[0036] Furthermore, time information may be input as the operation information. Specifically, the time information may include the start and end points of an object, the time at each point, and the time required from the start to the end point. When the user specifies the start and end points using an input device, the user also inputs the time information.

[0037] When the motion information is input, the image generation unit 13 generates a video in which an object moves based on the motion information. Specifically, the image generation unit 13 can generate the video by using image generation AI (Artificial Intelligence) for video generation. When the video is generated by the image generation unit 13, the training data generation unit 13 outputs the text information and the generated video as training data.

[0038] [Program] In the first embodiment, the program may be any program that causes a computer to execute steps A1 to A7 shown in Fig. 4. By installing and executing this program on a computer, the information processing device 10 and the information processing method can be realized. In this case, the processor of the computer functions as the display control unit 11, the input receiving unit 12, the image generation unit 13, the voice recognition unit 14, and the training data generation unit 15 and performs the processing.

[0039] In the first embodiment, the storage unit 16 may be implemented by a storage device such as a hard disk provided in the computer, or may be implemented by a storage device of another computer. Examples of the computer include a general-purpose PC, a server computer, a smartphone, and a tablet terminal device.

[0040] In addition, in the first embodiment, the program may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as one of the display control unit 11, the input receiving unit 12, the image generation unit 13, the voice recognition unit 14, and the training data generation unit 15.

[0041] Second Embodiment Next, in a second embodiment, a second example of an information processing device, an information processing method, and a program will be described with reference to FIGS.

[0042] [Device Configuration] First, the configuration of the second example of the information processing device will be described with reference to Fig. 5. Fig. 5 is a diagram showing the configuration of the second example of the information processing device.

[0043] 5, the information processing device 30 differs from the information processing device 10 described in the first embodiment in that it includes a document summarizing unit 31 and in the function of the image generating unit 13. The following description will focus on the differences from the first embodiment.

[0044] First, the document summarization unit 31 summarizes the sentences to generate text information. Specifically, the document summarization unit 31 acquires the results of the speech recognition process by the speech recognition unit 14 as a document, performs document summarization on the acquired document, and stores the results as text information in the storage unit 16. The document summarization process performed by the document summarization unit 31 uses existing technology that utilizes a neural network.

[0045] In the second embodiment, the image generation unit 13 uses text information to cause the image generation AI 32 to generate an initial image. In the example of Fig. 5, the image generation AI 32 is located outside the information processing device 30 and is connected to the information device 30 via a network or the like so as to be able to communicate data with the information device 30. The image generation AI 32 may also be provided in the information device 30.

[0046] The image generation AI 32 is a machine learning model trained to generate images from text. Specifically, the image generation AI 32 includes an image generation AI (reference literature) that generates vector images from text. This is because if the generated image is a vector image, it becomes easier for the user to change the position of an object within the image. Reference literature: IconShop: Text-Guided Vector Icon Synthesis with Autoregressive Transformers (acm.org), https: / / dl.acm.org / doi / abs / 10.1145 / 3618364?casa_token=IAijMJ279VgAAAAA%3A1DqDXeVbb4mmayNNHk1RVVffTdRiU72dG3cDzHa0jvr7leI9jVTS8Wm24xxIJTske-mmSVB2p0s2Tg

[0047] The image generation AI 32 may be an image generation AI that generates a raster image. The image generation AI 32 may also be a machine learning model that is trained using training data generated by the training data generation unit 15, or may be a machine learning model that is not trained.

[0048] Furthermore, the image generation unit 13 displays the initial image generated by the image generation AI on the image window 25 (see FIG. 3 ). Then, the image generation unit 13 places the selected object on the initial image at a specified position on the screen to generate an image.

[0049] The input accepting unit 12 can also accept corrections to the text information. In this case, the input accepting unit 12 corrects the text information stored in the storage unit 16 in accordance with the accepted corrections. As a result, the display control unit 11 reads the corrected text information from the storage unit 16 and updates the text window 24.

[0050] In this case, the image generation unit 13 modifies the image in the image window 25 in accordance with the modification of the text information. Specifically, the image generation unit 13 causes the image generation AI 32 to regenerate the initial image using the modified text information, and updates the image in the image window 25 with the newly generated initial image.

[0051] Updating an image by correcting text information will be described with reference to Fig. 6. Fig. 6 is a diagram showing a second example of an operation screen of an information processing device. Fig. 6 shows an example in which text information has been corrected from the example shown in Fig. 3 and the image has been updated accordingly.

[0052] 6, the text "Two black cars collided at the intersection" in the text window 24 is corrected to "A white car and a black car collided at the intersection" (top of FIG. 6). As a result, the image generator 13 corrects one of the black cars to a white car in the image window 25 and updates the image (bottom of FIG. 6).

[0053] [Device Operation] Next, the operation of the information processing device will be described with reference to FIG. 7. FIG. 7 is a flow diagram showing the operation of a second example of the information processing device. In the following description, reference will be made to FIGS. 5 and 6 as appropriate. In addition, in the second embodiment, an information processing method is implemented by operating the information processing device 30. Therefore, the description of the information processing method will be replaced by the description of the operation of the information processing device 30 below.

[0054] First, as a premise, audio data and object information are stored in the storage unit 16. Also, the display control unit 11 displays a user interface 21 on the screen of the display device 20 (see FIG. 3).

[0055] As shown in FIG. 7, first, when the user clicks the voice playback button 23, the voice recognition unit 14 executes voice recognition processing on the voice data stored in the storage unit 16 (step B1).

[0056] Next, the document summarizing unit 31 performs document summarizing on the document obtained by the speech recognition processing in step B1, and generates text information using the processing result (step B2). The document summarizing unit 31 stores the generated text information in the storage unit 16.

[0057] Next, the display control unit 11 displays the text information in the text window 24 and the object information in the object list 22 in the user interface 21 (step B3).

[0058] Next, the image generating unit 13 uses the text information to cause the image generating AI 32 to generate an initial image, and displays the initial image in the image window 25 (step B4).

[0059] Next, the input receiving unit 12 receives the user's selection of an object and the designation of its position, and notifies the image generating unit 13 of the selected object and its position (step B5).

[0060] Next, the image generating unit 13 places the object selected in step B5 at the specified position in the initial image displayed in step B4 in the image window 25 (step B6).

[0061] Next, the input receiving unit 12 determines whether or not the user has input a correction to the text information (step B7). If the result of the determination in step B7 is that a correction has been input, the input receiving unit 12 corrects the text information stored in the storage unit 16 in accordance with the received correction (step B8).

[0062] Next, the display control unit 11 reads out the text information corrected in step B8 from the storage unit 16 and updates the text window 24 (step B9).

[0063] Next, the image generation unit 13 uses the text information corrected in step B8 to have the image generation AI 32 regenerate the initial image, and updates the image in the image window 25 with the newly generated initial image (step B10).

[0064] If the result of the determination in step B7 is that no correction has been input or if step B10 has been executed, the input receiving unit 12 determines whether or not the user has instructed to end the process (step B11). If the result of the determination in step B11 is that the user has not instructed to end the process, step B5 is executed again.

[0065] On the other hand, if the result of the determination in step B11 indicates that the user has instructed to end the process, the image generator 13 sets the image in the image window 25 as the generated image (step B12). The image generator 13 also outputs the displayed text information and image data of the generated image to the training data generator 15.

[0066] Thereafter, the training data generating unit 15 outputs the output text information and the image data of the generated image to the outside as training data for learning (step B13).

[0067] [Effects of Embodiment 2] As described above, Embodiment 2 can also assist the user in generating images according to text. Furthermore, Embodiment 2 is also particularly useful when creating training data for a generation AI that generates images from text information, and reduces the time and effort required to create the training data.

[0068] Furthermore, in the second embodiment, the images are automatically corrected in response to corrections to the text information, which further reduces the time and effort required to create training data. Also, the modifications described in the first embodiment can be applied to the second embodiment.

[0069] [Program] In the second embodiment, the program may be any program that causes a computer to execute steps B1 to B13 shown in Fig. 7. By installing and executing this program on a computer, the information processing device 30 and the information processing method can be realized. In this case, the processor of the computer functions as the display control unit 11, the input receiving unit 12, the image generation unit 13, the speech recognition unit 14, the training data generation unit 15, and the document summarization unit 31 and performs the processing.

[0070] In the second embodiment, the storage unit 16 may be implemented by a storage device such as a hard disk provided in the computer, or may be implemented by a storage device of another computer. Examples of the computer include a general-purpose PC, a server computer, a smartphone, and a tablet terminal device.

[0071] In the second embodiment, the program may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as one of the display control unit 11, the input receiving unit 12, the image generation unit 13, the voice recognition unit 14, the training data generation unit 15, and the document summarization unit 31.

[0072] Third Embodiment Next, a third example of an information processing device, an information processing method, and a program according to a third embodiment will be described with reference to FIGS.

[0073] [Device Configuration] First, the configuration of the third example of the information processing device will be described with reference to Fig. 8. Fig. 8 is a diagram showing the configuration of the third example of the information processing device.

[0074] 8, information processing device 40 differs from the information processing device described in embodiment 1 in that it includes an object generation unit 41 and a text information correction unit 42, and in the function of image generation unit 13. The following description will focus on the differences from embodiment 1.

[0075] First, the object generation unit 41 extracts named entities from the text information, and then uses the extracted named entities to generate a group of objects using the image generation AI 43. The object generation unit 41 also stores the generated group of objects in the storage unit 16 as object information.

[0076] 8, the image generation AI 43 is located outside the information processing device 40 and is connected to the information device 40 via a network or the like so as to be able to communicate data with the information device 40. The image generation AI 43 may also be provided in the information device 40.

[0077] The image generation AI 43 is a machine learning model trained to generate images from text. The image generation AI 43 may be the same as the image generation AI 32. The image generation AI 43 may also be an image generation AI (see references) that generates vector images from text.

[0078] The image generation AI 43 may also be an image generation AI that generates a raster image, similar to the image generation AI 32. Furthermore, the image generation AI 43 may also be a machine learning model that is the subject of training using training data generated by the training data generation unit 15, or may also be a machine learning model that is not the subject of training.

[0079] The input receiving unit 12 can also receive corrections to the generated image. In this case, the image generating unit 13 corrects the image in the image window 25 in accordance with the received corrections.

[0080] When the input receiving unit 12 receives a correction to an image, the text information correcting unit 42 corrects the text information in accordance with the correction to the image and updates the text information stored in the storage unit 16. Specifically, the text information correcting unit 42 inputs the corrected image into a machine learning model that uses the text as a target variable and the image as an explanatory variable, and corrects the text information using the output text.

[0081] Updating an image by correcting text information will be described with reference to Fig. 9. Fig. 9 is a diagram showing a third example of an operation screen of an information processing device. Fig. 9 shows an example in which the image has been corrected from the example shown in Fig. 3 and the text information has been updated accordingly.

[0082] 9, the image generation unit 13, in response to the instruction, changes one of the black cars to a white car in the image window 25 and updates the image (top row in FIG. 9). As a result, "Two black cars collided at the intersection" in the text window 24 is changed to "A white car and a black car collided at the intersection" and the text information is updated (bottom row in FIG. 9).

[0083] [Device Operation] Next, the operation of the information processing device will be described with reference to FIG. 10. FIG. 10 is a flow diagram showing the operation of a third example of the information processing device. In the following description, reference will be made to FIGS. 8 and 9 as appropriate. In addition, in the third embodiment, an information processing method is implemented by operating the information processing device 40. Therefore, the description of the information processing method will be replaced by the following description of the operation of the information processing device 40.

[0084] First, as a premise, audio data is stored in the storage unit 16. Also, the display control unit 11 displays a user interface 21 on the screen of the display device 20 (see FIG. 3).

[0085] 10 , when the user first clicks the voice playback button 23, the voice recognition unit 14 executes a voice recognition process on the voice data stored in the storage unit 16 to generate text information (step C1). The voice recognition unit 14 also stores the generated text information in the storage unit 16.

[0086] Next, the object generation unit 41 extracts named entities from the text information, and causes the image generation AI 43 to generate a group of objects using the extracted named entities (step C2). The object generation unit 41 also stores the generated group of objects in the storage unit 16 as object information.

[0087] Next, the display control unit 11 displays the text information in the text window 24 and the object information in the object list 22 in the user interface 21 (step C3).

[0088] Next, the input receiving unit 12 receives the user's selection of an object and the designation of its position, and notifies the image generating unit 13 of the selected object and its position (step C4).

[0089] Next, the image generating unit 13 places the object selected in step C4 at the specified position in the image window 25 (step C5).

[0090] Next, the input receiving unit 12 determines whether or not the user has input any image corrections (step C6). If the result of the determination in step C6 is that corrections have been input, the image generating unit 13 corrects the image in accordance with the input corrections and updates the image in the image window 25 (step C7).

[0091] Next, the text information correcting section 42 corrects the text information in accordance with the correction of the input image, and updates the text information stored in the storage section 16 (step C8).

[0092] Next, the display control unit 11 reads out the text information corrected in step C6 from the storage unit 16 and updates the text window 24 (step C9).

[0093] If the result of the determination in step C6 is that no correction has been input or if step C9 has been executed, the input receiving unit 12 determines whether or not the user has instructed to end the process (step C10). If the result of the determination in step C10 is that the user has not instructed to end the process, step C4 is executed again.

[0094] On the other hand, if the result of the determination in step C10 indicates that the user has instructed to end the process, the image generator 13 sets the image in the image window 25 as the generated image (step C11). The image generator 13 also outputs the displayed text information and the image data of the generated image to the training data generator 15.

[0095] Thereafter, the training data generating unit 15 outputs the output text information and the image data of the generated image to the outside as training data for learning (step C12).

[0096] [Effects of Embodiment 3] As described above, Embodiment 3 can also assist the user in generating images according to text. Furthermore, Embodiment 3 is also particularly useful when creating training data for a generation AI that generates images from text information, and reduces the time and effort required to create the training data.

[0097] Furthermore, in the third embodiment, appropriate objects are prepared according to the input voice data, and further, text is automatically corrected according to image correction, further reducing the time and effort required to create training data. Also, the modifications described in the first embodiment can be applied to the third embodiment.

[0098] [Program] In the third embodiment, the program may be any program that causes a computer to execute steps C1 to C12 shown in Fig. 10. By installing and executing this program on a computer, the information processing device 40 and the information processing method can be realized. In this case, the processor of the computer functions as the display control unit 11, the input receiving unit 12, the image generation unit 13, the voice recognition unit 14, the training data generation unit 15, the object generation unit 41, and the text information correction unit 42, and performs the processing.

[0099] In the third embodiment, the storage unit 16 may be implemented by a storage device such as a hard disk provided in the computer, or may be implemented by a storage device of another computer. Examples of the computer include a general-purpose PC, a server computer, a smartphone, and a tablet terminal device.

[0100] In addition, in the third embodiment, the program may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as one of the display control unit 11, the input receiving unit 12, the image generation unit 13, the voice recognition unit 14, the training data generation unit 15, the object generation unit 41, and the text information correction unit 42.

[0101] [Physical Configuration] A computer that implements an information processing device by executing the programs in the first to third embodiments will now be described with reference to Fig. 11. Fig. 11 is a block diagram showing an example of a computer that implements an information processing device.

[0102] 11, the computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. These components are connected to each other via a bus 121 so as to be able to communicate data with each other.

[0103] Furthermore, the computer 110 may include a GPU (Graphics Processing Unit) or an FPGA (Field-Programmable Gate Array) in addition to or instead of the CPU 111. In this aspect, the GPU or FPGA can execute the programs in the embodiments.

[0104] The CPU 111 loads a program in the embodiment, which is composed of a group of codes and stored in the storage device 113, into the main memory 112 and executes each code in a predetermined order to perform various calculations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory).

[0105] The program in the embodiment is provided in a state stored in a computer-readable recording medium 120. The program in the embodiment may be distributed over the Internet connected via the communication interface 117.

[0106] Specific examples of the storage device 113 include a hard disk drive and a semiconductor storage device such as a flash memory. The input interface 114 mediates data transmission between the CPU 111 and input devices 118 such as a keyboard and a mouse. The display controller 115 is connected to a display device 119 and controls the display on the display device 119.

[0107] The data reader / writer 116 mediates data transmission between the CPU 111 and the recording medium 120, reads programs from the recording medium 120, and writes processing results from the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and other computers.

[0108] Specific examples of the recording medium 120 include general-purpose semiconductor storage devices such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), magnetic recording media such as flexible disks, or optical recording media such as CD-ROMs (Compact Disk Read Only Memory).

[0109] The information processing device may be realized not by a computer with a program installed, but by hardware corresponding to each unit, such as an electronic circuit. Furthermore, the information processing device may be realized in part by a program and in the remaining part by hardware. In the embodiments, the computer is not limited to the computer shown in FIG. 11 .

[0110] Some or all of the above-described embodiments can be expressed by (Supplementary Note 1) to (Supplementary Note 27) described below, but are not limited to the following descriptions.

[0111] (Supplementary Note 1) An information processing device comprising: a display control unit that displays text information and a group of objects that constitute an image on a screen; an input receiving unit that receives a selection of an object from the group of objects and a designation of a position on the screen of the selected object; and an image generating unit that places the selected object at a designated position on the screen and generates an image composed of the selected objects.

[0112] (Supplementary Note 2) The information processing device according to Supplementary Note 1, further comprising: a training data generation unit that generates the text information and image data of the generated image as training data for learning.

[0113] (Supplementary Note 3) The information processing device according to Supplementary Note 1, wherein the input accepting unit accepts a correction to the text information, and the image generating unit corrects the generated image in accordance with the accepted correction to the text information.

[0114] (Supplementary Note 4) The information processing device according to Supplementary Note 1, wherein the image generation unit uses the text information to cause an image generation AI that has been machine-learned to generate images from text to generate an initial image, and places the selected object on the generated initial image at a specified position on the screen to generate the image.

[0115] (Supplementary Note 5) The information processing device according to Supplementary Note 1, further comprising an object generation unit that extracts named entities from the text information and causes an image generation AI to generate the group of objects using the extracted named entities.

[0116] (Supplementary Note 6) The information processing device according to Supplementary Note 1, wherein the input accepting unit accepts a correction to the generated image, and the information processing device further includes a text information correcting unit that corrects the text information in accordance with the accepted correction to the image.

[0117] (Supplementary Note 7) The information processing device according to Supplementary Note 1, wherein the input accepting unit further accepts input of motion information necessary to move the object in the generated image, and the image generating unit generates, as the image, a video in which the object moves based on the motion information.

[0118] (Supplementary Note 8) The information processing device according to Supplementary Note 1, wherein the text information includes text data generated by a voice recognition process.

[0119] (Supplementary Note 9) The information processing device according to Supplementary Note 1, further comprising: a document summarizing unit that summarizes a sentence and generates the text information.

[0120] (Supplementary Note 10) An information processing method comprising: a display control step of displaying text information and a group of objects for constituting an image on a screen; an input receiving step of selecting an object from the group of objects and receiving a designation of a position on the screen of the selected object; and an image generating step of arranging the selected object at a designated position on the screen and generating an image composed of the selected objects.

[0121] (Supplementary Note 11) The information processing method according to Supplementary Note 10, further comprising a training data generating step of generating the text information and image data of the generated image as training data for learning.

[0122] (Supplementary Note 12) The information processing method according to Supplementary Note 10, wherein the input receiving step receives a correction to the text information, and the image generating step corrects the generated image in accordance with the received correction to the text information.

[0123] (Supplementary Note 13) The information processing method according to Supplementary Note 10, wherein in the image generating step, an initial image is generated by an image generation AI that has been machine-learned to generate images from text using the text information, and the selected object is placed at a specified position on the screen on the generated initial image to generate the image.

[0124] (Supplementary Note 14) The information processing method according to Supplementary Note 10, further comprising an object generation step of extracting named entities from the text information and causing an image generation AI to generate the group of objects using the extracted named entities.

[0125] (Supplementary Note 15) The information processing method according to Supplementary Note 10, wherein the input receiving step receives a correction to the generated image, and the information processing method further includes a text information correcting step of correcting the text information in accordance with the received correction to the image.

[0126] (Supplementary Note 16) The information processing method according to Supplementary Note 10, wherein the input receiving step further receives input of motion information required to move the object in the generated image, and the image generating step generates a video as the image in which the object moves based on the motion information.

[0127] (Supplementary Note 17) The information processing method according to Supplementary Note 10, wherein the text information includes text data generated by a voice recognition process.

[0128] (Supplementary Note 18) The information processing method according to Supplementary Note 10, further comprising a document summarizing step of summarizing a sentence to generate the text information.

[0129] (Supplementary Note 19) A computer-readable recording medium having recorded thereon a program including instructions for causing a computer to execute the following steps: a display control step of displaying text information and a group of objects for constituting an image on a screen; an input receiving step of selecting an object from the group of objects and receiving designation of a position on the screen of the selected object; and an image generating step of arranging the selected object at a designated position on the screen and generating an image composed of the selected objects.

[0130] (Supplementary Note 20) The computer-readable recording medium according to Supplementary Note 19, wherein the program further includes instructions for causing the computer to execute a training data generation step of generating the text information and image data of the generated image as training data for learning.

[0131] (Supplementary Note 21) The computer-readable recording medium according to Supplementary Note 19, wherein the input receiving step receives a correction to the text information, and the image generating step corrects the generated image in accordance with the received correction to the text information.

[0132] (Supplementary Note 22) The computer-readable recording medium according to Supplementary Note 19, wherein in the image generating step, an image generation AI that has been trained to generate images from text is caused to generate an initial image using the text information, and the selected object is placed at a specified position on the screen on the generated initial image to generate the image.

[0133] (Supplementary Note 23) The computer-readable recording medium according to Supplementary Note 19, wherein the program further includes instructions for causing the computer to execute an object generation step of extracting named entities from the text information and causing an image generation AI to generate the group of objects using the extracted named entities.

[0134] (Supplementary Note 24) The computer-readable recording medium according to Supplementary Note 19, wherein the input receiving step receives a correction to the generated image, and the computer further executes a text information correcting step of correcting the text information in accordance with the received correction to the image.

[0135] (Supplementary Note 25) The computer-readable recording medium according to Supplementary Note 19, wherein the input receiving step further receives input of motion information required to move the object in the generated image, and the image generating step generates a video as the image in which the object moves based on the motion information.

[0136] (Supplementary Note 26) The computer-readable storage medium according to Supplementary Note 19, wherein the text information includes text data generated by a speech recognition process.

[0137] (Supplementary Note 27) The computer-readable recording medium according to Supplementary Note 19, wherein the program further includes instructions for causing the computer to execute a document summarizing step of summarizing a sentence to generate the text information.

[0138] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0139] This application claims priority based on Japanese Patent Application No. 2024-133717, filed August 9, 2024, the disclosure of which is incorporated herein in its entirety by reference.

[0140] As described above, the present disclosure can assist a user in generating images according to text, and is particularly useful in fields that require the creation of training data that combines text and images.

[0141] REFERENCE SIGNS LIST 10 Information processing device (first embodiment) 11 Display control unit 12 Input acceptance unit 13 Image generation unit 14 Speech recognition unit 15 Training data generation unit 20 Display device (second embodiment) 21 User interface 22 Object list 23 Speech playback button 24 Text window 25 Image window 30 Information processing device (third embodiment) 31 Document summary unit 32 Image generation AI 40 Information processing device 41 Object generation unit 42 Text information correction unit 43 Image generation AI 110 Computer 111 CPU 112 Main memory 113 Storage device 114 Input interface 115 Display controller 116 Data reader / writer 117 Communication interface 118 Input device 119 Display device 120 Recording medium 121 Bus

Claims

1. An information processing device comprising: a display control means for displaying text information and a group of objects for constituting an image on a screen; an input receiving means for receiving a selection of an object from the group of objects and a specification of the position of the selected object on the screen; and an image generating means for placing the selected object at a specified position on the screen and generating an image composed of the selected objects.

2. The information processing device according to claim 1, further comprising training data generation means for generating the text information and image data of the generated image as training data for learning.

3. The information processing device according to claim 1, wherein the input accepting means accepts a correction to the text information, and the image generating means corrects the generated image in accordance with the accepted correction to the text information.

4. The information processing device according to claim 1, wherein the image generation means uses the text information to cause an image generation AI that has been machine-trained to generate images from text to generate an initial image, and places the selected object on the generated initial image at a specified position on the screen to generate the image.

5. The information processing device according to claim 1, further comprising an object generation means for extracting named entities from the text information and causing an image generation AI to generate the group of objects using the extracted named entities.

6. The information processing device according to claim 1, wherein the input accepting means accepts modifications to the generated image, and the information processing device further comprises text information correcting means for correcting the text information in accordance with the accepted modifications to the image.

7. An information processing device as described in claim 1, wherein the input accepting means further accepts input of motion information necessary to move the object in the generated image, and the image generating means generates, as the image, a video in which the object moves based on the motion information.

8. The information processing device according to claim 1, wherein the text information includes text data generated by a voice recognition process.

9. An information processing method comprising: displaying text information and a group of objects that constitute an image on a screen; accepting a selection of an object from the group of objects and a specification of a position on the screen of the selected object; arranging the selected object at the specified position on the screen; and generating an image composed of the selected objects.

10. A computer-readable recording medium having recorded thereon a program including instructions for causing a computer to display, on a screen, text information and a group of objects that constitute an image, accept the selection of an object from the group of objects and the specification of the position of the selected object on the screen, place the selected object at the specified position on the screen, and generate an image composed of the selected objects.

Images