Diary image generation apparatus, diary image generation method, and diary image generation program

Abstract

Provided is a diary image generation apparatus comprising: a diary text data acquisition unit that acquires information indicating diary text posted by a user; a keyword extraction unit that extracts a key word or phrase from the diary text; an emotion identification unit that identifies an emotion of the user on the basis of the key word or phrase; an input prompt generation unit that inputs the key word or phrase and the information indicating the emotion of the user to an input prompt generative model, thereby generating an input prompt for instructing a visual expression generative model so that a diary image, which is a picture corresponding to the diary text, is generated; a diary image generation unit that inputs the input prompt to the visual expression generative model, thereby generating a diary image; and an output unit that outputs the diary image. The input prompt generative model is a trained model that has undergone machine learning so as to output an input prompt in response to input of a key word or phrase which is extracted from diary text and information which indicates an emotion. The visual expression generative model is a trained model that has undergone machine learning so as to output a diary image in response to input of an input prompt.

Description

Diary image generation apparatus, diary image generation method, and diary image generation program

[0001] The present invention relates to a diary image generation apparatus, a diary image generation method, and a diary image generation program.

[0002] Communication services on networks including SNS (Social Networking Service) have become widespread, making it possible to easily record daily events and share them with others. As one of the technologies for realizing such communication services, there is a diary application that can record personal diaries on a network (see, for example, Non-Patent Document 1).

[0003] "Free Online Journal & Diary App Journey. Cloud," JOURNEY (registered trademark), Two App Studio Pte. Ltd., [searched March 6, 2025], Internet (URL: https: / / journey.cloud / )

[0004] In communication services on networks, information posted by a poster can be easily browsed by others. Depending on the service, an unspecified large number of people may easily access the posted information. Therefore, it is essential for posters to pay close attention not to post personally identifiable information. In addition, the more detailed texts and more images a poster posts, the more specific and closer the communication between the poster and others who respond to the post tends to become. As a result, in recent years, the psychological burden on posters in communication services has increased, and there is a problem that communication fatigue such as SNS fatigue is likely to occur, for example.

[0005] In view of the above circumstances, an object of the present invention is to provide a technique capable of reducing the psychological burden on diary posters.

[0006] One aspect of the present invention is a diary image generation device comprising: a diary text data acquisition unit that acquires information indicating diary entries posted by a user; a keyword extraction unit that extracts key words from the diary entries; an emotion identification unit that identifies the user's emotions based on the key words; an input instruction generation unit that acquires an input instruction generation model and generates an input instruction to instruct a visual representation generation model to generate a diary image, which is a picture corresponding to the diary entries, by inputting the key words and information indicating the user's emotions into the input instruction generation model; a diary image generation unit that acquires the visual representation generation model and generates the diary image by inputting the input instruction to the visual representation generation model; and an output unit that outputs the diary image, wherein the input instruction generation model is a machine learning model that outputs the input instruction in response to inputs of the key words extracted from the diary entries and information indicating the emotions, and the visual representation generation model is a machine learning model that outputs the diary image in response to inputs of the input instruction.

[0007] Furthermore, one aspect of the present invention is a computer-based method for generating diary images, comprising: a diary text data acquisition step of acquiring information indicating diary entries posted by a user; a keyword extraction step of extracting key words from the diary entries; an emotion identification step of identifying the user's emotions based on the key words; an input instruction generation step of acquiring an input instruction generation model and inputting the key words and information indicating the user's emotions into the input instruction generation model to generate an input instruction for instructing a visual representation generation model to generate a diary image, which is a picture corresponding to the diary entries; a diary image generation step of acquiring the visual representation generation model and inputting the input instruction into the visual representation generation model to generate the diary image; and an output step of outputting the diary image, wherein the input instruction generation model is a machine learning model that outputs the input instruction in response to inputs of the key words extracted from the diary entries and information indicating the emotions, and the visual representation generation model is a machine learning model that outputs the diary image in response to inputs of the input instruction.

[0008] Furthermore, one aspect of the present invention is a diary image generation program for causing a computer to perform the following steps: acquiring diary text data to acquire information indicating diary entries posted by a user; extracting key words from the diary entries; identifying the user's emotions based on the key words; acquiring an input instruction generation model and inputting the key words and information indicating the user's emotions into the input instruction generation model to generate an input instruction for instructing a visual representation generation model to generate a diary image, which is a picture corresponding to the diary entries; acquiring the visual representation generation model and inputting the input instruction into the visual representation generation model to generate the diary image; and outputting the diary image, wherein the input instruction generation model is a machine learning model that outputs the input instruction in response to inputs of the key words extracted from the diary entries and information indicating the emotions, and the visual representation generation model is a machine learning model that outputs the diary image in response to inputs of the input instruction.

[0009] This invention makes it possible to reduce the psychological burden on diary writers.

[0010] This is a schematic block diagram showing the overall configuration of the picture diary generation system 1 in one embodiment of the present invention. This is a schematic block diagram showing the functional configuration of the AI ​​processing server 10 in one embodiment of the present invention. This is a block diagram showing the functional configuration of the control unit 15 of the AI ​​processing server 10 in one embodiment of the present invention. This is a schematic block diagram showing the functional configuration of the storage server 20 in one embodiment of the present invention. This is a schematic block diagram showing the functional configuration of the API server 30 in one embodiment of the present invention. This is a schematic block diagram showing the functional configuration of the terminal device 40 in one embodiment of the present invention. This is a block diagram showing the functional configuration of the control unit 45 of the terminal device 40 in one embodiment of the present invention. This is a diagram showing the style selection screen Sa displayed on the terminal device 40 in one embodiment of the present invention. This is a diagram showing the diary creation screen Sb displayed on the terminal device 40 in one embodiment of the present invention. This is a diagram showing the removal target phrase registration screen Sc displayed on the terminal device 40 in one embodiment of the present invention. This is a diagram showing the correction location specification screen Sd displayed on the terminal device 40 in one embodiment of the present invention. This is a schematic block diagram showing the functional configuration of the authentication server 50 in one embodiment of the present invention. This is a flowchart showing the operation of the AI ​​processing server 10 when generating a diary image in one embodiment of the present invention. This is a flowchart showing the operation of the AI ​​processing server 10 when removing specific text in one embodiment of the present invention. This is a flowchart showing the operation of the AI ​​processing server 10 when removing a specific image area in one embodiment of the present invention. This flowchart shows the operation of the AI ​​processing server 10 when enhancing anonymity in one embodiment of the present invention. This flowchart shows the operation of the AI ​​processing server 10 when generating feedback text in one embodiment of the present invention. This flowchart shows the operation of the terminal device 40 when selecting a style in one embodiment of the present invention. This flowchart shows the operation of the terminal device 40 when creating a diary in one embodiment of the present invention. This flowchart shows the operation of the terminal device 40 when registering words to be removed in one embodiment of the present invention. This flowchart shows the operation of the terminal device 40 when specifying correction locations in one embodiment of the present invention. This flowchart shows the operation of the terminal device 40 when displaying diary images and feedback text in one embodiment of the present invention.This figure shows a schematic example of the hardware configuration of an information processing device 90 applied to one embodiment of the present invention.

[0011] Hereinafter, one embodiment of the diary image generation device, diary image generation method, and diary image generation program of the present invention will be described in detail with reference to the drawings.

[0012] The picture diary generation system 1 in one embodiment of the present invention described below is an information system that provides a communication platform that allows users (posters) to post picture diaries on a network and share them with others. Hereinafter, the service provided by the communication platform realized by the picture diary generation system 1 will be referred to as the "picture diary generation service". The picture diaries published by the picture diary generation service are not the information posted by the poster, but rather the pictures of the picture diary (hereinafter referred to as "diary images") that are automatically generated based on the text-format diary data posted by the poster (hereinafter referred to as "diary text data").

[0013] More specifically, the picture diary generation system 1 analyzes the input diary text data and generates a diary image representing the analyzed diary content in a pre-specified style (hereinafter referred to as "style"). Furthermore, the picture diary generation system 1 analyzes the generated diary image and detects image regions within the diary image that could lead to the identification of an individual (poster). The picture diary generation system 1 corrects the generated diary image to ensure anonymity by converting or deleting the detected image regions.

[0014] Thus, according to the picture diary generation system 1 of this embodiment, the contents of the diary are automatically converted from text data into pictures (images), and the anonymity of the images is ensured. As a result, the poster does not have to worry about being identified, and their psychological burden is reduced.

[0015] [Overall Configuration of the Picture Diary Generation System] The overall configuration of the picture diary generation system 1 in one embodiment of the present invention will be described below. Figure 1 is a schematic block diagram showing the overall configuration of the picture diary generation system 1 in one embodiment of the present invention. As shown in Figure 1, the picture diary generation system 1 is composed of an AI (Artificial Intelligence) processing server 10, a storage server 20, an API (Application Programming Interface) server 30, at least one terminal device 40, an authentication server 50, and a network 80.

[0016] The AI ​​processing server 10 is an example of the diary image generation device of the present invention. The AI ​​processing server 10 is a server device that performs processing such as text analysis of diary text data, generation of diary images, and image conversion to ensure anonymity. The AI ​​processing server 10 is configured using an information processing device such as a general-purpose computer. The AI ​​processing server 10 is connected to the storage server 20 and the API server 30, respectively.

[0017] The storage server 20 is a server device that stores diary images generated by the AI ​​processing server 10, diary text data entered by the poster, and various data and information indicating various execution conditions used in the processing performed by the AI ​​processing server 10. The storage server 20 is configured using an information processing device such as a general-purpose computer. The storage server 20 is connected to the AI ​​processing server 10.

[0018] The API server 30 is a server device that provides an interface (API) for relaying various data and function exchanges between the AI ​​processing server 10 and the terminal devices 40. The API server 30 is configured using, for example, an information processing device such as a general-purpose computer. The API server 30 is connected to the AI ​​processing server 10. The API server 30 is also connected to multiple terminal devices 40 via the network 80.

[0019] Terminal device 40 is a terminal device used by users (posters and viewers) who use the picture diary generation service. Terminal device 40 is an information terminal such as a smartphone, tablet, or personal computer. An application for the picture diary generation service (hereinafter referred to as the "picture diary app") is installed on terminal device 40. Posters post diary text data using the picture diary app from terminal device 40. Terminal device 40 connects to the API server 30 and the authentication server 50 via the network 80.

[0020] The authentication server 50 is a server device that performs user authentication for users of the picture diary generation service. Alternatively, the authentication server 50 may be a server device that performs device authentication for the terminal device 40 used to access the picture diary generation service. The authentication server 50 is configured using, for example, an information processing device such as a general-purpose computer. The authentication server 50 is connected to the terminal device 40 via the network 80.

[0021] Network 80 is a communication network that transmits information exchanged between the API server 30 and the terminal device 40, and information exchanged between the terminal device 40 and the authentication server 50. Network 80 may be a communication network using wireless communication, or a communication network using wired communication. Network 80 may also be a communication network that uses a combination of wireless and wired communication. Network 80 may be configured using, for example, the Internet, or a LAN (Local Area Network). Furthermore, Network 80 may be configured by combining multiple types of communication networks.

[0022] Furthermore, the AI ​​processing server 10 and the storage server 20, and the AI ​​processing server 10 and the API server 30 may also be connected via the network 80.

[0023] [Configuration of the AI ​​Processing Server] The configuration of the AI ​​processing server 10 will be described in more detail below. Figure 2 is a schematic block diagram showing the functional configuration of the AI ​​processing server 10 in one embodiment of the present invention. As shown in Figure 2, the AI ​​processing server 10 is configured to include a communication unit 11, a storage unit 14, and a control unit 15.

[0024] The communication unit 11 is a communication device. The communication unit 11 communicates with other devices via a communication line in accordance with the control of the control unit 15. The communication unit 11 may be a device that performs wired communication or a device that performs wireless communication. The communication unit 11 may be configured as, for example, a network interface.

[0025] The storage unit 14 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 14 stores data used by the control unit 15. The storage unit 14 stores data necessary when the control unit 15 performs processing.

[0026] The memory unit 14 functions as a storage area for storing list data such as a keyword dictionary and a specific text registration list, as described later. The memory unit 14 also functions as a storage area for storing learning models such as a prompt generation model, a visual representation generation model, a specific text detection model, a specific image region detection model, an anonymity enhancement model, and a feedback generation model, as described later. These list data and learning models may be stored in the memory unit 24 of the storage server 20, as described later.

[0027] The control unit 15 is configured using a processor such as a CPU (Central Processing Unit) and memory (main memory). The control unit 15 functions, for example, as described later, when the processor executes a program, including the style information acquisition unit 151, diary text data acquisition unit 152, keyword extraction unit 153, emotion identification unit 154, prompt generation unit 155, diary image generation unit 156, specific text detection unit 157, specific text correction unit 158, specific image area detection unit 159, specific image area correction unit 160, anonymity enhancement unit 161, feedback generation unit 162, and diary image publication control unit 163.

[0028] Furthermore, all or part of the functions of the control unit 15 may be implemented using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The above program may be recorded on a computer-readable recording medium. Computer-readable recording media include, for example, portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and semiconductor memory devices (e.g., SSDs: Solid State Drives), as well as storage devices such as hard disks and semiconductor memory devices built into computer systems. The above program may also be transmitted via a telecommunications line.

[0029] The control unit 15 may, for example, execute an application installed on its own device (AI processing server 10). The control unit 15 operates according to the program of the application being executed.

[0030] One concrete example of such an application is an application provided to the AI ​​processing server 10 as a dedicated application for the picture diary generation system 1. Another concrete example of such an application is a web browser application. Such applications may be pre-installed on the AI ​​processing server 10, or they may be downloaded each time communication processing is performed.

[0031] For example, if implemented as a web browser application, the AI ​​processing server 10 may download and execute the application from a device specified by the web server (for example, the web server itself or another server) in response to the AI ​​processing server 10 connecting to a specific web server.

[0032] The functional configuration of the control unit 15 will be described in detail below. Figure 3 is a block diagram showing the functional configuration of the control unit 15 of the AI ​​processing server 10 in one embodiment of the present invention. As shown in Figure 3, the control unit 15 is composed of a style information acquisition unit 151, a diary text data acquisition unit 152, a keyword extraction unit 153, an emotion identification unit 154, a prompt generation unit 155, a diary image generation unit 156, a specific text detection unit 157, a specific text correction unit 158, a specific image area detection unit 159, a specific image area correction unit 160, an anonymity enhancement unit 161, a feedback generation unit 162, and a diary image publication control unit 163.

[0033] The style information acquisition unit 151 acquires style information that has been transmitted in advance from the terminal device 40 via the network 80 and received by the communication unit 11. The style information acquisition unit 151 outputs the acquired style information to the prompt generation unit 155. The style information acquisition unit 151 also controls the communication unit 11 to output the acquired style information to the storage server 20. The style information is stored in the storage server 20 in association with information identifying the diary poster. Alternatively, the style information may be stored in the storage server 20 in association with information identifying the terminal device 40 used by the poster.

[0034] The style information referred to here is information that indicates the style of the diary images (illustrations in the picture diary), which is specified in advance by the diary poster. This style information may include, for example, "childlike style," "mature style," and "cartoon-like style." By specifying the style on the picture diary application running on the terminal device 40, the poster can have the picture diary generation service generate illustrations for their picture diary in the desired style.

[0035] If a "childlike drawing style" is specified, the AI ​​processing server 10 will generate a diary image with bright colors, for example, drawn with relatively simple lines and with faces of people drawn slightly larger than their bodies. If a "mature drawing style" is specified, the AI ​​processing server 10 will generate a diary image with calm colors, for example, that is realistic and has detailed depictions. If a "cartoon-like drawing style" is specified, the AI ​​processing server 10 will generate a diary image with vibrant colors and a sense of dynamism. If a "cartoon-like drawing style" is specified, the AI ​​processing server 10 may also generate a monochrome diary image, similar to a manga volume.

[0036] The diary text data acquisition unit 152 acquires diary text data transmitted from the terminal device 40 via the network 80 and received by the communication unit 11. The diary text data acquisition unit 152 outputs the acquired diary text data to the keyword extraction unit 153 and the feedback generation unit 162. The diary text data acquisition unit 152 also controls the communication unit 11 to output the acquired diary text data to the storage server 20. The diary text data is stored in the storage server 20, linked to information identifying the diary's author. Alternatively, the diary text data may be stored in the storage server 20, linked to information identifying the terminal device 40 used by the author.

[0037] The diary text data referred to here is, as mentioned above, diary data in text format posted by the poster. The poster generates the diary text data and posts it on the picture diary application running on the terminal device 40, thereby posting the diary text data to the picture diary generation service.

[0038] The keyword extraction unit 153 acquires the diary text data output from the diary text data acquisition unit 152. The keyword extraction unit 153 performs natural language processing on the acquired diary text data and extracts keywords contained in the diary text data. For example, the keyword extraction unit 153 performs linguistic analysis such as morphological analysis, which divides a sentence into individual words and determines the part of speech; syntactic analysis, which analyzes grammatical structures such as subjects and predicates; semantic analysis, which understands the meaning of sentences and words; document classification, which divides sentences into categories; and machine translation, which translates them into other languages.

[0039] The keyword extraction unit 153 analyzes the diary text data using a method for evaluating the importance of words contained in a document, such as TF-IDF (Term Frequency - Inverse Document Frequency). The keyword extraction unit 153 extracts the main keywords contained in the diary text data based on the analyzed importance. Any method can be used to extract the main (highly important) keywords from the diary text data.

[0040] Furthermore, the keyword extraction unit 153 outputs information indicating the extracted keywords (hereinafter referred to as "extracted keywords") to the emotion identification unit 154 and the prompt generation unit 155. The keyword extraction unit 153 also controls the communication unit 11 and outputs information indicating the extracted keywords to the storage server 20. The extracted keywords are stored in the storage server 20 in association with information identifying the diary poster. Alternatively, the extracted keywords may be stored in the storage server 20 in association with information identifying the terminal device 40 used by the poster.

[0041] The emotion identification unit 154 acquires information indicating the extracted keywords output from the keyword extraction unit 153. Based on the acquired information, the emotion identification unit 154 extracts keywords representing emotions (hereinafter referred to as "emotion keywords") from the extracted keywords. Alternatively, the emotion identification unit 154 may acquire diary text data from the diary text data acquisition unit 152 and extract emotion keywords from the entire diary text data.

[0042] The emotion specifying unit 154 reads a keyword dictionary stored in advance in the storage unit 14. The keyword dictionary as used herein is a dictionary in which emotions are associated with emotion keywords (that represent the emotions). The emotion specifying unit 154 collates the extracted emotion keywords with emotion keywords included in the keyword dictionary, and specifies the emotion of the poster in the diary. The keyword dictionary may be stored in the storage server 20, and the emotion specifying unit 154 may be configured to read the keyword dictionary stored in advance in the storage server 20.

[0043] The emotion specifying unit 154 may be configured to acquire diary text data from the diary text data acquisition unit 152 and perform language analysis on the entire diary text data to estimate the emotion of the poster in the diary. For example, the emotion specifying unit 154 may estimate the emotion of the poster in the diary by using a learning model generated in advance through prior machine learning. The learning model in this case is, for example, a learning model generated by performing machine learning using teacher data in which training diary text data is associated with information indicating an emotion that is a correct label.

[0044] The emotion specifying unit 154 may perform both the specification of the poster's emotion performed based on extraction of emotion keywords and the estimation of the poster's emotion using machine learning.

[0045] The emotion specifying unit 154 outputs information indicating the specified (estimated) emotion (hereinafter referred to as "emotion specifying information") to the prompt generating unit 155 and the feedback generating unit 162. Further, the emotion specifying unit 154 controls the communication unit 11 to output the emotion specifying information to the storage server 20. The emotion specifying information is associated with information for identifying the diary poster and stored in the storage server 20. The emotion specifying information may be associated with information for identifying the terminal device 40 used by the poster and stored in the storage server 20.

[0046] The prompt generation unit 155 acquires the style information output from the style information acquisition unit 151. The prompt generation unit 155 also acquires information indicating the extracted keyword output from the keyword extraction unit 153. The prompt generation unit 155 also acquires the emotion identification information output from the emotion identification unit 154.

[0047] The prompt generation unit 155 also reads the prompt generation model stored in the storage unit 14. The prompt generation model referred to herein is a learning model that, when style information, information indicating an extracted keyword, and emotion identification information are input thereto, automatically generates and outputs a prompt (input instruction sentence) to be input to a visual expression generation model (generative AI) described later.

[0048] The prompt generation model is generated in advance by performing machine learning using teacher data in which, for example, training style information, information indicating an extracted keyword, emotion identification information, and a prompt serving as a correct label are associated with each other. Note that as the prompt generation model, the AI model "GPT-4 (Generative Pre-trained Transformer 4)" (registered trademark) provided by OpenAI, LLC may be used.

[0049] Note that the visual expression generation model to which the generated prompt is input is generative AI that generates a diary image (a picture in a picture diary) generated by a picture diary generation service. A description of the visual expression generation model will be given later.

[0050] The prompt generation unit 155 inputs the acquired style information, the information indicating the extracted keyword, and the emotion identification information into the read prompt generation model. Accordingly, the prompt generation unit 155 obtains a prompt output from the prompt generation model.

[0051] Two examples of prompts generated by the prompt generation model are described below.

[0052] (Example 1) ・Diary text data: "Today I had fun running and playing with my friends at the park." ・Input data style information: "Childish" Extracted keywords: "friends", "running", "park" Emotion identification information: "fun" ・Output data prompt: "A lively park depicted in bright colors. Children enjoying themselves running with friends. Childish style."

[0053] (Example 2) • Diary text data: "I wore a yukata and went to a summer festival. The fireworks were so beautiful and I had a great time." • Input data style information: "Childish" Extracted keywords: "Summer festival", "Fireworks", "Yukata" Emotion identification information: "Happy" • Output data prompt: "A cute, childlike illustration of a summer festival at night. Bright fireworks light up the sky, while children in colorful yukatas happily enjoy the festival. The atmosphere is warm and lively, filled with glowing lanterns and food stalls. The expressions of the characters are full of excitement and joy, capturing the fun and wonder of the moment."

[0054] The prompt generation unit 155 outputs information indicating the prompt output from the prompt generation model (hereinafter referred to as "prompt information") to the diary image generation unit 156. The prompt generation unit 155 also controls the communication unit 11 to output the prompt information to the storage server 20. The prompt information is stored in the storage server 20 in association with information identifying the diary poster. Alternatively, the prompt information may be stored in the storage server 20 in association with information identifying the terminal device 40 used by the poster.

[0055] The diary image generation unit 156 acquires prompt information output from the prompt generation unit 155. The diary image generation unit 156 also reads the visual representation generation model stored in the memory unit 14. The visual representation generation model referred to here is a learning model that, when a prompt is input, automatically generates and outputs diary images (pictures for the picture diary) that are generated by the picture diary generation service.

[0056] The visual representation generation model is pre-generated, for example, by machine learning using training data in which learning prompts are associated with diary images that serve as correct labels. Alternatively, the AI ​​model "GPT-4" (registered trademark) provided by OpenAI, LLC may be used as the visual representation generation model.

[0057] The diary image generation unit 156 inputs prompts based on the acquired prompt information to the read-out visual representation generation model. As a result, the diary image generation unit 156 obtains a diary image output from the visual representation generation model.

[0058] Below are examples of diary images generated by the visual representation generation model.

[0059] (Example) ・Input data prompt: "A cute, childlike illustration of a summer festival at night. Bright fireworks light up the sky, while children in colorful yukatas happily enjoy the festival. The atmosphere is warm and lively, filled with glowing lanterns and food stalls. The expressions of the characters are full of excitement and joy, capturing the fun and wonder of the moment." ・Output data diary image: (An image depicting the following scene) ・A summer festival venue at night. ・Fireworks in the sky. ・Children in yukatas are having fun. ・A cute drawing style with slightly rounded lines. ・An illustration with a soft touch.

[0060] The diary image generation unit 156 outputs the diary image data (hereinafter referred to as "diary image data") output from the visual representation generation model to the specific text detection unit 157. The diary image generation unit 156 also controls the communication unit 11 to output the diary image data to the storage server 20. The diary image data is stored in the storage server 20, linked to information identifying the diary's author. Alternatively, the diary image data may be stored in the storage server 20, linked to information identifying the terminal device 40 used by the author.

[0061] The specific text detection unit 157 acquires the diary image data output from the diary image generation unit 156. The specific text detection unit 157 also reads the specific text registration list stored in the storage unit 14. The specific text registration list is a list of words that should be removed from diary images, which have been registered in advance by users such as the poster. The specific text registration list includes words that indicate personal information such as names and contact information, as well as information that could lead to the identification of an individual, such as place names and school names.

[0062] The specific text detection unit 157 detects image regions in the diary image where text information (words or phrases) is written by performing image analysis. The specific text detection unit 157 reads the text information (words or phrases) from the detected image regions using, for example, OCR (Optical Character Recognition).

[0063] The specific text detection unit 157 compares the read text information (words) with the words included in the specific text registration list. If the read text information (words) is a word included in the specific text registration list, the specific text detection unit 157 determines that correction is necessary for the image area in which the text information (words) is depicted.

[0064] The specific text detection unit 157 outputs information indicating the image area that has been determined to require correction (for example, information indicated by coordinate values ​​within the image; hereinafter referred to as "corrected image area information") to the specific text correction unit 158. The specific text detection unit 157 also controls the communication unit 11 to output the corrected image area information to the storage server 20. The corrected image area information is stored in the storage server 20 in association with information identifying the diary poster. Alternatively, the corrected image area information may be stored in the storage server 20 in association with information identifying the terminal device 40 used by the poster.

[0065] Furthermore, the specific text detection unit 157 reads out the specific text detection model stored in the storage unit 14. The specific text detection model, as referred to here, is a learning model that, when an image is input, detects image regions in the image that contain text information (words) that could lead to the identification of an individual, and outputs information indicating the detected image region as corrected image region information.

[0066] Furthermore, the text information (words) that may lead to the identification of an individual, as used herein, is not limited to words that can identify an individual on their own, such as words that indicate personal information such as names, addresses, and telephone numbers. The text information (words) that may lead to the identification of an individual, as used herein, is not limited to these, and may also include words that indicate dates and times, places, etc., and words that indicate identifying information such as user IDs, which may lead to the identification of an individual when combined with other information.

[0067] The specific text detection model is pre-generated by machine learning using training data that associates diary images with information indicating image regions containing text information (words) that could lead to the identification of an individual, which are the correct labels (for example, information indicated by coordinate values ​​within the image). The training data may also include information indicating the text information (words). The AI ​​model "GPT-4" (registered trademark) provided by OpenAI, LLC may be used as the specific text detection model.

[0068] The specific text detection unit 157 inputs the diary image data output from the diary image generation unit 156 to the read specific text detection model. As a result, the specific text detection unit 157 obtains the corrected image region information output from the specific text detection model.

[0069] The specific text detection unit 157 outputs the diary image data output from the diary image generation unit 156 and the corrected image area information output from the specific text detection model to the specific text correction unit 158. The specific text detection unit 157 also controls the communication unit 11 to output the corrected image area information to the storage server 20. The corrected image area information is stored in the storage server 20 in association with information identifying the diary's author. Alternatively, the corrected image area information may be stored in the storage server 20 in association with information identifying the terminal device 40 used by the author.

[0070] The specific text correction unit 158 ​​acquires diary image data and corrected image area information output from the specific text detection unit 157. Based on the acquired diary image data and corrected image area information, the specific text correction unit 158 ​​identifies image areas in the diary images where text information (words) that could lead to the identification of an individual are depicted. The specific text correction unit 158 ​​corrects the identified image areas so as to remove the text information (words) depicted in those image areas.

[0071] Specifically, for example, the specific text correction unit 158 ​​performs image processing to remove text information (phrases) from the image region based on the acquired corrected image region information by blurring it, filling it with the surrounding background color, or replacing it with an image of arbitrary text information (phrases) that does not lead to the identification of an individual. Any image processing technique can be used for these image processing operations.

[0072] The specific text correction unit 158 ​​corrects the image area containing text information (phrases) that could lead to the identification of an individual, and outputs the diary image data (hereinafter referred to as "diary image data after specific text removal") with the text information (phrases) removed to the specific image area detection unit 159. The specific text correction unit 158 ​​also controls the communication unit 11 to output the diary image data after specific text removal to the storage server 20. The diary image data after specific text removal is stored in the storage server 20 in association with information that identifies the diary's author. Alternatively, the diary image data after specific text removal may be stored in the storage server 20 in association with information that identifies the terminal device 40 used by the author.

[0073] The specific text correction unit 158 ​​may also control the communication unit 11 to transmit image data of the image area containing text information (words) that could lead to the identification of an individual to the terminal device 40 via the network 80. Furthermore, the specific text correction unit 158 ​​may be controlled so that the poster can choose whether or not to remove the text information (words) depicted in the image of the image area. In this case, the specific text correction unit 158 ​​will only perform correction on the image area that the poster has determined needs to be removed.

[0074] The specific image region detection unit 159 acquires the image data after the specific text has been removed, which is output from the specific text correction unit 158. The specific image region detection unit 159 also reads the specific image region detection model stored in the storage unit 14. The specific image region detection model, as referred to here, is a learning model that, when an image is input, detects image regions in the image that may lead to the identification of an individual, and outputs information indicating the detected image region as corrected image region information.

[0075] In other words, while the aforementioned specific text detection unit 157 is configured to detect image areas in diary images that contain text information (words) that could lead to the identification of an individual, the specific image area detection unit 159 is configured to detect image areas in diary images that contain images that could lead to the identification of an individual. Images that could lead to the identification of an individual, as used here, include, for example, images showing faces, full bodies, clothing, and accessories (that could lead to the identification of an individual), and images showing buildings, landscapes, and logos (that could lead to the identification of an individual's location).

[0076] Furthermore, while the aforementioned specific text correction unit 158 ​​is configured to remove text information (words) that could lead to the identification of an individual from an image area in a diary image, the specific image area correction unit 160, described below, is configured to remove images that could lead to the identification of an individual from an image area in a diary image.

[0077] The specific image region detection model is pre-generated by machine learning using training data that associates diary images with information indicating image regions containing images that could lead to the identification of an individual (for example, information indicated by coordinate values ​​within the diary images). The training data may also include the images themselves that could lead to the identification of an individual. The AI ​​model "GPT-4" (registered trademark) provided by OpenAI, LLC may be used as the specific image region detection model.

[0078] The specific image region detection unit 159 inputs the diary image data output from the diary image generation unit 156 to the read specific image region detection model. As a result, the specific image region detection unit 159 obtains corrected image region information output from the specific image region detection model.

[0079] The specific image region detection unit 159 outputs the diary image data after specific text removal, output from the specific text correction unit 158, and the corrected image region information, output from the specific image region detection model, to the specific image region correction unit 160. The specific image region detection unit 159 also controls the communication unit 11 to output the corrected image region information to the storage server 20. The corrected image region information is stored in the storage server 20 in association with information identifying the diary poster. Alternatively, the corrected image region information may be stored in the storage server 20 in association with information identifying the terminal device 40 used by the poster.

[0080] The specific image area correction unit 160 acquires the diary image data after specific text removal and the corrected image area information output from the specific text correction unit 158. Based on the acquired diary image data after specific text removal and the corrected image area information, the specific image area correction unit 160 identifies an image area in the diary image that contains images that could lead to the identification of an individual. The specific image area correction unit 160 corrects the identified image area so as to remove any images that could lead to the identification of an individual from that image area.

[0081] Specifically, for example, the specific image area correction unit 160 performs image processing on the image area based on the acquired corrected image area information, such as blurring it, filling it with the surrounding background color, or replacing it with an image that does not lead to the identification of an individual. Any image processing technology can be used for these image processing operations.

[0082] For example, the specific image area correction unit 160 performs image processing on the image area based on the acquired corrected image area information, thereby correcting it to replace it with an image that does not lead to the identification of an individual, as follows.

[0083] (Correction Example 1) Before correction: Diary image depicting a real-life train station building or a real-life school building. After correction: Diary image depicting a generic-looking train station building or a generic-looking school building.

[0084] (Correction Example 2) Before correction: Diary image depicting a person with a distinctive hairstyle or wearing a yukata with a unique pattern. After correction: Diary image depicting a person with a simplified hairstyle or wearing a yukata with a simple pattern.

[0085] (Correction Example 3) Before correction: Diary image with a famous landmark in the background. After correction: Diary image with a typical streetscape and trees in the background.

[0086] The specific image area correction unit 160 corrects the image area containing images that could lead to the identification of an individual, and outputs the diary image data (hereinafter referred to as "diary image data after specific image area removal") with the images that could lead to the identification of the individual removed, to the anonymity enhancement unit 161. The specific image area correction unit 160 also controls the communication unit 11 to output the diary image data after specific image area removal to the storage server 20. The diary image data after specific image area removal is stored in the storage server 20 in association with information that identifies the diary's author. Alternatively, the diary image data after specific image area removal may be stored in the storage server 20 in association with information that identifies the terminal device 40 used by the author.

[0087] The specific image area correction unit 160 may also control the communication unit 11 to transmit image data of the image area containing images that could lead to the identification of an individual to the terminal device 40 via the network 80. The specific image area correction unit 160 may also be controlled so that the poster can choose whether or not to remove images that could lead to the identification of an individual contained in the image area. In this case, the specific image area correction unit 160 will only perform correction on the image area containing images that the poster has determined need to be removed.

[0088] The anonymity enhancement unit 161 acquires the diary image data after the removal of specific image regions, which is output from the specific image region correction unit 160. The anonymity enhancement unit 161 also reads the anonymity enhancement model stored in the memory unit 14. The anonymity enhancement model is a learning model that, when a diary image is input, outputs diary image data (hereinafter referred to as "anonymity-enhanced diary image data") in which image regions containing images that could lead to the identification of an individual are converted into image regions that do not lead to the identification of an individual. In other words, the anonymity enhancement model is a learning model specifically designed to enhance the anonymity of images.

[0089] The anonymity enhancement model is pre-generated by machine learning using training data that associates diary images with anonymized diary images that serve as the correct labels. The training data may also include images that could potentially identify individuals. The AI ​​model "GPT-4" (registered trademark) provided by OpenAI, LLC may be used as the anonymity enhancement model.

[0090] The anonymity enhancement unit 161 inputs the post-removal image data of the specific image region output from the specific image region correction unit 160 into the read-out anonymity enhancement model. As a result, the anonymity enhancement unit 161 obtains the post-removal image data of the anonymity enhancement model output from the anonymity enhancement model.

[0091] In this way, the anonymity enhancement unit 161 further uses an anonymity enhancement model to convert the post-image data after the removal of specific image region information (words) and images that could lead to the identification of an individual, which have been removed by the specific text correction unit 158 ​​and the specific image region correction unit 160, into post-image data after anonymity enhancement that is even more anonymized.

[0092] The anonymity enhancement unit 161 outputs the anonymity-enhanced diary image data output from the anonymity enhancement model to the feedback generation unit 162. The anonymity enhancement unit 161 also controls the communication unit 11 to output the anonymity-enhanced diary image data to the storage server 20. The anonymity-enhanced diary image data is stored in the storage server 20, linked to information identifying the diary's author. Alternatively, the anonymity-enhanced diary image data may be stored in the storage server 20, linked to information identifying the terminal device 40 used by the author.

[0093] The feedback generation unit 162 acquires the anonymity-enhanced diary image data output from the anonymity enhancement unit 161. The feedback generation unit 162 also acquires the diary text data output from the diary text data acquisition unit 152. The feedback generation unit 162 also acquires the emotion identification information output from the emotion identification unit 154. The feedback generation unit 162 also reads the feedback generation model stored in the memory unit 14.

[0094] The feedback generation model referred to here is a learning model that automatically generates and outputs feedback statements when diary text data and emotion identification information are input. Furthermore, the feedback statements referred to here are positive statements about the post that the picture diary generation service responds to the poster in response to the poster's diary text data submission.

[0095] The feedback generation model is pre-generated by machine learning using training data, for example, in which diary text data and sentiment identification information are associated with feedback sentences that serve as correct labels. Alternatively, the AI ​​model "GPT-4" (registered trademark) provided by OpenAI, LLC may be used as the prompt generation model.

[0096] Furthermore, the feedback generation model may be pre-generated by machine learning using training data that further includes templates for empathetic expressions. These templates for empathetic expressions are basic example sentences (standard phrases) corresponding to the type of response the picture diary generation service sends to the poster. Three examples of these empathetic expression templates are shown below.

[0097] (Examples) - If the response is encouraging: "You're doing a great job." - If the response is empathetic: "I completely understand how you feel." - If the response is complimentary: "That's a great idea."

[0098] The feedback generation unit 162 inputs the acquired diary text data and emotion identification information into the read feedback generation model. As a result, the feedback generation unit 162 obtains text information indicating the feedback statement output from the feedback generation model.

[0099] Below are two examples of feedback statements generated by the feedback generation model.

[0100] (Example 1) • Input data: Diary text data: "I failed, but I feel like I'll do better next time." Specific emotional information: "Positive intention" • Output data: Feedback text: "That positive attitude is important. You're sure to get better results next time."

[0101] (Example 2) ・Input data: Diary text data: "I had fun playing with my friends at the park today." Specific emotion information: "Fun" ・Output data: Feedback sentence: "What a wonderful day. You were able to relax in nature."

[0102] The feedback generation unit 162 controls the communication unit 11 and outputs data to the storage server 20 that associates the anonymity-enhanced diary image data output from the anonymity enhancement unit 161 with the text information indicating the feedback statement output from the feedback generation model. This associated data is stored in the storage server 20 in association with information identifying the diary poster. Alternatively, this associated data may be stored in the storage server 20 in association with information identifying the terminal device 40 used by the poster.

[0103] The diary image publication control unit 163 retrieves the anonymized diary image data and feedback text stored in the storage server 20 and controls their publication on the network 80. The published diary image data and feedback text can be viewed by picture diary applications and web browsers installed on terminal devices 40. The diary image publication control unit 163 is a functional unit that provides the functions of a conventional diary application service.

[0104] The anonymity enhancement function for diary images by the anonymity enhancement unit 161 and the feedback text generation function by the feedback generation unit 162, as described above, are positioned as optional functions of the picture diary generation service in this embodiment. In other words, these functions may be omitted. If these functions are omitted, the diary image publication control unit 163 will acquire only the diary image data after removing specific image areas stored in the storage server 20 and perform control to publish it on the network 80. Alternatively, the diary image publication control unit 163 may perform control to acquire only the diary image data after removing specific text stored in the storage server 20 and publish it on the network 80.

[0105] [Storage Server Configuration] The configuration of the storage server 20 will be described below. Figure 4 is a schematic block diagram showing the functional configuration of the storage server 20 in one embodiment of the present invention. As shown in Figure 4, the storage server 20 is composed of a communication unit 21, a storage unit 24, and a control unit 25.

[0106] The communication unit 21 is a communication device. The communication unit 21 communicates with other devices via a communication line in accordance with the control unit 25. The communication unit 21 may be a wired communication device or a wireless communication device. The communication unit 21 may also be configured, for example, as a network interface.

[0107] The storage unit 24 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 24 stores data used by the control unit 25. The storage unit 24 stores data necessary when the control unit 25 performs processing.

[0108] The memory unit 24 functions as a storage area for storing data such as the aforementioned style information, diary text data, extracted keywords, emotion identification information, prompt information, diary image data, corrected image area information, diary image data after removal of specific text, diary image data after removal of specific image area, diary image data after anonymity enhancement, and feedback text. These data may also be stored in the memory unit 14 of the AI ​​processing server 10.

[0109] The control unit 25 is composed of a processor such as a CPU and memory. The control unit 25 functions as a memory control unit that controls the storage of various diary image data processed by the AI ​​processing server 10, the original diary text data posted by the poster, various data and information indicating various execution conditions used in the processing performed by the AI ​​processing server 10, and various data and programs used in its own device (storage server 20), as the processor executes a program.

[0110] Furthermore, all or part of the functions of the control unit 25 may be implemented using hardware such as ASICs, PLDs, or FPGAs. The above program may be recorded on a computer-readable recording medium. Computer-readable recording media include, for example, portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and semiconductor memory devices (e.g., SSDs), as well as storage devices such as hard disks and semiconductor memory devices built into computer systems. The above program may also be transmitted via a telecommunications line.

[0111] The control unit 25 may, for example, execute an application installed on its own device (storage server 20). The control unit 25 operates according to the program of the application being executed.

[0112] One concrete example of such an application is an application provided to the storage server 20 as a dedicated application for the picture diary generation system 1. Another concrete example of such an application is a web browser application. Such applications may be pre-installed on the storage server 20, or they may be downloaded each time a communication process is performed.

[0113] For example, if the application is implemented as a web browser application, the storage server 20 may download and execute the application from a device specified by the web server (for example, the web server itself or another server) when the storage server 20 connects to a specific web server.

[0114] [API Server Functional Configuration] The configuration of the API server 30 will be described below. Figure 5 is a schematic block diagram showing the functional configuration of the API server 30 in one embodiment of the present invention. As shown in Figure 5, the API server 30 is composed of a communication unit 31, a storage unit 34, and a control unit 35.

[0115] The communication unit 31 is a communication device. The communication unit 31 communicates with other devices via a communication line in accordance with the control of the control unit 35. The communication unit 31 may be a device that performs wired communication or a device that performs wireless communication. The communication unit 31 may be configured as, for example, a network interface.

[0116] The storage unit 34 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 34 stores data used by the control unit 35. The storage unit 34 stores data necessary when the control unit 35 performs processing.

[0117] The control unit 35 is composed of a processor such as a CPU and memory. The control unit 35 functions as an API providing control unit that provides an interface (API) for relaying various data and function exchanges between the AI ​​processing server 10 and the terminal device 40 when the processor executes a program.

[0118] Furthermore, all or part of the functions of the control unit 35 may be implemented using hardware such as ASICs, PLDs, or FPGAs. The above program may be recorded on a computer-readable recording medium. Computer-readable recording media include, for example, portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and semiconductor memory devices (e.g., SSDs), as well as storage devices such as hard disks and semiconductor memory devices built into computer systems. The above program may be transmitted via a telecommunications line.

[0119] The control unit 35 may, for example, execute an application installed on its own device (API server 30). The control unit 35 operates according to the program of the application being executed.

[0120] One concrete example of such an application is an application provided to the API server 30 as a dedicated application for the picture diary generation system 1. Another concrete example of such an application is a web browser application. Such applications may be pre-installed on the API server 30, or they may be downloaded each time a communication process is performed.

[0121] For example, if implemented as a web browser application, the API server 30 may download and execute the application from a device specified by the web server (for example, the web server itself or another server) in response to the API server 30 connecting to a specific web server.

[0122] [Functional Configuration of Terminal Device] The functional configuration of the terminal device 40 will be described below. Figure 6 is a schematic block diagram showing the functional configuration of the terminal device 40 in one embodiment of the present invention. The terminal device 40 is configured using information equipment such as a smartphone, tablet, personal computer, or dedicated device. As shown in Figure 6, the terminal device 40 is configured to include a communication unit 41, an input unit 42, an output unit 43, a storage unit 44, and a control unit 45.

[0123] The communication unit 41 is a communication device. The communication unit 41 communicates with other devices (for example, the API server 30 and the authentication server 50) via the network 80 in accordance with the control of the control unit 45. The communication unit 41 may be a wireless communication device or a wired communication device. The communication unit 41 may also be configured as, for example, a network interface.

[0124] The input unit 42 is configured using existing input devices such as a keyboard, pointing device (mouse, tablet, etc.), input buttons, touch panel, and microphone. The input unit 42 is operated by the user when inputting user instructions or speech into the terminal device 40. The user referred to here is, for example, a user of the picture diary generation service (i.e., a poster who submits diary text data, a viewer who views published diary images, etc.).

[0125] The input unit 42 may also be an input interface for connecting an input device (an external device not shown) to the terminal device 40. In this case, the input unit 42 receives input signals generated in response to user input in the input device and inputs them to its own device (terminal device 40).

[0126] The input unit 42 may be configured in any way as long as it is capable of inputting user instructions and utterances into the terminal device 40. The input unit 42 may also be configured using a speech recognition device. In this case, the input unit 42, for example, acquires an acoustic signal generated by the user's utterance and inputs it to the terminal device 40. Alternatively, the input unit 42 may perform speech recognition on the words uttered by the user and input the resulting string information to the terminal device 40. The speech recognition process may be performed by the control unit 45.

[0127] The output unit 43 outputs information in a format that the user can recognize. The output unit 43 may be an image display device such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The output unit 43 may also be an interface for connecting an image display device (an external device not shown) to the terminal device 40. In this case, the output unit 43 generates a video signal for displaying image data and outputs the video signal to the image display device connected to it.

[0128] The output unit 43 may be a device that outputs sound, such as a speaker. The output unit 43 may also be an interface for connecting an audio output device (external device not shown), such as a speaker or headphones, to the terminal device 40. In this case, the output unit 43 generates an audio signal for playing audio data and outputs the audio signal to the audio output device connected to it.

[0129] The output unit 43 may be configured as a touch panel integrated with the input unit 42.

[0130] The storage unit 44 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 44 stores data used by the control unit 45. The storage unit 44 stores data necessary when the control unit 45 performs processing.

[0131] The control unit 45 is composed of a processor such as a CPU and memory (main memory). The control unit 45 functions when the processor executes a program. Note that all or part of the functions of the control unit 45 may be implemented using hardware such as an ASIC, PLD, or FPGA. The above program may be recorded on a computer-readable recording medium. Computer-readable recording media include, for example, portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and semiconductor memory devices (e.g., SSDs), as well as storage devices such as hard disks and semiconductor memory devices built into computer systems. The above program may be transmitted via a telecommunications line.

[0132] The control unit 45 may, for example, execute an application installed on its own device (terminal device 40). The control unit 45 operates according to the program of the application being executed.

[0133] A concrete example of such an application is an application provided to the terminal device 40 as a dedicated application for the picture diary generation system 1. For example, one of the dedicated applications for the picture diary generation system 1 is the aforementioned picture diary application. Another concrete example of such an application is a web browser application. Such applications may be pre-installed on the terminal device 40, or they may be downloaded each time a communication process is performed.

[0134] For example, if the application is implemented as a web browser application, the terminal device 40 may download and execute the application from a device specified by the web server (for example, the web server itself or another server) when the terminal device 40 connects to a specific web server.

[0135] The control unit 45 controls the terminal device 40 in response to user (poster, etc.) operations and information received from the API server 30 and the authentication server 50. For example, the control unit 45 transmits information entered by the user (poster, etc.) through the input unit 42 to the API server 30 and the authentication server 50 using the communication unit 41.

[0136] For example, when the control unit 45 receives information transmitted from the API server 30 and the authentication server 50 at the communication unit 41, it generates screen data based on the received information and displays the screen data on the output unit 43. Such screen data includes images and characters that represent the information transmitted from the API server 30 and the authentication server 50.

[0137] Furthermore, for example, when the control unit 45 receives information transmitted from the API server 30 and the authentication server 50 at the communication unit 41, it generates audio data based on the received information and outputs the audio data from the output unit 43.

[0138] The functional configuration of the control unit 45 will be described in detail below. Figure 7 is a block diagram showing the functional configuration of the control unit 45 of the terminal device 40 in one embodiment of the present invention. As shown in Figure 7, the control unit 45 is composed of a style specification unit 451, a diary text data transmission unit 452, a specific text registration unit 453, a correction necessity specification unit 454, a diary image acquisition unit 455, and a feedback information acquisition unit 456.

[0139] The style specification unit 451 controls the transmission of style information indicating the style of the diary images (illustrations in picture diaries), which is specified in advance by the diary poster, to the AI ​​processing server 10. More specifically, the style specification unit 451 first obtains a list of candidate style information transmitted from the AI ​​processing server 10. The candidate style information here refers to a list that includes style information such as "childish style," "mature style," and "cartoon-like style," as mentioned above.

[0140] The style selection unit 451 loads a list of candidate style information into the picture diary app and generates a style selection screen. The style selection screen, in this context, is a display image on the picture diary app screen that allows the poster to select their desired style information. The style selection unit 451 displays the style selection screen using an output unit 43 (for example, a touch panel).

[0141] The style selection unit 451 detects the style information selected by the poster on the style selection screen using the input unit 42 (for example, a touch panel). The style selection unit 451 controls the communication unit 41 and transmits the style information selected by the poster to the AI ​​processing server 10 via the network 80.

[0142] (Example of a style selection screen) An example of a style selection screen will be described below. Figure 8 is a diagram showing a style selection screen Sa displayed on a terminal device 40 in one embodiment of the present invention. The style selection screen Sa shown in Figure 8 is, for example, a display image displayed on the touch panel which is the input unit 42 and output unit 43 of the terminal device 40.

[0143] As shown in Figure 8, the top of the style selection screen Sa displays the title of this screen, "Style Selection." Below that, the text "Please select a style for the illustrations in your picture diary" is displayed to prompt the diary poster to select the style of illustrations for the picture diary generated by the picture diary generation service.

[0144] Further down, there are multiple image areas (image areas Sa1 to Sa4, etc.) each displaying an image of the shape of an input button. As shown in Figure 8, image area Sa1 displays the words "childish image", image area Sa2 displays the words "adult image", image area Sa3 displays the words "cartoon-like image", and image area Sa4 displays the words "confirm".

[0145] The poster selects the desired style (artistic style) for the picture in their picture diary by, for example, tapping one of the image areas Sa1 to Sa3 on the touch panel. After selecting a style, the poster confirms the style selection for the picture in the picture diary by, for example, tapping the image area Sa4 on the touch panel. The style specification unit 451 sends style information indicating the confirmed style to the AI ​​processing server 10.

[0146] For example, as shown in Figure 8, when the poster taps part of the image area Sa1 on the touch panel, and then taps part of the image area Sa4, style information indicating a "childish drawing style" is transmitted from the terminal device 40 to the AI ​​processing server 10.

[0147] The diary text data transmission unit 452 receives input of diary text from the diary author and controls the transmission of diary text data to the AI ​​processing server 10. More specifically, the diary text data transmission unit 452 launches the picture diary application and generates a diary creation screen. The diary creation screen, in this context, is a display image on the picture diary application screen that includes an input form for the author to enter the diary text. The diary text data transmission unit 452 displays the diary creation screen using an output unit 43 (for example, a touch panel).

[0148] The diary text data transmission unit 452 detects when the poster has finished entering the text of the diary on the diary creation screen. The diary text data transmission unit 452 controls the communication unit 41 and transmits the diary text data, which represents the text of the diary entered by the poster in the input form, to the AI ​​processing server 10 via the network 80.

[0149] (Example of a diary creation screen) An example of a diary creation screen will be described below. Figure 9 is a diagram showing a diary creation screen Sb displayed on a terminal device 40 in one embodiment of the present invention. The diary creation screen Sb shown in Figure 9 is, for example, a display image displayed on the touch panel which is the input unit 42 and output unit 43 of the terminal device 40.

[0150] As shown in Figure 9, the title of the diary creation screen Sb, "Create Diary," is displayed at the top. Below that, the text "Please enter the date and diary entry and submit" is displayed to prompt the diary writer to create a diary entry by entering the date and diary entry, and to encourage them to submit the completed diary entry.

[0151] Further down, there is an image area Sb1 containing an image of an input form for entering a date, and an image area Sb2 containing an image of an input form for entering the text of a diary entry. Further down, there is an image area Sa3 containing an image of the shape of an input button. As shown in Figure 9, the word "Post" is displayed in image area Sb3.

[0152] The poster activates the input form that accepts date input by, for example, tapping the image area Sb1 on the touch panel. The poster enters the date of the diary entry to be created. Note that the input form that accepts date input may be a dropdown form that allows the poster to select the date. Next, the poster activates the input form that accepts the text of the diary entry by, for example, tapping the image area Sb2 on the touch panel. The poster enters the text of the diary entry to be created.

[0153] Finally, the poster completes the diary entry by, for example, tapping the image area Sb3 on the touch panel. The diary text data transmission unit 452 sends the diary text data, which is linked to information indicating the date of the diary entry, to the AI ​​processing server 10.

[0154] For example, as shown in Figure 9, the poster taps the image area Sb1 on the touch panel to input "2025 / 03 / 18", then taps the image area Sb2 to input "I had fun playing with my friends at the park today," and then taps the image area Sb3. As a result, diary text data, including the text "I had fun playing with my friends at the park today," which is linked to the date March 18, 2025, is transmitted from the terminal device 40 to the AI ​​processing server 10.

[0155] The specific text registration unit 453 receives input from the diary poster of words to be removed from diary images and controls the transmission of information indicating the specific text registration list, which is a list of the input words, to the AI ​​processing server 10. More specifically, the specific text registration unit 453 launches the picture diary application and generates a screen for registering words to be removed. The screen for registering words to be removed is a display image on the picture diary application screen that includes an input form for the poster to input words to be removed from diary images. The specific text registration unit 453 displays the screen for registering words to be removed using an output unit 43 (for example, a touch panel).

[0156] The specific text registration unit 453 detects when the poster has finished entering the words to be removed from the diary image on the registration screen for words to be removed. The specific text registration unit 453 controls the communication unit 41 and sends the specific text registration list, which is a list of words to be removed from the diary image entered by the poster in the input form, to the AI ​​processing server 10 via the network 80.

[0157] (Example of a screen for registering words to be removed) An example of a screen for registering words to be removed will be described below. Figure 10 is a diagram showing the screen Sc for registering words to be removed displayed on the terminal device 40 in one embodiment of the present invention. The screen Sc for registering words to be removed shown in Figure 10 is, for example, a display image displayed on the touch panel which is the input unit 42 and output unit 43 of the terminal device 40.

[0158] As shown in Figure 10, the top of the registration screen Sc for words to be removed displays the title of this screen, "Register words to be removed." One row below that displays the message, "Please register the words you want to remove from the pictures in your picture diary," prompting the diary poster to enter the words to be removed from the diary images.

[0159] Further down, there are multiple image areas (image areas Sc1 to Sc4) containing an image with an input form that accepts input of words to be removed from the diary image, and an image area Sc3 containing an image with the shape of an input button. As shown in Figure 10, the word "Register" is displayed in image area Sc3.

[0160] The poster activates an input form that accepts input of words to be removed from the diary image by, for example, tapping one of the image areas Sc1 to Sc4 on the touch panel. The poster enters the desired number of words to be removed from the diary image. Finally, the poster completes the input of words to be removed from the diary image by, for example, tapping the image area Sc5 on the touch panel. The specific text registration unit 453 sends the specific text registration list, which is a list of words to be removed from the diary image, to the AI ​​processing server 10 via the network 80.

[0161] For example, as shown in Figure 10, the poster taps the image area Sc1 on the touch panel to input "○○ town", then taps the image area Sc2 to input "△△ school", and then taps the image area Sc5. As a result, a specific text registration list containing the phrases "○○ town" and "△△ school" is transmitted from the terminal device 40 to the AI ​​processing server 10.

[0162] The correction necessity / request unit 454 receives a selection from the diary poster regarding the necessity of removing images that could lead to the identification of an individual contained in the diary image, and controls the transmission of information indicating the selection result for each image area to the AI ​​processing server 10. More specifically, first, the correction necessity / request unit 454 obtains information transmitted from the AI ​​processing server 10 indicating the diary image and the image areas within the diary image that are presumed to contain images that could lead to the identification of an individual.

[0163] The correction necessity / non-necessity designation unit 454 reads the diary image and information indicating the image area within the diary image that is presumed to contain images that could lead to the identification of an individual into the picture diary app, and generates a correction area designation screen. The correction area designation screen, as referred to here, is a display image on the picture diary app screen that allows the poster to select whether or not to remove images that could lead to the identification of an individual contained in the diary image, for each image area. The correction necessity / non-necessity designation unit 454 displays the correction area designation screen via an output unit 43 (for example, a touch panel).

[0164] The correction necessity / request unit 454 detects that the poster has completed selecting whether or not to remove images in the diary image that could lead to the identification of an individual, as done on the correction area selection screen. The correction necessity / request unit 454 controls the communication unit 41 and transmits information indicating the poster's selection results for each image area to the AI ​​processing server 10 via the network 80.

[0165] (Example of the correction location specification screen) An example of the correction location specification screen will be described below. Figure 11 is a diagram showing the correction location specification screen Sd displayed on the terminal device 40 in one embodiment of the present invention. The correction location specification screen Sd shown in Figure 11 is, for example, a display image displayed on the touch panel which is the input unit 42 and output unit 43 of the terminal device 40.

[0166] As shown in Figure 11, the top of the correction area selection screen Sd displays the title of this screen, "Correction Area Selection". One row below that, the text "Do you want to correct the following parts of the picture?" is displayed to prompt the diary poster to choose whether or not to remove images that could lead to the identification of an individual.

[0167] Further down, an image area Sd1 is located, displaying the diary image generated by the AI ​​processing server 10. As shown in Figure 11, two areas within the displayed diary image are enclosed by dashed lines, and these two areas are labeled "(1)" and "(2)" respectively. These two areas are areas of the image that could lead to the identification of an individual, as detected by the specific image area detection unit 159 of the AI ​​processing server 10.

[0168] Further down, image areas Sd2 and Sd3 are located, containing images depicting the shapes of input buttons. As shown in Figure 11, the words "(1) location (actual station name?)" are displayed to the left of image area Sd2, and the words "Correct" are displayed in image area Sd2. Further down, image area Sd4 is located, containing an image depicting the shapes of input buttons. As shown in Figure 11, the words "Confirm" are displayed in image area Sd4.

[0169] In other words, the area labeled "(1)" is an image region that the AI ​​processing server 10 has estimated to contain the name of an actual station, and the poster can specify whether or not to correct the image in that region by tapping the image region Sd2 on the touch panel. Correction, as mentioned above, refers to image processing such as blurring the image in that region, filling it with the surrounding background color, or replacing it with an image of arbitrary text information (words) that does not lead to the identification of an individual.

[0170] The text displayed in image area Sd2 switches between "Correct" and "Do not correct" each time it is tapped. If the poster wants to correct the part (1) in the diary image, they should input the text so that the text displayed in image area Sd2 is "Correct," and if they do not want to correct the part (1) in the diary image, they should input the text so that the text displayed in image area Sd2 is "Do not correct."

[0171] Furthermore, as shown in Figure 11, the words "(2) location (actual time?)" are displayed on the left side of image region Sd3, and the words "Do not correct" are displayed in image region Sd3.

[0172] In other words, the area labeled "(2)" is an image region that the AI ​​processing server 10 estimates to depict the actual time (a clock representing the time), and the poster can specify whether or not to correct the image in that region by tapping the image region Sd3 on the touch panel. Correction, as mentioned above, refers to image processing such as blurring the image in that region, filling it with the surrounding background color, or replacing it with an image of arbitrary text information (words) that does not lead to the identification of an individual.

[0173] The text displayed in image area Sd3 switches between "Correct" and "Do not correct" each time it is tapped. If the poster wants to correct the part (2) in the diary image, they should input the text so that the text displayed in image area Sd3 is "Correct," and if they do not want to correct the part (2) in the diary image, they should input the text so that the text displayed in image area Sd3 is "Do not correct."

[0174] Finally, the poster completes the selection of whether or not to remove images that could lead to the identification of an individual by, for example, tapping the image area Sd4 on the touch panel. The correction requirement specification unit 454 sends information indicating the selection result for each image area to the AI ​​processing server 10.

[0175] For example, as shown in Figure 11, the poster taps image area Sd2 on the touch panel with their finger to input "Correct" so that the text displayed in image area Sd2 is "Do not correct", and then taps image area Sd3 with their finger to input "Do not correct" so that the text displayed in image area Sd2 is "Do not correct", and then taps image area Sd4. As a result, information indicating the selection results for each image area is transmitted from the terminal device 40 to the AI ​​processing server 10, instructing that correction be performed on image areas labeled "(1)" and not on image areas labeled "(2)".

[0176] The diary image acquisition unit 455 acquires diary image data that has been made public on the network 80 by the diary image publication control unit 163 of the AI ​​processing server 10. The diary image acquisition unit 455 displays the diary image based on the acquired diary image data on the output unit 43 (for example, a touch panel).

[0177] More specifically, the diary image acquisition unit 455 launches the picture diary application and loads the acquired diary image data into the picture diary application. As a result, the picture diary generated by the picture diary generation service is displayed on the screen of the picture diary application. Alternatively, the diary image acquisition unit 455 may also load the acquired diary image data into a web browser. As a result, the picture diary generated by the picture diary generation service is displayed in the web browser.

[0178] The feedback information acquisition unit 456 acquires information indicating the feedback text published on the network 80 by the diary image publication control unit 163 of the AI ​​processing server 10. The diary image acquisition unit 455 displays the feedback text based on the acquired information on the output unit 43 (for example, a touch panel).

[0179] More specifically, the diary image acquisition unit 455 launches the picture diary application and loads the information indicating the acquired feedback text into the picture diary application. As a result, the feedback text generated by the picture diary generation service is displayed on the screen of the picture diary application. Alternatively, the diary image acquisition unit 455 may also load the information indicating the acquired feedback text into a web browser. As a result, the feedback text generated by the picture diary generation service is displayed in the web browser.

[0180] [Functional Configuration of the Authentication Server] The configuration of the authentication server 50 will be described below. Figure 12 is a schematic block diagram showing the functional configuration of the authentication server 50 in one embodiment of the present invention. As shown in Figure 12, the authentication server 50 is composed of a communication unit 51, a storage unit 54, and a control unit 55.

[0181] The communication unit 51 is a communication device. The communication unit 51 communicates with other devices via a communication line in accordance with the control of the control unit 55. The communication unit 51 may be a device that performs wired communication or a device that performs wireless communication. The communication unit 51 may be configured as, for example, a network interface.

[0182] The storage unit 54 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 54 stores data used by the control unit 55. The storage unit 54 stores data necessary when the control unit 55 performs processing.

[0183] The control unit 55 is configured using a processor such as a CPU and memory. The control unit 55 functions as an authentication control unit that controls user authentication for users using the picture diary generation service by having the processor execute a program. Alternatively, the control unit 55 may be configured to function as an authentication control unit that controls device authentication for the terminal device 40 used to use the picture diary generation service by having the processor execute a program.

[0184] Furthermore, all or part of the functions of the control unit 55 may be implemented using hardware such as ASICs, PLDs, or FPGAs. The above program may be recorded on a computer-readable recording medium. Computer-readable recording media include, for example, portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and semiconductor memory devices (e.g., SSDs), as well as storage devices such as hard disks and semiconductor memory devices built into computer systems. The above program may also be transmitted via a telecommunications line.

[0185] The control unit 55 may, for example, execute an application installed on its own device (authentication server 50). The control unit 55 operates according to the program of the application being executed.

[0186] One concrete example of such an application is an application provided to the authentication server 50 as a dedicated application for the picture diary generation system 1. Another concrete example of such an application is a web browser application. Such applications may be pre-installed on the authentication server 50, or they may be downloaded each time a communication process is performed.

[0187] For example, if implemented as a web browser application, the authentication server 50 may download and execute the application from a device specified by the web server (for example, the web server itself or another server) in response to the authentication server 50 connecting to a specific web server.

[0188] [Operation of the AI ​​Processing Server] An example of the operation of the AI ​​processing server 10 will be described below. In the following description, the operation of the basic service portion of the AI ​​processing server 10, up to the point when the diary image generation unit 156 generates a diary image, and the operation of the optional service portion thereafter, which is divided into the operation of the specific text correction unit 158 ​​when removing specific text from the diary image, the operation of the specific image area correction unit 160 when removing a specific image area from the diary image, the operation of the anonymity enhancement unit 161 when enhancing the anonymity of the diary image, and the operation of the feedback generation unit 162 when generating feedback text.

[0189] (Operation during diary image generation) First, we will explain the operation of the AI ​​processing server 10 until it generates a diary image. Figure 13 is a flowchart showing the operation of the AI ​​processing server 10 during diary image generation in one embodiment of the present invention.

[0190] First, the style information acquisition unit 151 acquires style information transmitted from the terminal device 40 via the network 80 and received by the communication unit 11 (step S001). The style information acquisition unit 151 outputs the acquired style information to the prompt generation unit 155. The style information acquisition unit 151 also controls the communication unit 11 to output the acquired style information to the storage server 20. The style information is stored in the storage server 20, linked to information identifying the diary poster.

[0191] Next, the diary text data acquisition unit 152 acquires the diary text data transmitted from the terminal device 40 via the network 80 and received by the communication unit 11 (step S002). The diary text data acquisition unit 152 outputs the acquired diary text data to the keyword extraction unit 153 and the feedback generation unit 162. The diary text data acquisition unit 152 also controls the communication unit 11 to output the acquired diary text data to the storage server 20. The diary text data is stored in the storage server 20, linked to information identifying the diary's author.

[0192] Next, the keyword extraction unit 153 acquires the diary text data output from the diary text data acquisition unit 152. The keyword extraction unit 153 performs natural language processing on the acquired diary text data and extracts keywords contained in the diary text data (step S003).

[0193] For example, the keyword extraction unit 153 performs morphological analysis to divide the text into individual words and determine the part of speech, syntactic analysis to analyze grammatical structures such as subjects and predicates, semantic analysis to understand the meaning of sentences and words, document classification to divide the text into categories, and linguistic analysis such as machine translation to translate into other languages. The keyword extraction unit 153 analyzes the diary text data using a method that evaluates the importance of words contained in the document, such as TF-IDF. The keyword extraction unit 153 extracts specific keywords contained in the diary text data based on the analyzed importance.

[0194] The keyword extraction unit 153 outputs information indicating the extracted keywords to the emotion identification unit 154 and the prompt generation unit 155. The keyword extraction unit 153 also controls the communication unit 11 and outputs information indicating the extracted keywords to the storage server 20. The extracted keywords are linked to information identifying the diary's author and stored in the storage server 20.

[0195] Next, the emotion identification unit 154 acquires information indicating the extracted keywords output from the keyword extraction unit 153. The emotion identification unit 154 extracts emotion keywords from the extracted keywords based on the acquired information (step S004). The emotion identification unit 154 reads the keyword dictionary that is stored in the memory unit 14 in advance. The keyword dictionary here is a dictionary in which emotions and emotion keywords (that represent those emotions) are associated. The emotion identification unit 154 compares the extracted emotion keywords with the emotion keywords included in the keyword dictionary to identify the poster's emotions in the diary (step S005).

[0196] The emotion identification unit 154 outputs emotion identification information indicating the identified (estimated) emotion to the prompt generation unit 155 and the feedback generation unit 162. The emotion identification unit 154 also controls the communication unit 11 to output the emotion identification information to the storage server 20. The emotion identification information is stored in the storage server 20, linked to information identifying the diary poster.

[0197] Next, the prompt generation unit 155 acquires the style information output from the style information acquisition unit 151. The prompt generation unit 155 also acquires information indicating the extracted keywords output from the keyword extraction unit 153. The prompt generation unit 155 also acquires the emotion identification information output from the emotion identification unit 154.

[0198] Furthermore, the prompt generation unit 155 reads the prompt generation model stored in the memory unit 14. The prompt generation model, as mentioned above, is a learning model that, when style information, information indicating extracted keywords, and emotion identification information are input, automatically generates and outputs prompts to be input to the visual expression generation model (generating AI) described later. The prompt generation model is pre-generated by machine learning using, for example, training data in which learning style information, information indicating extracted keywords, and emotion identification information are associated with prompts that are the correct labels.

[0199] The prompt generation unit 155 inputs the acquired style information, information indicating extracted keywords, and emotion identification information into the read prompt generation model. As a result, the prompt generation unit 155 obtains a prompt output from the prompt generation model (step S006). The prompt generation unit 155 outputs the prompt information output from the prompt generation model to the diary image generation unit 156. The prompt generation unit 155 also controls the communication unit 11 to output the prompt information to the storage server 20. The prompt information is stored in the storage server 20, linked to information identifying the diary's author.

[0200] Next, the diary image generation unit 156 acquires the prompt information output from the prompt generation unit 155. The diary image generation unit 156 also reads the visual representation generation model stored in the memory unit 14. The visual representation generation model, as mentioned above, is a learning model that automatically generates and outputs diary images (pictures in the picture diary) generated by the picture diary generation service when a prompt is input. The visual representation generation model is pre-generated, for example, by machine learning using training data in which learning prompts and diary images, which are the correct labels, are associated.

[0201] The diary image generation unit 156 inputs prompts based on the acquired prompt information to the read-out visual representation generation model. As a result, the diary image generation unit 156 obtains a diary image output from the visual representation generation model (step S007).

[0202] The diary image generation unit 156 outputs the diary image data output from the visual representation generation model to the specific text detection unit 157. The diary image generation unit 156 also controls the communication unit 11 to output the diary image data to the storage server 20 (step S008). The diary image data is stored in the storage server 20, linked to information identifying the diary's author.

[0203] This completes the operation of the AI ​​processing server 10 during diary image generation, as shown in the flowchart of Figure 13.

[0204] (Operation when removing specific text) Next, we will explain the operation of the AI ​​processing server 10 when removing specific text from diary images. Figure 14 is a flowchart showing the operation of the AI ​​processing server 10 when removing specific text in one embodiment of the present invention.

[0205] First, the specific text detection unit 157 acquires diary image data output from the diary image generation unit 156. Alternatively, the specific text detection unit 157 acquires diary image data stored in the storage server 20 (Step S101). Next, the specific text detection unit 157 reads the specific text registration list stored in the storage unit 14 (Step S102). The specific text registration list, as mentioned above, is a list of words or phrases that should be removed from diary images, which have been registered in advance by users such as the poster.

[0206] Next, the specific text detection unit 157 detects image regions in the diary image where text information (words) is written by performing image analysis (step S103). The specific text detection unit 157 reads the text information (words) from the detected image regions using, for example, OCR technology.

[0207] Next, the specific text detection unit 157 compares the read text information (phrase) with the phrases included in the specific text registration list (step S104). If the read text information (phrase) is a phrase included in the specific text registration list (step S105, YES), the specific text detection unit 157 determines that correction of the image area in which the text information (phrase) is depicted is necessary (step S106). If the read text information (phrase) is a phrase not included in the specific text registration list (step S105, NO), the specific text detection unit 157 determines that correction of the image area in which the text information (phrase) is depicted is unnecessary.

[0208] The specific text detection unit 157 outputs corrected image area information, which indicates the image area deemed to require correction, to the specific text correction unit 158. The specific text detection unit 157 also controls the communication unit 11 to output the corrected image area information to the storage server 20. The corrected image area information is stored in the storage server 20, linked to information identifying the diary's author.

[0209] Next, the specific text detection unit 157 reads out the specific text detection model stored in the storage unit 14 (step S107). The specific text detection model, as mentioned above, is a learning model that, when an image is input, detects image regions in the image that contain text information (words) that could lead to the identification of an individual, and outputs information indicating the detected image region as corrected image region information.

[0210] Furthermore, the text information (words) that may lead to the identification of an individual, as used herein, is not limited to words that can identify an individual on their own, such as words that indicate personal information such as names, addresses, and telephone numbers, as mentioned above. The text information (words) that may lead to the identification of an individual, as used herein, is not limited to these, and may also include words that indicate dates and times, places, etc., and words that indicate identifying information such as user IDs, which may lead to the identification of an individual when combined with other information.

[0211] As mentioned above, the specific text detection model is pre-generated by machine learning using training data that associates, for example, diary images with information indicating image regions containing text information (words) that could lead to the identification of an individual, which are the correct labels (for example, information indicated by coordinate values ​​within the image).

[0212] Next, the specific text detection unit 157 inputs the diary image data output from the diary image generation unit 156 into the read specific text detection model. As a result, the specific text detection unit 157 obtains the corrected image region information output from the specific text detection model (step S108).

[0213] Next, the specific text detection unit 157 outputs the diary image data output from the diary image generation unit 156 and the corrected image area information output from the specific text detection model to the specific text correction unit 158. The specific text detection unit 157 also controls the communication unit 11 and outputs the corrected image area information to the storage server 20. The corrected image area information is stored in the storage server 20, linked to information identifying the diary's author.

[0214] Next, the specific text correction unit 158 ​​acquires the diary image data and corrected image area information output from the specific text detection unit 157. Based on the acquired diary image data and corrected image area information, the specific text correction unit 158 ​​identifies an image area in the diary image in which text information (words) that could lead to the identification of an individual is written (step S109). The specific text correction unit 158 ​​corrects the image area so as to remove the text information (words) written in the identified image area (step S110).

[0215] Specifically, for example, the specific text correction unit 158 ​​performs image processing on the image region based on the acquired corrected image region information, such as blurring it, filling it with the surrounding background color, or replacing it with an image of arbitrary text information (words) that does not lead to the identification of an individual.

[0216] Next, the specific text correction unit 158 ​​corrects the image area in which text information (words) that could lead to the identification of an individual is depicted, and outputs the diary image data after the specific text has been removed to the specific image area detection unit 159. The specific text correction unit 158 ​​also controls the communication unit 11 to output the diary image data after the specific text has been removed to the storage server 20 (step S111). The diary image data after the specific text has been removed is stored in the storage server 20, linked to information that identifies the diary's author.

[0217] This completes the operation of the AI ​​processing server 10 when removing specific text, as shown in the flowchart of Figure 14.

[0218] (Operation when removing a specific image region) Next, the operation of the AI ​​processing server 10 when removing a specific image region from a diary image will be described. Figure 15 is a flowchart showing the operation of the AI ​​processing server 10 when removing a specific image region in one embodiment of the present invention.

[0219] First, the specific image region detection unit 159 acquires the post-specific text removal image data output from the specific text correction unit 158. Alternatively, the specific image region detection unit 159 acquires the post-specific text removal image data stored in the storage server 20 (step S201). Next, the specific image region detection unit 159 reads the specific image region detection model stored in the storage unit 14 (step S202). The specific image region detection model, as described above, is a learning model that, when an image is input, detects image regions in the image that could lead to the identification of an individual, and outputs information indicating the detected image regions as corrected image region information.

[0220] As mentioned above, the specific image region detection model is pre-generated by machine learning using training data that associates, for example, diary images with information indicating image regions containing images that could lead to the identification of an individual, which are the correct labels (for example, information indicated by coordinate values ​​within the diary images).

[0221] Next, the specific image region detection unit 159 inputs the diary image data output from the diary image generation unit 156 into the read specific image region detection model. As a result, the specific image region detection unit 159 obtains corrected image region information output from the specific image region detection model (step S203).

[0222] Next, the specific image region detection unit 159 outputs the image data after specific text removal output from the specific text correction unit 158 ​​and the corrected image region information output from the specific image region detection model to the specific image region correction unit 160. The specific image region detection unit 159 also controls the communication unit 11 and outputs the corrected image region information to the storage server 20.

[0223] Next, the specific image area correction unit 160 acquires the diary image data after specific text removal and the corrected image area information output from the specific text correction unit 158. Based on the acquired diary image data after specific text removal and the corrected image area information, the specific image area correction unit 160 identifies an image area in the diary image that contains images that could lead to the identification of an individual (step S204). Next, the specific image area correction unit 160 corrects the identified image area so as to remove any images that could lead to the identification of an individual from that image area (step S205).

[0224] Specifically, for example, the specific image area correction unit 160 performs image processing on the image area based on the acquired corrected image area information, such as blurring it, filling it with the surrounding background color, or replacing it with an image that does not lead to the identification of an individual. For example, the specific image area correction unit 160 performs correction by replacing the image area based on the acquired corrected image area information with an image that does not lead to the identification of an individual.

[0225] Next, the specific image area correction unit 160 corrects the image area containing images that could lead to the identification of an individual, and outputs the diary image data after the specific image area has been removed to the anonymity enhancement unit 161. The specific image area correction unit 160 also controls the communication unit 11 to output the diary image data after the specific image area has been removed to the storage server 20 (step S206). The diary image data after the specific image area has been removed is stored in the storage server 20, linked to information that identifies the diary's author.

[0226] This completes the operation of the AI ​​processing server 10 when removing a specific image region, as shown in the flowchart of Figure 15.

[0227] (Operation during anonymity enhancement) Next, we will describe the operation of the AI ​​processing server 10 when it performs image processing to further enhance anonymity on the diary images. Figure 16 is a flowchart showing the operation of the AI ​​processing server 10 during anonymity enhancement in one embodiment of the present invention.

[0228] First, the anonymity enhancement unit 161 acquires the post-specific image region removal image data output from the specific image region correction unit 160. Alternatively, the anonymity enhancement unit 161 acquires the post-specific image region removal image data stored in the storage server 20 (step S301). Next, the anonymity enhancement unit 161 reads the anonymity enhancement model stored in the storage unit 14 (step S302).

[0229] As mentioned above, an anonymity enhancement model is a learning model that, when given a diary image as input, outputs an anonymity-enhanced diary image data in which image regions containing images that could lead to the identification of an individual are transformed into image regions containing images that cannot lead to the identification of an individual. The anonymity enhancement model is pre-generated, for example, by performing machine learning using training data in which diary images are associated with the correct labels, which are the anonymity-enhanced diary images.

[0230] Next, the anonymity enhancement unit 161 inputs the post-removal image data of the specific image region output from the specific image region correction unit 160 into the read-out anonymity enhancement model. As a result, the anonymity enhancement unit 161 obtains the post-removal image data of the anonymity enhancement model output (step S303).

[0231] Next, the anonymity enhancement unit 161 outputs the anonymity-enhanced diary image data output from the anonymity enhancement model to the feedback generation unit 162. The anonymity enhancement unit 161 also controls the communication unit 11 to output the anonymity-enhanced diary image data to the storage server 20 (step S303). The anonymity-enhanced diary image data is stored in the storage server 20, linked to information that identifies the diary's author.

[0232] This completes the operation of the AI ​​processing server 10 during anonymity enhancement, as shown in the flowchart of Figure 16.

[0233] (Operation during feedback statement generation) Next, we will describe the operation of the AI ​​processing server 10 when it performs image processing to generate a feedback statement. Figure 17 is a flowchart showing the operation of the AI ​​processing server 10 during feedback statement generation in one embodiment of the present invention.

[0234] First, the feedback generation unit 162 acquires the anonymity-enhanced diary image data output from the anonymity enhancement unit 161. Alternatively, the feedback generation unit 162 acquires the anonymity-enhanced diary image data stored in the storage server 20 (step S401). Next, the feedback generation unit 162 acquires the diary text data output from the diary text data acquisition unit 152 (step S402). Next, the feedback generation unit 162 acquires the emotion identification information output from the emotion identification unit 154 (step S403). The feedback generation unit 162 also reads the feedback generation model stored in the memory unit 14 (step S404).

[0235] The feedback generation model referred to here is, as mentioned above, a learning model that automatically generates and outputs feedback sentences when diary text data and sentiment identification information are input. Furthermore, the feedback sentences referred to here are, as mentioned above, positive statements about the post that the picture diary generation service responds to the poster in response to the poster's diary text data submission. The feedback generation model is pre-generated, for example, by performing machine learning using training data in which diary text data and sentiment identification information are associated with the correct labels, which are the feedback sentences.

[0236] Next, the feedback generation unit 162 inputs the acquired diary text data and emotion identification information into the read feedback generation model. As a result, the feedback generation unit 162 obtains text information indicating the feedback statement output from the feedback generation model (step S405).

[0237] The feedback generation unit 162 controls the communication unit 11 and outputs data to the storage server 20 that associates the anonymity-enhanced diary image data output from the anonymity enhancement unit 161 with the text information indicating the feedback statement output from the feedback generation model (step S406). This associated data is linked to information identifying the diary's author and stored in the storage server 20.

[0238] This completes the operation of the AI ​​processing server 10 during feedback statement generation, as shown in the flowchart of Figure 17.

[0239] [Operation of the terminal device] An example of the operation of the terminal device 40 will be described below. In the following description, the operation of the terminal device 40 will be divided into the following: the operation when a style is selected by the style specification unit 451, the operation when a diary is created by the diary text data transmission unit 452, the operation when a specific text registration unit 453 registers words to be removed, the operation when a correction location is specified by the correction necessity specification unit 454, and the operation when a diary image is displayed by the diary image acquisition unit 455 and when a feedback text is displayed by the feedback information acquisition unit 456.

[0240] (Operation when selecting a style) First, we will explain the operation of the terminal device 40 when the poster selects the style (artistic style) of the picture diary generated by the picture diary generation service. Figure 18 is a flowchart showing the operation of the terminal device 40 when selecting a style in one embodiment of the present invention.

[0241] First, the style specification unit 451 obtains a list of candidate style information sent from the AI ​​processing server 10 (step S501). The candidate style information referred to here is a list that includes style information such as "childish style," "mature style," and "manga-like style," as mentioned above.

[0242] Next, the style selection unit 451 loads a list of candidate style information into the picture diary app and generates a style selection screen (step S502). The style selection screen, as mentioned above, is a display image on the picture diary app screen that allows the poster to select their desired style information. The style selection unit 451 displays the style selection screen using an output unit 43 (for example, a touch panel).

[0243] Next, the style specification unit 451 detects the style information selected by the poster on the style selection screen using the input unit 42 (for example, a touch panel) (step S503). Next, the style specification unit 451 controls the communication unit 41 and transmits the style information selected by the poster to the AI ​​processing server 10 via the network 80 (step S504).

[0244] This completes the operation of the terminal device 40 when selecting a style, as shown in the flowchart of Figure 18.

[0245] (Operation during diary creation) Next, we will explain the operation of the terminal device 40 when the poster creates a diary entry and submits it to the picture diary generation service. Figure 19 is a flowchart showing the operation of the terminal device 40 during diary creation in one embodiment of the present invention.

[0246] First, the diary text data transmission unit 452 launches the picture diary application and generates a diary creation screen (step S601). The diary creation screen, as mentioned above, is a display image on the picture diary application screen that includes an input form for the poster to enter the text of their diary. The diary text data transmission unit 452 displays the diary creation screen using an output unit 43 (for example, a touch panel).

[0247] Next, the diary text data transmission unit 452 detects that the poster has finished entering the diary text on the diary creation screen (step S602). Then, the diary text data transmission unit 452 controls the communication unit 41 and transmits the diary text data, which represents the diary text entered by the poster in the input form, to the AI ​​processing server 10 via the network 80 (step S603).

[0248] This completes the operation of the terminal device 40 when creating a diary, as shown in the flowchart of Figure 19.

[0249] (Operation when registering words to be removed) Next, we will explain the operation of the terminal device 40 when the poster enters words to be removed from the diary image. Figure 20 is a flowchart showing the operation of the terminal device 40 when registering words to be removed in one embodiment of the present invention.

[0250] First, the specific text registration unit 453 launches the picture diary application and generates a registration screen for words to be removed (step S701). The registration screen for words to be removed, as mentioned above, is a display image on the picture diary application screen that includes an input form for the poster to enter words to be removed from the diary images. The specific text registration unit 453 displays the registration screen for words to be removed using an output unit 43 (for example, a touch panel).

[0251] Next, the specific text registration unit 453 detects that the poster has completed entering the words to be removed from the diary image on the registration screen for words to be removed (step S702). Next, the specific text registration unit 453 controls the communication unit 41 and sends the specific text registration list, which is a list of words to be removed from the diary image entered by the poster in the input form, to the AI ​​processing server 10 via the network 80 (step S703).

[0252] This completes the operation of the terminal device 40 when registering the words to be removed, as shown in the flowchart of Figure 20.

[0253] (Operation when specifying correction areas) Next, we will explain the operation of the terminal device 40 when the poster decides whether or not to remove images that could lead to the identification of individuals contained in the diary images, for each image area. Figure 21 is a flowchart showing the operation of the terminal device 40 when specifying correction areas in one embodiment of the present invention.

[0254] First, the correction necessity designation unit 454 acquires information transmitted from the AI ​​processing server 10 indicating the diary image and the image area within the diary image that is presumed to contain images that could lead to the identification of an individual (step S801).

[0255] Next, the correction necessity / non-necessity designation unit 454 loads information indicating the diary image and the image area within the diary image that is presumed to contain images that could lead to the identification of an individual into the picture diary app, and generates a correction area designation screen (step S802). The correction area designation screen, as described above, is a display image on the picture diary app screen that allows the poster to select whether or not to remove images that could lead to the identification of an individual contained in the diary image, for each image area. The correction necessity / non-necessity designation unit 454 displays the correction area designation screen using an output unit 43 (for example, a touch panel).

[0256] Next, the correction necessity / request unit 454 detects that the poster has completed selecting whether or not to remove images in the diary image that could lead to the identification of an individual, as done on the correction area selection screen (step S803). Next, the correction necessity / request unit 454 controls the communication unit 41 and transmits information indicating the poster's selection results for each image area to the AI ​​processing server 10 via the network 80 (step S804).

[0257] This completes the operation of the terminal device 40 when specifying the correction point, as shown in the flowchart of Figure 21.

[0258] (Operation when displaying diary images and feedback text) Next, we will explain the operation of the terminal device 40 when the poster refers to the picture (diary image) and feedback text of the picture diary generated by the picture diary generation service. Figure 22 is a flowchart showing the operation of the terminal device 40 when displaying diary images and feedback text in one embodiment of the present invention.

[0259] First, the diary image acquisition unit 455 acquires diary image data that has been made public on the network 80 by the diary image publication control unit 163 of the AI ​​processing server 10 (step S901).

[0260] Next, the diary image acquisition unit 455 displays the diary image based on the acquired diary image data on the output unit 43 (for example, a touch panel) (step S902). More specifically, the diary image acquisition unit 455 launches the picture diary application and loads the acquired diary image data into the picture diary application. As a result, the picture diary image generated by the picture diary generation service is displayed on the screen of the picture diary application.

[0261] Next, the feedback information acquisition unit 456 acquires information indicating the feedback text that has been published on the network 80 by the diary image publication control unit 163 of the AI ​​processing server 10 (step S903).

[0262] Next, the diary image acquisition unit 455 displays a feedback message based on the acquired information on the output unit 43 (for example, a touch panel) (step S904). More specifically, the diary image acquisition unit 455 launches the picture diary application and loads the information indicating the acquired feedback message into the picture diary application. As a result, the feedback message generated by the picture diary generation service is displayed on the screen of the picture diary application.

[0263] This completes the operation of the terminal device 40 when displaying the diary image and the feedback text, as shown in the flowchart of Figure 22.

[0264] [Hardware Configuration Example] Figure 23 is a schematic diagram of a hardware configuration example of an information processing device 90 applied to one embodiment of the present invention. The information processing device 90 comprises a processor 91, a main memory 92, a communication interface 93, an auxiliary storage device 94, an input / output interface 95, and an internal bus 96. The processor 91, the main memory 92, the communication interface 93, the auxiliary storage device 94, and the input / output interface 95 are connected to each other via the internal bus 96 so as to be able to communicate with each other.

[0265] The information processing device 90 may be applied to, for example, an AI processing server 10, a storage server 20, an API server 30, a terminal device 40, and an authentication server 50. In this case, for example, the communication unit 11, the communication unit 21, the communication unit 31, the communication unit 41, and the communication unit 51 may be configured using a communication interface 93. Also, for example, the storage unit 14, the storage unit 24, the storage unit 34, the storage unit 44, and the storage unit 54 may be configured using an auxiliary storage device 94. Also, for example, the control unit 15, the control unit 25, the control unit 35, the control unit 45, and the control unit 55 may be configured using a processor 91 and a main memory device 92. Also, for example, the input unit 42 and the output unit 43 may be configured using an input / output interface 95.

[0266] As described above, in the illustrated diary generation system 1 of one embodiment of the present invention, the AI ​​processing server 10 extracts key keywords from the diary entries posted by the diary author. The AI ​​processing server 10 identifies the emotions of the diary author from the extracted key keywords (or from the content of the entire diary entry). Based on the extracted key keywords and the information indicating the identified emotions, the AI ​​processing server 10 generates prompts (input instructions) for the visual representation generation model to generate illustrated diary entries (diary images). The AI ​​processing server 10 generates diary images by inputting the generated prompts into the visual representation generation model. The AI ​​processing server 10 then publishes the automatically generated diary images based on the diary entries posted by the author.

[0267] Thus, in one embodiment of the present invention, the picture diary generation system 1 does not publish specific and detailed information about the poster, such as the diary text itself or photographs posted by the poster, as a diary. Instead, it publishes more abstract information (pictures) automatically generated by the AI ​​processing server 10 as a picture diary. By having this configuration, the picture diary generation system 1 can prevent the communication between the diary poster and others who react to the posted diary from becoming too specific and intimate. As a result, the picture diary generation system 1 can reduce the psychological burden on posters in communication services, making it less likely for them to experience communication fatigue such as SNS fatigue.

[0268] Furthermore, in the illustrated diary generation system 1 of one embodiment of the present invention, the diary author can specify the style of the illustrations (diary images) in the generated illustrated diary. With such a configuration, the diary author can more easily reflect their own personality in the diary images automatically generated by the AI ​​processing server 10.

[0269] Furthermore, the AI ​​processing server 10 of the picture diary generation system 1 in one embodiment of the present invention detects image regions within the generated diary images that contain words or images that could lead to the identification of an individual. The AI ​​processing server 10 then performs image processing on the detected image regions, such as blurring them, filling them with the surrounding background color, or replacing them with images of arbitrary text information (words) that do not lead to the identification of an individual. By having such a configuration, the picture diary generation system 1 can further enhance the anonymity of the pictures (diary images) in the publicly released picture diaries.

[0270] Furthermore, the AI ​​processing server 10 of the picture diary generation system 1 in one embodiment of the present invention uses an anonymity enhancement model specifically designed to enhance the anonymity of diary images to perform further image transformations on the generated diary images. With this configuration, the picture diary generation system 1 can further enhance the anonymity of the pictures (diary images) in the publicly available picture diary.

[0271] Furthermore, in one embodiment of the present invention, the AI ​​processing server 10 of the picture diary generation system 1 generates positive feedback texts, such as encouragement, empathy, and praise, in response to the diary entries submitted by the diary writer and the emotions of the writer identified from those entries, and publishes them together with the picture diary entries (diary images). With such a configuration, the picture diary generation system 1 can, for example, motivate the writer to continue posting diary entries.

[0272] Furthermore, according to the picture diary generation system 1 in one embodiment of the present invention, anonymity is ensured, so it is expected that participation in the picture diary generation service will be promoted among a wide range of users (for example, children, adults, the elderly, and foreigners). Since users can easily record and share daily events, the picture diary generation system 1 is expected to contribute to users' self-expression and stress relief. In addition, the picture diary generation system 1 can provide an environment in which users all over the world can post diverse picture diaries, and it is expected that this will further stimulate users' creativity.

[0273] According to the embodiment described above, the diary image generation device comprises a diary text data acquisition unit, a keyword extraction unit, an emotion identification unit, an input instruction sentence generation unit, a diary image generation unit, and an output unit. For example, the diary image generation device is the AI ​​processing server 10 in the embodiment, the diary text data acquisition unit is the diary text data acquisition unit 152 in the embodiment, the keyword extraction unit is the keyword extraction unit 153 in the embodiment, the emotion identification unit is the emotion identification unit 154 in the embodiment, the input instruction sentence generation unit is the prompt generation unit 155 in the embodiment, the diary image generation unit is the diary image generation unit 156 in the embodiment, and the output unit is the diary image publication control unit 163 in the embodiment.

[0274] The diary text data acquisition unit acquires information indicating diary entries posted by the user. For example, the user is the poster in the embodiment, and the information indicating the diary entries is the diary text data in the embodiment. The keyword extraction unit extracts key words and phrases from the diary entries. For example, key words and phrases are the key keywords in the embodiment. The emotion identification unit identifies the user's emotions based on the key words and phrases. The input instruction generation unit acquires an input instruction generation model and, by inputting the key words and information indicating the user's emotions into the input instruction generation model, generates an input instruction to instruct the visual representation generation model to generate a diary image, which is a picture corresponding to the diary entries. For example, the input instruction generation model is the prompt generation model in the embodiment, and the input instruction is the prompt in the embodiment. The diary image generation unit acquires a visual representation generation model and generates a diary image by inputting the input instruction into the visual representation generation model. The output unit outputs the diary image. The input instruction generation model is a machine learning model that outputs an input instruction in response to input of key words and phrases extracted from the diary entries and information indicating emotions. The visual representation generation model is a machine learning model that is designed to output diary images in response to input instructions.

[0275] The above-described diary image generation device may further include a style information acquisition unit. For example, the style information acquisition unit is the style information acquisition unit 151 in the embodiment. The style information acquisition unit acquires information indicating the style of the diary image specified by the user. The input instruction sentence generation unit generates an input instruction sentence by further inputting the information indicating the style into the input instruction sentence generation model.

[0276] The diary image generation device described above may further include a specific text detection unit and a specific text correction unit. For example, the specific text detection unit is the specific text detection unit 157 in the embodiment, and the specific text correction unit is the specific text correction unit 158 ​​in the embodiment. The specific text detection unit obtains a list of specific words and phrases specified in advance and detects a specific word area from the diary image, which is an image area in which a specific word or phrase included in the list is depicted. For example, the specific word or phrase is text information (word or phrase) that can lead to the identification of an individual in the embodiment. The specific text correction unit replaces the image of the detected specific word or phrase area with another image.

[0277] The above-described diary image generation device may further include a specific image region detection unit and a specific image region correction unit. For example, the specific image region detection unit is the specific image region detection unit 159 in the embodiment, and the specific image region correction unit is the specific image region correction unit 160 in the embodiment. The specific image region detection unit extracts a specific image region detection model and inputs the diary image into the specific image region detection model to detect a specific image region in the diary image, which is an image region in the diary image that may lead to the identification of an individual. The specific image region correction unit replaces the image in the detected specific image region with another image.

[0278] The above-described diary image generation device may further include an anonymity enhancement unit. The anonymity enhancement unit acquires an anonymity enhancement model and generates diary images with higher anonymity by inputting diary images into the anonymity enhancement model. For example, a diary image with higher anonymity is the anonymity-enhanced diary image data in the embodiment. The anonymity enhancement model is a machine learning model that outputs a diary image with higher anonymity in response to a diary image input.

[0279] The above-described diary image generation device may further include a feedback generation unit. For example, the feedback generation unit is the feedback generation unit 162 in the embodiment. The feedback generation unit acquires a response sentence generation model and generates a response sentence for the content of the diary sentence by inputting the diary sentence and information indicating the user's emotions into the response sentence generation model. For example, the response sentence generation model is the feedback generation model in the embodiment. The output unit outputs the response sentence along with the diary image. For example, the response sentence is the feedback sentence in the embodiment. The response sentence generation model is a machine learning model that outputs a response sentence in response to inputs of the diary sentence and information indicating the user's emotions.

[0280] The configuration of the AI ​​processing server 10 in the above-described embodiment may be implemented using a computer. In that case, the program for implementing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into the computer system and executed. Here, "computer system" includes hardware such as the OS and peripheral devices. Furthermore, "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into the computer system. Moreover, "computer-readable recording medium" may also include those that dynamically hold programs for a short period of time, such as communication lines used when transmitting programs via networks such as the Internet or communication lines such as telephone lines, and those that hold programs for a certain period of time, such as volatile memory inside the computer system that acts as a server or client in that case. Furthermore, the above-mentioned program may be for implementing a part of the above-mentioned function, or it may be a program that can implement the above-mentioned function in combination with a program already recorded in the computer system, or it may be implemented using a programmable logic device such as an FPGA (Field Programmable Gate Array).

[0281] While embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and includes designs and the like that do not depart from the spirit of this invention.

[0282] 1 Picture Diary Generation System 10 AI Processing Server 11 Communication Unit 14 Memory Unit 15 Control Unit 20 Storage Server 21 Communication Unit 24 Memory Unit 25 Control Unit 30 API Server 31 Communication Unit 34 Memory Unit 35 Control Unit 40 Terminal Device 41 Communication Unit 42 Input Unit 43 Output Unit 44 Memory Unit 45 Control Unit 50 Authentication Server 51 Communication Unit 54 Memory Unit 55 Control Unit 80 Network 90 Information Processing Device 91 Processor 92 Main Memory 93 Communication Interface 94 Auxiliary Storage Device 95 Input / Output Interface 96 Internal Bus 151 Style Information Acquisition Unit 152 Diary Text Data Acquisition Unit 153 Keyword Extraction Unit 154 Emotion Identification Unit 155 Prompt Generation Unit 156 Diary Image Generation Unit 157 Specific Text Detection Unit 158 Specific text correction unit 159 Specific image area detection unit 160 Specific image area correction unit 161 Anonymity enhancement unit 162 Feedback generation unit 163 Diary image publication control unit 451 Style specification unit 452 Diary text data transmission unit 453 Specific text registration unit 454 Correction necessity specification unit 455 Diary image acquisition unit 456 Feedback information acquisition unit

Claims

1. A diary image generation device comprising: a diary text data acquisition unit that acquires information indicating diary entries posted by a user; a keyword extraction unit that extracts key words from the diary entries; an emotion identification unit that identifies the user's emotions based on the key words; an input instruction generation unit that acquires an input instruction generation model and generates an input instruction to instruct a visual representation generation model to generate a diary image, which is a picture corresponding to the diary entries, by inputting the key words and information indicating the user's emotions into the input instruction generation model; a diary image generation unit that acquires the visual representation generation model and generates the diary image by inputting the input instruction to the visual representation generation model; and an output unit that outputs the diary image, wherein the input instruction generation model is a machine learning model that outputs the input instruction in response to inputs of key words extracted from the diary entries and information indicating emotions, and the visual representation generation model is a machine learning model that outputs the diary image in response to inputs of the input instruction.

2. The diary image generation device according to claim 1, further comprising a style information acquisition unit that acquires information indicating the style of the diary image specified by the user, wherein the input instruction statement generation unit generates the input instruction statement by further inputting the information indicating the style of the diary image into the input instruction statement generation model.

3. The diary image generation device according to claim 1, further comprising: a specific text detection unit that obtains a pre-specified list of specific words and phrases and detects a specific word region from the diary image which is an image region in which the specific words included in the list are depicted; and a specific text correction unit that replaces the image of the detected specific word region with another image.

4. A diary image generation device according to claim 1, further comprising: a specific image region detection unit that extracts a specific image region detection model and inputs the diary image to the specific image region detection model to detect a specific image region in the diary image that is an image region in which an image that could lead to the identification of an individual is depicted; and a specific image region correction unit that replaces the image of the detected specific image region with another image, wherein the specific image region detection model is a learning model that has been machine-trained to output the specific image region in response to the input of the diary image.

5. The diary image generation device according to claim 1, further comprising an anonymity enhancement unit that acquires an anonymity enhancement model and inputs the diary images into the anonymity enhancement model to generate diary images with higher anonymity, wherein the anonymity enhancement model is a machine learning model that outputs diary images with higher anonymity in response to the input of diary images.

6. The diary image generation device according to claim 1, further comprising a feedback generation unit that acquires a response sentence generation model and generates a response sentence for the content of the diary sentence by inputting the diary sentence and the information indicating the user's emotions into the response sentence generation model, the output unit outputs the response sentence together with the diary image, and the response sentence generation model is a machine learning model that has been trained to output the response sentence in response to the input of the diary sentence and the information indicating the user's emotions.

7. A method for generating diary images by computer, comprising: a diary text data acquisition step of acquiring information indicating diary entries posted by a user; a keyword extraction step of extracting key words from the diary entries; an emotion identification step of identifying the user's emotions based on the key words; an input instruction generation step of acquiring an input instruction generation model and inputting the key words and information indicating the user's emotions into the input instruction generation model to generate an input instruction for instructing a visual representation generation model to generate a diary image which is a picture corresponding to the diary entries; a diary image generation step of acquiring a visual representation generation model and inputting the input instruction into the visual representation generation model to generate the diary image; and an output step of outputting the diary image, wherein the input instruction generation model is a machine learning model that outputs the input instruction in response to inputs of the key words extracted from the diary entries and information indicating the emotions, and the visual representation generation model is a machine learning model that outputs the diary image in response to inputs of the input instruction.

8. A diary image generation program for causing a computer to perform the following steps: acquiring diary text data to acquire information indicating diary entries posted by a user; extracting key words from the diary entries; identifying the user's emotions based on the key words; acquiring an input instruction generation model and inputting the key words and information indicating the user's emotions into the input instruction generation model to generate an input instruction for instructing a visual representation generation model to generate a diary image, which is a picture corresponding to the diary entries; acquiring the visual representation generation model and inputting the input instruction into the visual representation generation model to generate the diary image; and outputting the diary image, wherein the input instruction generation model is a machine learning model that outputs the input instruction in response to inputs of the key words extracted from the diary entries and information indicating the emotions, and the visual representation generation model is a machine learning model that outputs the diary image in response to inputs of the input instruction.

Images