Information processing methods, information processing systems, and programs

Abstract

Generate a life plan chart that leaves a stronger impression on customers and is more likely to resonate with them. [Solution] The information processing method includes the steps of: acquiring data from interview records concerning the life plan of a customer's family; acquiring images of the family; generating illustrations for each of several life events related to the family in the life plan based on the images and the data from the interview records; placing the generated illustrations at the positions of each of the several life events in a life plan table showing the life plan; and outputting the life plan table.

Description

[Technical Field]

[0001] The present invention relates to an information processing method, an information processing system, and a program. [Background Art]

[0002] Patent Literature 1 discloses a system that generates topics for customers and presents topics during an interview on a timeline along with the elapsed time during the interview. [Prior Art Literature] [Patent Literature]

[0003] [Patent Literature 1] Japanese Patent No. 7458679 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] There is room for improvement in generating life plan tables that leave a stronger impression on customers and facilitate gaining empathy. [Means for Solving the Problem]

[0005] According to one aspect of the present disclosure, an information processing method executed by an information processing system is provided. The information processing method includes the steps of: acquiring interview record data regarding a life plan of a customer's family; acquiring an image of the family; generating an illustration for each of a plurality of life events related to the family in the life plan based on the image and the interview record data; arranging the generated corresponding illustration at each position of the plurality of life events in a life plan table indicating the life plan; and outputting the life plan table. [Brief Description of the Drawings]

[0006] [Figure 1] FIG. 1 is a diagram showing an example of the system configuration of the information processing system. [Figure 2]Figure 2 shows an example of the hardware configuration of a server device. [Figure 3] Figure 3 shows an example of the hardware configuration of a client device. [Figure 4] Figure 4 is a sequence diagram (part 1) showing an example of information processing in an information processing system. [Figure 5] Figure 5 shows an example of the life event selection screen in a life plan table. [Figure 6] Figure 6 is a sequence diagram (part 2) showing an example of information processing in an information processing system. [Figure 7] Figure 7 shows an example of a life plan chart screen. [Figure 8] Figure 8 is an example (part 1) of an illustration list. [Figure 9] Figure 9 is an example (part 2) of an illustration list. [Modes for carrying out the invention]

[0007] Embodiments of the present invention will be described below with reference to the drawings. The various features shown in the embodiments below (including modified examples; the same applies hereinafter) can be combined with each other.

[0008] <Embodiment 1> 1. System Configuration Diagram Figure 1 shows an example of the system configuration of the information processing system 1000. As shown in Figure 1, the information processing system 1000 includes, as a system configuration, a server device 100, a client device 110, a server device 120, and a server device 130. The server device 100 is an example of a computer. The server device 100, the client device 110, the server device 120, and the server device 130 are connected to each other via a network 150. The network 150 may include a WAN (Wide Area Network), a LAN (Local Area Network), and the Internet, or both. The network 150 is configured to allow communication between devices connected to the network 150 via wired and / or wireless connections. The information processing system 1000 is a system that provides so-called SaaS (Software as a Service) functionality.

[0009] The server device 100 performs information processing, etc., as described in Embodiment 1, in response to requests from the client device 110. Details of the processing performed by the server device 100 will be explained later using sequence diagrams, etc.

[0010] The client device 110 is a device operated by an operator of the information processing system 1000 and is a device that utilizes the functions provided by the server device 100. The client device 110 is, for example, a device operated by a service provider that provides services to a customer (family in the example of the specification) while sharing the screen with the customer.

[0011] Server device 120 is a device that provides services for large-scale language models. A large-scale language model is a computer language model composed of an artificial neural network with numerous parameters, and is a language model trained using self-supervised or semi-supervised learning with vast amounts of unlabeled text. Furthermore, a large-scale language model can also be described as an advanced artificial intelligence model that can learn from large amounts of text data and generate, translate, and answer questions using natural language used by humans. The large-scale language model may also be implemented on the server device 100.

[0012] Server device 130 is a device that provides an illustration generation model service. The illustration generation model is a computer illustration model composed of an artificial neural network with many parameters, and is a model that can generate new illustrations from input data such as text and / or images. The illustration generation model may also be implemented in the server device 100.

[0013] The specification uses a large-scale language model as an example of a model that outputs text and an illustration generation model as an example of a model that outputs illustrations. However, the information processing system 1000 may include a server device that provides a service for a single multimodal model having the functions of both a large-scale language model and an illustration generation model. The server device 100 may then utilize the multimodal model service. In such a configuration, the multimodal model is an example of an illustration generation model.

[0014] In Figure 1, a notebook PC (Personal Computer) is shown as an example of the client device 110, but it is not limited to a notebook PC; it could be a desktop PC, a tablet device, or a smartphone. The client device 110 can be any device that can input operator information and display the results of processing on the server device 100.

[0015] Here, the information processing system recited in the claims may be configured with a plurality of devices or may be configured with a single device. When the information processing system recited in the claims is configured with a single device, an example of the device is the server device 100, for example. When the information processing system recited in the claims is configured with a plurality of devices, examples of the plurality of devices are, for instance, a distributed system that provides the functions of the server device 100, or the server device 100 and the server device 120, or the server device 100 and the server device 130, or the server device 100 and the client device 110.

[0016] 2. Hardware Configuration (1) Hardware Configuration of Server Device 100 Figure 2 is a diagram showing an example of the hardware configuration of the server device 100. As shown in Figure 2, the server device 100 includes, as a hardware configuration, a control unit 210, a storage unit 220, a communication unit 230, and an internal bus 240. The control unit 210, the storage unit 220, and the communication unit 230 are electrically connected via the internal bus 240.

[0017] The control unit 210 is a CPU (Central Processing Unit) or the like, and controls the entire server device 100.

[0018] The storage unit 220 is one of the following, or any combination thereof, of an HDD (Hard Disk Drive), ROM (Read Only Memory), RAM (Random Access Memory), SSD (Solid State Drive), etc., and stores the program and data used by the control unit 210 when executing processing based on the program (for example, the compiler, linker, loader, library, family images, interview record data, etc., as described later). In the following description, the server device 100 will be described as having ROM, RAM, HDD, and SSD as storage units. The storage unit 220 is an example of a storage medium. In this specification, the data used by the control unit 210 when executing processing based on the program will be described as being stored in the storage unit 220, but it may also be stored in the storage unit of another device that can communicate with the server device 100. That is, the data may be stored in the storage unit of any device as long as the control unit 210 can access or retrieve it. The control unit 210 executes processing based on the program stored in the storage unit 220, thereby realizing the functions of the server device 100 and the processing of the server device 100 as shown in the sequence diagrams in Figures 4 and 6, which will be described later.

[0019] If the source code's programming language is compiled, the control unit 210 uses a compiler to convert the source code stored in ROM, HDD, or SSD into object code. The control unit 210 then uses a linker to link the object code and libraries to generate a load module. The control unit 210 then uses a loader to load the load module into memory such as RAM. The control unit 210 executes the load module loaded into memory to realize the processing of the server device 100 in the sequence diagrams shown in Figures 4 and 6, which will be described later. If the source code's programming language is interpreted, the control unit 210 reads the source code stored in ROM, HDD, or SSD into RAM or other memory, and processes the loaded source code line by line during execution, performing lexical and / or syntactic analysis, thereby realizing the processing of the server device 100 as shown in the sequence diagrams in Figures 4 and 6 described later.

[0020] The communications unit 230 connects the server device 100 to the network 150 and manages communication with other devices.

[0021] (2) Hardware configuration of client device 110 Figure 3 shows an example of the hardware configuration of the client device 110. As shown in Figure 3, the client device 110 includes, as a hardware configuration, a control unit 310, a storage unit 320, an imaging unit 330, an input unit 340, an output unit 350, a communication unit 360, and an internal bus 370. The control unit 310, storage unit 320, imaging unit 330, input unit 340, output unit 3550, and communication unit 360 are electrically connected via the internal bus 370.

[0022] The control unit 310 is a CPU or the like, and controls the entire client device 110.

[0023] The storage unit 320 stores data used by the program and the control unit 310 when executing processing based on the program, and is one of the following: HDD, ROM, RAM, SSD, etc., or any combination thereof. The storage unit 320 is an example of a storage medium.

[0024] The imaging unit 330 is a camera or the like that takes images of the subject. An example of the subject is the customer's family (each member of the family), which will be described later.

[0025] The input unit 340 is a keyboard and mouse, etc., which inputs information based on selection and / or input operations performed by the operator. The output unit 350 is a display or the like, which displays input information from the operator via the screen and the results of processing by the control unit 310.

[0026] The communications unit 360 connects the client device 110 to the network 150 and manages communication with other devices.

[0027] 3. Information Processing (1) Overview of the process The control unit 210 acquires data from interview records regarding the customer's family's life plan. The control unit 210 acquires images of the family. Based on the images and the interview record data, the control unit 210 generates illustrations for each of the multiple life events related to the family in the life plan. The control unit 210 places the generated illustrations at the positions of each of the multiple life events in the life plan table that shows the life plan. The control unit 210 outputs the life plan table. This process makes it possible to generate a life plan chart that leaves a stronger impression on customers and is more likely to resonate with them.

[0028] (2) Details of the process Figure 4 is a sequence diagram (part 1) showing an example of information processing in the information processing system 1000. In sequence SQ401, the control unit 310 of the client device 110 transmits images of each member of the family, along with family composition information, to the server device 100. The family images can be described as digital photographs of the family. Family composition information includes who lives in a household, their relationships, the number of people and their ages, their hobbies and preferences, etc. A household is a group of people who live together and are economically connected. Relationships refer to information indicating the relationships between members of the household, such as spouse, children, and parents. In addition, each image of a family member includes attribute information related to the image, such as who the family member is (father, mother, etc.). In sequence SQ401, the information transmitted by the client device 110 may be information captured and input by the client device 110, or it may be information previously entered by family members via a questionnaire screen or the like.

[0029] When the server device 100 receives images of the family and family configuration information from the client device 110, in sequence SQ402, the control unit 210 of the server device 100 associates the images of the family and family configuration information and stores them together with identification information to identify the family in a predetermined storage area such as the storage unit 220. The identification information to identify the family is input by the client device 110.

[0030] Note that the processing of sequence SQ401 can be performed at any time, as long as it is performed before sequence SQ601, which will be described later, regardless of the timing shown in Figure 4.

[0031] In sequence SQ403, the control unit 310 transmits data of the interview record regarding life planning with family members to the server device 100. The data of the interview record regarding life planning is video data or audio data of the interview regarding life planning. For simplicity, in this specification, the data of the interview record regarding life planning will be described as video data of the interview regarding life planning. The interview record data may be recorded and audio recorded of an in-person interview by the client device 110, or it may be recorded and audio recorded of a web interview.

[0032] When the server device 100 receives interview record data from the client device 110, in sequence SQ404, the control unit 210 of the server device 100 stores the interview record data regarding the family's life plan, along with identification information to identify the family, in a predetermined storage area such as the storage unit 220.

[0033] In sequence SQ405, the control unit 210 acquires family interview record data, identified by identification information, from a predetermined storage area such as the memory unit 220. The control unit 210 also acquires family composition information, etc., identified by identification information, from a predetermined storage area such as the memory unit 220. In sequence SQ406, the control unit 210 extracts audio data based on the interview record data. The control unit 210 converts the extracted audio data into text using speech recognition technology. The control unit 210 then adjusts the text to be grammatically correct and natural Japanese using natural language processing technology. If the audio contains the voices of multiple speakers, the control unit 210 may use speaker separation technology to convert the audio into text for each speaker.

[0034] In sequence SQ407, the control unit 210 generates a prompt for extracting (or generating) life event ideas for each family member based on the text of the interview record. A prompt is an instruction or question to AI (Artificial Intelligence), etc. This prompt may include, for example, speaker-separated text of the interview record regarding the family's life plan, and an instruction to extract (or generate) life event ideas (content of the life event + timing of the life event) for each family member from the interview record text. The prompt may also include information about the family composition as reference information for extracting (or generating) life event ideas. Life event ideas can also be called candidate life events. The timing of life events can also be called the approximate timing of the life events. The prompt may also include an instruction to predict the approximate amount of money involved if the life event is expensive.

[0035] In sequence SQ408, the control unit 210 transmits the generated prompt to the large-scale language model. In sequence SQ409, the control unit 210 receives life event ideas for each member of the family, which have been extracted (or generated) in the large-scale language model. A life event is an event in a person's life that has a significant impact on their subsequent life. Examples of life events include important milestones in a person's life, such as entering kindergarten or school, getting married, having children, starting a job, changing jobs, buying a house, retiring, or traveling abroad. Life events (or life event ideas) are assigned attributes related to money. These attributes related to money are various characteristics (or manners) that show how money is involved in life events. More specifically, these attributes include, for example, that the life event involves ongoing or irregular expenses, or that the life event requires funds.

[0036] In sequence SQ410, the control unit 210 generates a life event selection screen (hereinafter also simply referred to as the selection screen) for a life plan table, as shown in Figure 5, which will be described later, and includes ideas for the life events of each member of the family that has been received. In sequence SQ411, the control unit 210 transmits the generated selection screen to the client device 110. In sequence SQ412, the control unit 310 of the client device 110 displays a selection screen on the output unit 350. The selection screen is displayed, for example, on the web browser of the client device 110.

[0037] As another example, the control unit 210 may transmit data used for the selection screen to the client device 110. The control unit 310 of the client device 110 may then generate and display the selection screen based on the received data. This also applies to the life plan table screen, which will be described later. However, for the sake of simplicity, the following explanation will assume that the control unit 210 generates the screen.

[0038] Figure 5 shows an example of the life event selection screen 500 of the life plan table. The specification will explain using an example of generating a life plan table for a family consisting of a 34-year-old father, a 30-year-old mother, a 3-year-old eldest daughter named Ichiha, and a 1-year-old eldest son named Yushin. As shown in Figure 5, the life event selection screen 500 of the life plan table is configured to display the life plans of each member of the family. Row 550 is the row that displays the father's life plan. Row 551 is the row that displays the mother's life plan. Row 552 is the row that displays the eldest daughter's life plan. Row 553 is the row that displays the eldest son's life plan.

[0039] Vertical line 540 represents this year. Vertical line 541 represents 5 years from now. Vertical line 542 represents 10 years from now. Vertical line 543 represents 15 years from now. Vertical line 544 represents 20 years from now. Vertical line 545 represents 25 years from now. Vertical line 546 represents 30 years from now.

[0040] In the selection screen 500, each family member's frame (or area) displays the corresponding family member's age. For example, in row 551, the frame enclosed by vertical lines 540 and 541 represents the period from this year to 5 years from now in Mama's life plan, and this frame displays image 560, which represents Mama's age this year, 30. Similarly, in row 551, the frame enclosed by vertical lines 541 and 542 represents the period from 5 years from now to 10 years from now in Mama's life plan, and this frame displays image 561, which represents Mama's age in 5 years, 35. Similarly, image 562 is an image representing Mama's age in 10 years, 40. Image 563 is an image representing Mama's age in 15 years, 45. Image 564 is an image representing Mama's age in 20 years, 50. Image 565 is an image representing Mama's age in 25 years, 55. The same applies to other family members.

[0041] Selection screen 500 displays GUI (Graphical User Interface) components that allow users to select life event ideas extracted (or generated) in a large-scale language model. For the sake of simplicity, Figure 5 shows some of the GUI components that allow users to select life event ideas for the eldest daughter, and some of the GUI components that allow users to select life event ideas for the eldest son. GUI components 510, 511, and 512 are GUI components that allow users to select ideas for the eldest daughter's life events. GUI components 520, 521, and 522 are GUI components that allow users to select ideas for the eldest son's life events. For the sake of simplicity, GUI components that allow users to select ideas for life events are also displayed for the father and mother. For example, GUI component 510 is a GUI component that allows users to select options such as kindergarten enrollment or nursery school enrollment.

[0042] In the example in Figure 5, a dropdown list is shown as an example of a GUI component, but any component that allows the user to select one idea from multiple life event ideas would suffice. For example, a carousel or image slider could be used, where the user can select one idea from multiple life event ideas by sliding multiple content items (kindergarten enrollment, nursery school enrollment, etc.) horizontally or horizontally through a predetermined operation such as clicking or tapping.

[0043] As mentioned above, the life events included in the selection screen 500 can be said to have been added based on the interview record data. As another example, since life events such as entering kindergarten, elementary school, and junior high school are predetermined according to age, the control unit 210 may be pre-configured according to the age of each member of the family. In other words, the life event candidates included in the selection screen 500 may include those that are pre-set according to the age of each member of the family, and those that are extracted or generated based on interview record data. Furthermore, for example, the control unit 210, etc., may make it possible to distinguish whether the life event candidates displayed on the selection screen 500 are pre-set or extracted or generated based on interview record data. Examples of distinguishing between pre-set and extracted or generated life event candidates include using different fonts for the life event candidates, using different font colors, or using predetermined marks to indicate each.

[0044] The selection screen 500 includes an illustration generation button 530. The illustration generation button 530 is selected when instructing the generation of illustrations for each member of the family.

[0045] Figure 6 is a sequence diagram (part 2) showing an example of information processing in the information processing system 1000. On the selection screen 500, when an idea for a life event for each member of the family is selected and the illustration generation button 530 is selected, in sequence SQ601, the control unit 310 of the client device 110 sends an illustration generation instruction to the server device 100. The illustration generation instruction includes information indicating that an illustration should be generated, and identification information to identify the family.

[0046] When an illustration generation instruction is received from the client device 110, in sequence SQ602, the control unit 210 of the server device 100 retrieves images of each member constituting the family from a predetermined storage area such as the storage unit 220, based on the identification information for identifying the family included in the illustration generation instruction. The control unit 210 also retrieves family composition information (more specifically, the age and hobbies and preferences of each family member) stored in association with each image of the family member. This process is an example of a process for retrieving information regarding the hobbies and preferences of each family member.

[0047] In sequence SQ603, the control unit 210 generates a prompt for illustration generation, which includes an image of each family member and an instruction to generate an illustration of each family member. This prompt includes the life events (content of the life event + timing of the life event) and the age and hobbies and preferences of each family member, as selected on the selection screen 500. More specifically, the prompt includes an instruction to generate an illustration that reflects the hobbies and preferences of each family member.

[0048] Furthermore, the prompt includes instructions to generate corresponding illustrations for each member of the family, showing how they change over time according to the timing of life events. More preferably, the prompt includes instructions to generate corresponding illustrations for each member of the family, showing how they change over time according to the timing of life events for members of the family who are below a threshold age. Even more preferably, the prompt includes instructions to generate corresponding illustrations for each member of the family, showing how they change over time according to the timing of life events up to the threshold age. The processing of sequence SQ603 is an example of a process that generates prompts for illustration generation based on image and interview record data. The prompts request the generation of illustrations for each of several life events related to the family in the life plan.

[0049] In sequence SQ604, the control unit 210 transmits the generated prompt to the illustration generation model. In sequence SQ605, the control unit 210 receives illustrations generated in the illustration generation model, corresponding to the timing of life events for each member of the family. Sequence SQ605, or sequences SQ604 and SQ605, is an example of a process that retrieves illustrations for each of several life events related to a family, which are generated by an illustration generation model based on prompts.

[0050] In sequence SQ606, the control unit 210 generates a life plan table screen, as shown in Figure 7 (described later), in which the generated illustrations are placed at the positions of each of the multiple life events in the life plan table that shows the life plan. The control unit 210 generates a life plan table screen in which the illustrations are arranged so that they do not overlap each other. The control unit 210 appropriately changes the size of the illustrations so that they do not overlap and includes them in the life plan table. In sequence SQ607, the control unit 210 transmits the generated life plan table screen to the client device 110. In sequence SQ608, the control unit 310 of the client device 110 displays the life plan table screen on the output unit 350. The life plan table screen is displayed, for example, on the client device 110's web browser.

[0051] Figure 7 shows an example of the life plan table screen 700. As shown in Figure 7, the life plan table screen 700 contains a life plan table. The life plan table contains the life plans of each member of the family. As shown in Figure 7, the illustrations placed in the life plan chart include corresponding illustrations for each member of the family. Illustration 710 is an illustration of the father. Along with the father's illustration, illustration 710 includes an illustration of a fish, reflecting the father's hobby of fishing. Illustration 720 is an illustration of the mother. Along with the mother's illustration, illustration 720 includes an illustration of her favorite travel destination, reflecting the mother's hobby of traveling. Illustration 730 is an illustration of the eldest daughter. Along with the eldest daughter's illustration, illustration 730 includes an illustration of a cucumber, reflecting the eldest daughter's hobby of cucumbers. Illustration 740 is an illustration of the eldest son. Along with the eldest son's illustration, illustration 740 includes an illustration of a ball, reflecting the eldest son's hobby of balls. This is an example of how illustrations placed in a life plan chart should reflect the hobbies and preferences of each member of the family. By using such illustrations, it is possible to create a life plan chart that is more relatable.

[0052] As shown in Figure 7, the illustrations placed in the life plan chart include corresponding illustrations for each family member, showing how they change over time according to the timing of life events. To explain using the eldest son as an example, illustrations 750, 751, 752, 753, 754, 755, and 756 are illustrations that are placed in the eldest son's life plan chart, and they are illustrations that have changed over time (more specifically, grown up) according to the timing of life events. These illustrations can be described as illustrations generated based on the instructions included in the prompt, which specify the generation of corresponding illustrations for each member of the family, showing how they change over time according to the timing of life events.

[0053] In the illustrations placed on the life plan chart, the illustrations of family members below the threshold age may include corresponding illustrations of each family member that have changed over time according to the timing of life events. Examples of threshold ages include 18 or 20 years old. These illustrations can be described as illustrations generated based on instructions included in the prompt, which specify that for family members below a certain age, the corresponding illustrations should be generated that change over time according to the timing of life events. By using illustrations like these, it becomes possible to generate a life plan chart that is more relatable and easier to understand.

[0054] Furthermore, the illustrations placed in the life plan chart may include illustrations of family members below the threshold age, showing how the timing of life events changes over time according to the timing of life events up to the threshold age. These illustrations can be said to be illustrations generated based on the prompt's instruction that for family members below a threshold age, the timing of life events up to the threshold age should be changed over time according to the timing of life events for each family member. By using illustrations like these, it becomes possible to generate a life plan chart that is more relatable and easier to understand.

[0055] As described above, according to Embodiment 1, it is possible to generate a life plan chart that makes a stronger impression on customers and is more likely to evoke empathy.

[0056] (Variation 1) The following describes Modification 1 of Embodiment 1. Modification 1 is included in Embodiment 1 and is not a different embodiment from Embodiment 1. Configurations and other aspects not described in Modification 1 are the same as in Embodiment 1.

[0057] In the modified example 1, the control unit 210 (hereinafter simply referred to as the control unit 210) generates a prompt in sequence SQ603 that further includes an instruction to generate multiple illustrations (which can also be called multiple illustration candidates) corresponding to each member of the family according to the timing of life events. The control unit 210 then receives the multiple illustration candidates generated in the illustration generation model, corresponding to the timing of life events for each member of the family. The control unit 210 generates a life plan table screen in which the generated multiple illustration candidates are placed at the positions of each of the multiple life events in the life plan table showing the life plan.

[0058] The control unit 210 selects an illustration from multiple displayed illustration candidates in response to a selection operation made via the life plan screen. The control unit 210 may also use a mechanism, such as a carousel or image slider, to allow the user to select one illustration from multiple illustration candidates by sliding multiple illustration candidates (for example, multiple illustration candidates corresponding to the life event of marriage) horizontally or otherwise in response to a predetermined operation such as a click or tap by the user. These multiple illustration candidates can be described as multiple illustration candidates generated based on the instructions included in the prompt, which specify that multiple illustration candidates should be generated for each member of the family depending on the timing of life events.

[0059] In other words, multiple candidate illustrations are displayed that can be placed at the corresponding life event location in the life plan chart. The control unit 210 selects an illustration from the multiple displayed candidate illustrations according to the user's selection operation.

[0060] According to the processing in Modification 1, it is possible to generate a life plan table that is more likely to evoke empathy.

[0061] (Modification 2) The following describes a modified example 2 of Embodiment 1. Modified example 2 is included in Embodiment 1 and is not a different embodiment from Embodiment 1. Configurations and other aspects not described in Modified Example 2 are the same as in Embodiment 1.

[0062] The control unit 210 of the modified example 2 (hereinafter simply referred to as the control unit 210) outputs a list of illustrations based on the attributes of the life events corresponding to the illustrations, along with the illustrations included in the life plan table. Figure 8 is an example of an illustration list (part 1). Figure 9 is an example of an illustration list (part 2). In the example in Figure 8, life events that involve ongoing or irregular financial expenses are extracted and output together as illustration list 800. Illustration list 800 includes life events such as "weekly family trips to a family restaurant," "monthly family trips to a hot spring," "annual domestic family trips," and "annual family trips to a theme park." In the example in Figure 9, life events that require financial assistance are extracted and output together as illustration list 900. Illustration list 900 includes life events such as "support for wedding expenses in 20 years," "purchase of a large two-door car that can accommodate the whole family within 2 years," "a two-story house within 10 years," and "a family trip abroad at least once before the children get jobs."

[0063] According to the processing method of Modification 2, an illustrated list corresponding to life events can be generated from the life plan table. By translating an abstract vision of the future into concrete events and creating a list corresponding to each life event, it becomes easier to visually grasp what might happen and when, and to prepare funds and other resources with ample time to spare.

[0064] (Variation 3) The following describes a modification 3 of Embodiment 1. Modification 3 is included in Embodiment 1 and is not a different embodiment from Embodiment 1. Configurations and other aspects not described in Modification 3 are the same as in Embodiment 1.

[0065] The control unit 210 in Modification 3 (hereinafter simply referred to as the control unit 210) may generate multiple prompts for extracting (or generating) life events, generate a screen in which one prompt can be selected from the multiple prompts, and send it to the client device 110. The control unit 210 may then send the selected prompt via the screen to a large language model. The screen may be configured to allow modification of the prompts.

[0066] Similarly, the control unit 210 may generate multiple prompts for illustration generation, generate a screen in which one prompt can be selected from the multiple prompts, and send it to the client device 110. The control unit 210 may then send the selected prompt via the screen to the illustration generation model. The screen may be configured to allow modification of the prompts.

[0067] According to variation 3, it is possible to select the appropriate prompt, extract (or generate) life events, or generate illustrations.

[0068] <Note> This embodiment includes the following disclosures.

[0069] (Note 1) An information processing method performed by an information processing system, We obtained data from interview records regarding the life plans of the customer's family. Obtain images of the aforementioned family, Based on the aforementioned image and the interview record data, illustrations are generated for each of the multiple life events related to the family in the life plan. The generated illustrations are placed at the positions of each of the multiple life events in the life plan chart that shows the life plan. Output the aforementioned life plan table. Information processing methods.

[0070] (Note 2) The information processing method described in Appendix 1, The aforementioned life plan table includes the life plans of each member of the family. Information processing methods.

[0071] (Note 3) The information processing method described in Appendix 2, The illustrations placed in the aforementioned life plan table include corresponding illustrations for each member of the family. Information processing methods.

[0072] (Note 4) The information processing method described in Appendix 3, The illustrations placed in the life plan table include corresponding illustrations of each member of the family, which change over time according to the timing of the life events. Information processing methods.

[0073] (Note 5) The information processing method described in Appendix 3, Among the illustrations placed in the life plan table, the illustrations of members of the family who are below the threshold age include corresponding illustrations of each member of the family that have been changed over time according to the timing of the life events. Information processing methods.

[0074] (Note 6) The information processing method described in Appendix 5, Among the illustrations placed in the life plan table, the illustrations of members of the family who are below the threshold age include corresponding illustrations of each member of the family whose life events change over time according to the timing of those life events up to the threshold age. Information processing methods.

[0075] (Note 7) An information processing method described in any one of the appendices 1 to 6, In the life plan chart, the illustrations are arranged so that they do not overlap with each other. Information processing methods.

[0076] (Note 8) An information processing method described in any one of the appendices 1 to 7, Multiple candidate illustrations are displayed to be placed at the corresponding life event location in the life plan chart. Select an illustration from a list of multiple illustration options displayed based on your selection. Information processing methods.

[0077] (Note 9) An information processing method described in any one of the appendices 1 to 8, Obtain information regarding the hobbies and preferences of each member of the aforementioned family, The illustrations placed in the aforementioned life plan table include illustrations that reflect the hobbies and preferences of each member of the family in question. Information processing methods.

[0078] (Note 10) An information processing method described in any one of the appendices 1 to 9, Based on the aforementioned image and the interview record data, a prompt for illustration generation is generated. The prompt is a prompt requesting the generation of illustrations for each of the multiple life events relating to the family in the life plan, Based on the prompt, obtain illustrations of each of the multiple life events related to the family that have been generated by the illustration generation model. Information processing methods.

[0079] (Note 11) An information processing method described in any one of the appendices 1 to 10, The aforementioned life events are pre-set. Information processing methods.

[0080] (Note 12) The information processing method described in Appendix 11, Based on the data from the aforementioned interview records, add life events. Information processing methods.

[0081] (Note 13) There is an information processing system, Having at least one control unit, The control unit, We obtained data from interview records regarding the life plans of the customer's family. Obtain images of the aforementioned family, Based on the aforementioned image and the interview record data, illustrations are generated for each of the multiple life events related to the family in the life plan. The generated illustrations are placed at the positions of each of the multiple life events in the life plan chart that shows the life plan. Output the aforementioned life plan table. Information processing system.

[0082] (Note 14) It is a program, On the computer, A program for executing the information processing method described in any one of the appendices 1 through 12.

[0083] Although embodiments have been described above, these are presented as examples and are not intended to limit the scope of the invention. Novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. The embodiments are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of Symbols]

[0084] 100: Server device 110: Client device 150: Network 210: Control Unit 220: Storage section 230: Communications Department 1000: Information Processing System

Claims

1. An information processing method performed by an information processing system, We obtained data from interview records regarding the life plans of the customer's family. Obtain images of the aforementioned family, Based on the aforementioned image and the interview record data, illustrations are generated for each of the multiple life events related to the family in the life plan. The generated illustrations are placed at the positions of each of the multiple life events in the life plan chart that shows the life plan. Output the aforementioned life plan table. Information processing methods.

2. The information processing method according to claim 1, The aforementioned life plan table includes the life plans of each member of the family. Information processing methods.

3. The information processing method according to claim 2, The illustrations placed in the aforementioned life plan table include corresponding illustrations for each member of the family. Information processing methods.

4. The information processing method according to claim 3, The illustrations placed in the life plan table include corresponding illustrations of each member of the family, which change over time according to the timing of the life events. Information processing methods.

5. The information processing method according to claim 3, Among the illustrations placed in the life plan table, the illustrations of members of the family who are below the threshold age include corresponding illustrations of each member of the family that have been changed over time according to the timing of the life events. Information processing methods.

6. The information processing method according to claim 5, Among the illustrations placed in the life plan table, the illustrations of members of the family who are below the threshold age include corresponding illustrations of each member of the family whose life events change over time according to the timing of those life events up to the threshold age. Information processing methods.

7. The information processing method according to claim 1, In the life plan chart, the illustrations are arranged so that they do not overlap with each other. Information processing methods.

8. The information processing method according to claim 1, Multiple candidate illustrations are displayed to be placed at the corresponding life event location in the life plan chart. Select an illustration from a list of multiple illustration options displayed based on your selection. Information processing methods.

9. The information processing method according to claim 1, Obtain information regarding the hobbies and preferences of each member of the aforementioned family, The illustrations placed in the aforementioned life plan table include illustrations that reflect the hobbies and preferences of each member of the family in question. Information processing methods.

10. The information processing method according to claim 1, Based on the aforementioned image and the interview record data, a prompt for illustration generation is generated. The prompt is a prompt requesting the generation of illustrations for each of the multiple life events relating to the family in the life plan, Based on the aforementioned prompt, obtain illustrations of each of the multiple life events related to the family, which are generated by the illustration generation model. Information processing methods.

11. The information processing method according to claim 1, The aforementioned life events are pre-set. Information processing methods.

12. The information processing method according to claim 11, Based on the data from the aforementioned interview records, add life events. Information processing methods.

13. There is an information processing system, Having at least one control unit, The control unit, We obtained data from interview records regarding the life plans of the customer's family. Obtain images of the aforementioned family, Based on the aforementioned image and the interview record data, illustrations are generated for each of the multiple life events related to the family in the life plan. The generated illustrations are placed at the positions of each of the multiple life events in the life plan chart that shows the life plan. Output the aforementioned life plan table. Information processing system.

14. It is a program, On the computer, A program for causing the information processing method described in any one of claims 1 to 12.

Images