Information processing methods, information processing systems, and programs

Abstract

Create an intuitively easy-to-understand process chart. [Solution] The present invention provides an information processing method to be executed by an information processing system. The information processing method includes the steps of: acquiring text data containing event information relating to an event; generating multiple illustrations corresponding to each of the multiple events based on the text data; placing the generated illustrations at the positions of each of the multiple events in a schedule; and outputting the schedule.

Description

Technical Field

[0001] The present invention relates to an information processing method, an information processing system, and a program. Background Art

[0002] Patent Document 1 discloses an apparatus that supports creation of a process chart. Prior Art Literature Patent Literature

[0003] Patent Document 1 Japanese Unexamined Patent Publication No. 2024-165804 Summary of the Invention Problems to be Solved by the Invention

[0004] There is a problem in creating an intuitively easy-to-understand process chart. Means for Solving the Problems

[0005] According to one aspect of the present disclosure, there is provided an information processing method executed by an information processing system. The information processing method includes the steps of: acquiring text data including event information related to events; generating a plurality of illustrations respectively corresponding to the plurality of events based on the text data; arranging the generated corresponding illustrations at respective positions of the plurality of events in the process chart; and outputting the process chart. Brief Description of the Drawings

[0006] [Figure 1] Figure 1 is a diagram showing an example of the system configuration of an information processing system. [Figure 2] Figure 2 is a diagram showing an example of the hardware configuration of a server device. [Figure 3] Figure 3 is a diagram showing an example of the hardware configuration of a client device. [Figure 4]Figure 4 is a sequence diagram (part 1) showing an example of information processing in an information processing system. [Figure 5] Figure 5 shows an example of the event selection screen for a project. [Figure 6] Figure 6 is a sequence diagram (part 2) showing an example of information processing in an information processing system. [Figure 7] Figure 7 shows an example of a project schedule screen. [Modes for carrying out the invention]

[0007] Embodiments of the present invention will be described below with reference to the drawings. The various features shown in the embodiments below (including modified examples; the same applies hereinafter) can be combined with each other.

[0008] <Embodiment 1> 1. System Configuration Diagram Figure 1 shows an example of the system configuration of the information processing system 1000. As shown in Figure 1, the information processing system 1000 includes, as a system configuration, a server device 100, a client device 110, a server device 120, and a server device 130. The server device 100 is an example of a computer. The server device 100, the client device 110, the server device 120, and the server device 130 are connected to each other via a network 150. The network 150 may include a WAN (Wide Area Network), a LAN (Local Area Network), and the Internet, or both. The network 150 is configured to allow communication between devices connected to the network 150 via wired and / or wireless connections. The information processing system 1000 is a system that provides so-called SaaS (Software as a Service) functionality.

[0009] The server device 100 performs information processing, etc., as described in Embodiment 1, in response to requests from the client device 110. Details of the processing performed by the server device 100 will be explained later using sequence diagrams, etc.

[0010] The client device 110 is a device operated by an operator of the information processing system 1000 and utilizes the functions provided by the server device 100. The client device 110 is, for example, a device operated by an executor involved in the execution of a project (hereinafter simply referred to as a project member, or member). Note that, for the sake of simplicity, Figure 1 shows only one client device 110 included in the information processing system 1000; however, the information processing system 1000 may include as many client devices 110 as there are project members. A project refers to the entire activity in which multiple events are interconnected and progress toward a single goal. A project can also be called a business. An event is an occurrence that takes place during the progress of a project. An event can also be called a task. In this specification, software development is used as an example of a project. However, this does not limit the embodiments.

[0011] Server device 120 is a device that provides services for large-scale language models. A large-scale language model is a computer language model composed of an artificial neural network with numerous parameters, and is a language model trained using self-supervised or semi-supervised learning with vast amounts of unlabeled text. Furthermore, a large-scale language model can also be described as an advanced artificial intelligence model that can learn from large amounts of text data and generate, translate, and answer questions using natural language used by humans. The large-scale language model may also be implemented on the server device 100.

[0012] Server device 130 is a device that provides an illustration generation model service. The illustration generation model is a computer illustration model composed of an artificial neural network with many parameters, and is a model that can generate new illustrations from input data such as text and / or images. The illustration generation model may also be implemented in the server device 100.

[0013] The specification uses a large-scale language model as an example of a model that outputs text and an illustration generation model as an example of a model that outputs illustrations. However, the information processing system 1000 may include a server device that provides a service for a single multimodal model having the functions of both a large-scale language model and an illustration generation model. The server device 100 may then utilize the multimodal model service. In such a configuration, the multimodal model is an example of an illustration generation model.

[0014] In Figure 1, a notebook PC (Personal Computer) is shown as an example of the client device 110, but it is not limited to a notebook PC; it could be a desktop PC, a tablet device, or a smartphone. The client device 110 can be any device that can input operator information and display the results of processing on the server device 100.

[0015] Here, the information processing system described in the claims may consist of multiple devices or of one device. If the information processing system described in the claims consists of one device, an example of such device is, for example, a server device 100. If the information processing system described in the claims consists of multiple devices, examples of multiple devices include, for example, a distributed system that provides the functions of server device 100, or server device 100 and server device 120, or server device 100 and server device 130, or server device 100 and at least one client device 110.

[0016] 2. Hardware Configuration (1) Hardware Configuration of Server Apparatus 100 FIG. 2 is a diagram showing an example of the hardware configuration of the server apparatus 100. As shown in FIG. 2, the server apparatus 100 includes, as a hardware configuration, a control unit 210, a storage unit 220, a communication unit 230, and an internal bus 240. The control unit 210, the storage unit 220, and the communication unit 230 are electrically connected via the internal bus 240.

[0017] The control unit 210 is configured by a CPU (Central Processing Unit) or the like, and controls the entire server apparatus 100.

[0018] The storage unit 220 is any one of an HDD (Hard Disk Drive), a ROM (Read Only Memory), a RAM (Random Access Memory), an SSD (Solid State Drive) or the like, or any combination thereof, and stores programs and data used when the control unit 210 executes processing based on the programs (for example, a compiler, a linker, a loader, a library, images of project members, interview record data, etc., which will be described later). In the following description, it is assumed that the server apparatus 100 includes a ROM, a RAM, an HDD, and an SSD as storage units. The storage unit 220 is an example of a storage medium. In the present specification, description is given on the assumption that data used when the control unit 210 executes processing based on a program is stored in the storage unit 220, but the data may be stored in a storage unit or the like of another apparatus capable of communicating with the server apparatus 100. That is, the data may be stored in a storage unit of any apparatus as long as the data can be referred to or acquired by the control unit 210. When the control unit 210 executes processing based on a program stored in the storage unit 220, the functions of the server apparatus 100 and the processing of the server apparatus 100 in the sequence diagrams shown in FIG. 4, FIG. 6, and the like which will be described later are realized.

[0019] If the source code's programming language is compiled, the control unit 210 uses a compiler to convert the source code stored in ROM, HDD, or SSD into object code. The control unit 210 then uses a linker to link the object code and libraries to generate a load module. The control unit 210 then uses a loader to load the load module into memory such as RAM. The control unit 210 executes the load module loaded into memory to realize the processing of the server device 100 in the sequence diagrams shown in Figures 4 and 6, which will be described later. If the source code's programming language is interpreted, the control unit 210 reads the source code stored in ROM, HDD, or SSD into RAM or other memory, and processes the loaded source code line by line during execution, performing lexical and / or syntactic analysis, thereby realizing the processing of the server device 100 as shown in the sequence diagrams in Figures 4 and 6 described later.

[0020] The communications unit 230 connects the server device 100 to the network 150 and manages communication with other devices.

[0021] (2) Hardware configuration of client device 110 Figure 3 shows an example of the hardware configuration of the client device 110. As shown in Figure 3, the client device 110 includes, as a hardware configuration, a control unit 310, a storage unit 320, an imaging unit 330, an input unit 340, an output unit 350, a communication unit 360, and an internal bus 370. The control unit 310, storage unit 320, imaging unit 330, input unit 340, output unit 3550, and communication unit 360 are electrically connected via the internal bus 370.

[0022] The control unit 310 is a CPU or the like, and controls the entire client device 110.

[0023] The storage unit 320 stores data used by the program and the control unit 310 when executing processing based on the program, and is one of the following: HDD, ROM, RAM, SSD, etc., or any combination thereof. The storage unit 320 is an example of a storage medium.

[0024] The imaging unit 330 is a camera or the like that captures images of a subject. An example of a subject is a project member operating the client device 110.

[0025] The input unit 340 is a keyboard and mouse, etc., which inputs information based on selection and / or input operations performed by the operator. The output unit 350 is a display or the like, which displays input information from the operator via the screen and the results of processing by the control unit 310.

[0026] The communications unit 360 connects the client device 110 to the network 150 and manages communication with other devices.

[0027] 3. Information Processing (1) Overview of the process The control unit 210 acquires text data containing event information related to an event. The text data is, for example, text data related to interview records of members regarding the project. Based on the text data, the control unit 210 generates multiple illustrations corresponding to each of the multiple events. The control unit 210 places the generated illustrations at the positions of each of the multiple events in the schedule. The control unit 210 outputs the schedule. The schedule is, for example, a schedule for the project. This process makes it possible to create an intuitively easy-to-understand process chart.

[0028] (2) Details of the process Figure 4 is a sequence diagram (part 1) showing an example of information processing in the information processing system 1000. In sequence SQ401, the control unit 310 of the client device 110 transmits images of each project member, along with member configuration information, to the server device 100. The member images can be considered digital photographs of the members. The member configuration information includes the affiliation, position, number of members, and each member's hobbies and preferences. In addition, each project member's image includes attribute information related to the image, such as who the member is. In sequence SQ401, the information transmitted by the client device 110 may be information captured and input by the client device 110, or it may be information that has been previously input by a member via a predetermined screen or the like.

[0029] When the server device 100 receives member images and member configuration information from the client device 110, in sequence SQ402, the control unit 210 of the server device 100 stores the member images and member configuration information together with identification information (e.g., project name) that identifies the project in a predetermined storage area such as the storage unit 220. The identification information that identifies the project is input by the client device 110, for example.

[0030] Note that the processing of sequence SQ401 can be performed at any time, as long as it is performed before sequence SQ601, which will be described later, regardless of the timing shown in Figure 4.

[0031] In sequence SQ403, the control unit 310 transmits project-related interview record data with each member to the server device 100. The project-related interview record data is either video or audio data of the project-related interview. For simplicity, this specification will describe the project-related interview record data as video data of the project-related interview. The interview record data may be recorded by the client device 110 during in-person interviews or during web interviews. An interview can also be referred to as a meeting.

[0032] When the server device 100 receives interview record data from the client device 110, in sequence SQ404, the control unit 210 of the server device 100 stores the interview record data regarding the project by the member, along with identification information that identifies the member, in a predetermined storage area such as the storage unit 220.

[0033] In sequence SQ405, the control unit 210 acquires member interview record data, identified by identification information, from a predetermined storage area such as the memory unit 220. The control unit 210 also acquires member configuration information, etc., identified by identification information, from a predetermined storage area such as the memory unit 220. In sequence SQ406, the control unit 210 extracts audio data based on the interview record data. The control unit 210 converts the extracted audio data into text data using speech recognition technology. The control unit 210 then adjusts the text data into grammatically correct and natural Japanese sentences using natural language processing technology. If the audio contains the voices of multiple speakers, the control unit 210 may use speaker separation technology to convert the audio into text data for each speaker.

[0034] In sequence SQ407, the control unit 210 generates prompts for extracting (or generating) event ideas for each member based on text data related to the interview record. A prompt is an instruction or question to AI (Artificial Intelligence), etc. This prompt may include, for example, speaker-separated text data of the interview record regarding the member's project, and an instruction to extract (or generate) event ideas (event content + event timing) for each member from the interview record text data. The prompt may also include member composition information as reference information for extracting (or generating) event ideas. Event ideas can also be called event candidates. Event timing can also be called the approximate timing of the event. The prompt may also include an instruction to predict the approximate amount for each event.

[0035] In sequence SQ408, the control unit 210 transmits the generated prompt to the large-scale language model. In sequence SQ409, the control unit 210 receives the event ideas for each member, which have been extracted (or generated) in the large-scale language model.

[0036] In sequence SQ410, the control unit 210 generates a selection screen for process schedule events (hereinafter also simply called the selection screen), as shown in Figure 5, which will be described later, and includes the events of each member that it has received. In sequence SQ411, the control unit 210 transmits the generated selection screen to the client device 110. In sequence SQ412, the control unit 310 of the client device 110 displays a selection screen on the output unit 350. The selection screen is displayed, for example, on the web browser of the client device 110.

[0037] As another example, the control unit 210 may transmit data used for the selection screen to the client device 110. The control unit 310 of the client device 110 may then generate and display the selection screen based on the received data. This also applies to the process schedule screen, which will be described later. However, for the sake of simplicity, the following explanation will assume that the control unit 210 generates the screen.

[0038] Figure 5 shows an example of the project event selection screen 500. The specification will explain using an example of generating a project schedule for members A, B, C, and D. As shown in Figure 5, the project event selection screen 500 is configured to display events for each member. Row 550 is the row that displays A's project. Row 551 is the row that displays B's project. Row 552 is the row that displays C's project. Row 553 is the row that displays D's project.

[0039] Vertical line 560 represents the present time. This can also be described as the time when the selection screen 500 is displayed. Vertical line 561 represents one month later. Vertical line 562 represents two months later. Vertical line 563 represents three months later. The time period represented by each vertical line varies depending on the data generated in the large-scale language model.

[0040] Selection screen 500 displays GUI (Graphical User Interface) components that allow users to select event ideas extracted (or generated) in a large-scale language model. GUI components 510 and 511 are GUI components that allow selection of event ideas for person A, respectively. GUI component 520 is a GUI component that allows selection of event ideas for person B, GUI component 530 is a GUI component that allows selection of event ideas for person C, and GUI component 540 is a GUI component that allows selection of event ideas for person D. The width of each GUI component indicates the length of the event period.

[0041] Note that while Figure 5 shows a dropdown list as an example of a GUI component, any mechanism that allows the user to select one idea from multiple event ideas in the project is acceptable. For example, a carousel or image slider could be used, where the user selects one idea from multiple event ideas by sliding multiple content items horizontally or horizontally through a predetermined operation such as clicking or tapping.

[0042] As mentioned above, the project events included in selection screen 500 can be said to have been added based on the interview record data.

[0043] The selection screen 500 includes a generate button 570. The generate button 570 is an illustration generation button that is selected when instructing the generation of illustrations for each member and for the selected event.

[0044] Figure 6 is a sequence diagram (part 2) showing an example of information processing in the information processing system 1000. On the selection screen 500, when each member's event is selected and the generate button 570 is selected, in sequence SQ601, the control unit 310 of the client device 110 sends an illustration generation instruction to the server device 100. The illustration generation instruction includes information indicating that an illustration should be generated, and identification information to identify the member.

[0045] When an illustration generation instruction is received from the client device 110, in sequence SQ602, the control unit 210 of the server device 100 retrieves images of each member of the corresponding project from a predetermined storage area such as the storage unit 220, based on the identification information that identifies the member included in the illustration generation instruction. The control unit 210 may also retrieve the member configuration information stored in association with each image of the project member.

[0046] In sequence SQ603, the control unit 210 generates a prompt for illustration generation, which includes an image of each project member and an instruction to generate an illustration of each member. This prompt also includes the event for each member (event content + event timing) selected on the selection screen 500, and an instruction to generate an illustration corresponding to the event. The processing of sequence SQ603 is an example of a process that generates prompts for illustration generation based on text data. The prompts request the generation of illustrations for each of several events related to members in the project.

[0047] In sequence SQ604, the control unit 210 transmits the generated prompt to the illustration generation model. In sequence SQ605, the control unit 210 receives the illustrations of each member and the illustrations corresponding to the project events, which are generated in the illustration generation model. Sequence SQ605, or sequences SQ604 and SQ605, is an example of a process that retrieves multiple illustrations corresponding to multiple events, which are generated by the illustration generation model based on prompts.

[0048] In sequence SQ606, the control unit 210 generates a schedule screen, as shown in Figure 7 (described later), in which the generated illustrations are placed at the positions of multiple events in the schedule representing the project. The control unit 210 generates a schedule screen in which the illustrations are arranged so that they do not overlap each other. The control unit 210 appropriately changes the size of the illustrations so that they do not overlap and includes them in the schedule. In sequence SQ607, the control unit 210 transmits the generated process schedule screen to the client device 110. In sequence SQ608, the control unit 310 of the client device 110 displays the process schedule screen on the output unit 350. The process schedule screen is displayed, for example, on the web browser of the client device 110.

[0049] Figure 7 shows an example of the process schedule screen 700. As shown in Figure 7, the schedule screen 700 contains the schedule. The schedule includes events for each member. As shown in Figure 7, the illustrations placed in the process chart include the corresponding illustration for each member. Illustration 710 is an illustration of Person A. Illustration 710 may include illustrations of Person A along with illustrations that reflect Person A's hobbies and preferences. Illustration 720 is an illustration of Person B. Illustration 720 may include illustrations that reflect Person B's hobbies and preferences. Illustration 730 is an illustration of Person C. Illustration 730 may include illustrations that reflect Person C's hobbies and preferences. Illustration 740 is an illustration of Person D. Illustration 740 may include illustrations that reflect Person D's hobbies and preferences. This is one example of how illustrations placed in a project schedule may reflect the hobbies and preferences of each member. By using such illustrations, it is possible to create a project schedule that is more relatable and easier to understand.

[0050] As shown in Figure 7, the illustrations placed in the process chart include illustrations corresponding to events, depending on the timing of the events. Illustration 711 is an illustration corresponding to a design event performed by Person A. Illustration 712 is an illustration corresponding to a development event performed by Person A. Illustration 721 is an illustration corresponding to a development event conducted by Person B. Illustration 731 is an illustration corresponding to the test event that person C is taking. Illustration 741 is an illustration corresponding to the requirements definition event performed by person D. The width of the outer frame of each illustration indicates the length of the period related to that event.

[0051] As described above, according to Embodiment 1, it is possible to create a process chart that is intuitively easy to understand.

[0052] (Variation 1) The following describes Modification 1 of Embodiment 1. Modification 1 is included in Embodiment 1 and is not a different embodiment from Embodiment 1. Configurations and other aspects not described in Modification 1 are the same as in Embodiment 1.

[0053] In the modified example 1, the control unit 210 (hereinafter simply referred to as the control unit 210) generates a prompt in sequence SQ603 that further includes an instruction to generate multiple illustrations (which can also be called multiple illustration candidates) corresponding to the timing of the event. The control unit 210 then receives the multiple illustration candidates corresponding to the timing of the event, which were generated in the illustration generation model. The control unit 210 generates a project schedule screen in which the generated multiple illustration candidates are placed at the positions of each of the multiple events in the project schedule.

[0054] The control unit 210 selects an illustration from multiple displayed illustration candidates in response to a selection operation made via the process chart screen. The control unit 210 may also use a mechanism, such as a carousel or image slider, to allow the user to select one illustration from multiple illustration candidates by sliding multiple illustration candidates (for example, multiple illustration candidates corresponding to an event such as requirements definition) horizontally or otherwise in response to a predetermined operation such as a click or tap by the operator. These multiple illustration candidates can be described as multiple illustration candidates generated based on the instructions included in the prompt, which specify that multiple illustration candidates corresponding to each member should be generated depending on the timing of the event.

[0055] In other words, multiple candidate illustrations are displayed that can be placed at the corresponding event location in the process chart. The control unit 210 selects an illustration from among several displayed illustration candidates in response to the user's selection operation.

[0056] According to the process in Modification 1, a more intuitive and easy-to-understand process chart can be created.

[0057] (Modification 2) The following describes a modified example 2 of Embodiment 1. Modified example 2 is included in Embodiment 1 and is not a different embodiment from Embodiment 1. Configurations and other aspects not described in Modified Example 2 are the same as in Embodiment 1.

[0058] The control unit 210 of Modification 2 (hereinafter simply referred to as the control unit 210) extracts keywords related to events from text data. Keywords are words, phrases, etc. that indicate the goals and / or content of a project, and may be set for each project or generated for each project by a large-scale language model, etc. Based on the text data and keywords, the control unit 210 generates prompts for illustration generation, which include instructions to generate multiple illustrations corresponding to each of the multiple events and illustrations corresponding to the keywords. The control unit 210 then transmits the generated prompts to the illustration generation model.

[0059] According to the processing method of Modification 2, keywords are extracted from the project interview record data, illustrations corresponding to the extracted keywords are generated and displayed in the appropriate positions on the project schedule, making it possible to create a more intuitive and easy-to-understand project schedule. In addition, by concisely expressing the purpose and content of project events (or tasks) using keywords, the overall structure of the project can be easily grasped visually. Furthermore, this allows all members involved in the project to share not only their own areas of responsibility but also the overall goals and progress of the project.

[0060] <Note> This embodiment includes the following disclosures.

[0061] (Note 1) An information processing method performed by an information processing system, Retrieve text data containing event information about the event, Based on the aforementioned text data, multiple illustrations corresponding to each of the multiple events are generated. The generated illustration is placed at the location of each of the multiple events in the process chart. Output the aforementioned process schedule. Information processing methods.

[0062] (Note 2) The information processing method described in Appendix 1, In the aforementioned process chart, the illustrations are arranged so that they do not overlap with each other. Information processing methods.

[0063] (Note 3) The information processing method described in Appendix 1 or Appendix 2, Multiple candidate illustrations are displayed to be placed at the corresponding event location in the aforementioned process chart. The user selects an illustration from a list of multiple options displayed based on their selection. Information processing methods.

[0064] (Note 4) An information processing method described in any one of the appendices 1 to 3, Based on the aforementioned text data, a prompt for illustration generation is generated. The aforementioned prompt is a prompt requesting the generation of an illustration, Based on the aforementioned prompts, obtain multiple illustrations corresponding to each of the multiple events, which are generated by the illustration generation model. Information processing methods.

[0065] (Note 5) An information processing method described in any one of the appendices 1 to 4, Keywords related to the event are extracted from the aforementioned text data, Based on the aforementioned text data and keywords, multiple illustrations corresponding to each of multiple events and illustrations corresponding to the aforementioned keywords are generated. Information processing methods.

[0066] (Note 6) There is an information processing system, Having at least one control unit, The control unit, Retrieve text data containing event information about the event, Based on the aforementioned text data, multiple illustrations corresponding to each of the multiple events are generated. The generated illustration is placed at the location of each of the multiple events in the process chart. Output the aforementioned process schedule. Information processing system.

[0067] (Note 7) It is a program, On the computer, A program for executing the information processing method described in any one of the appendices 1 through 5.

[0068] Although embodiments have been described above, these are presented as examples and are not intended to limit the scope of the invention. Novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. The embodiments are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. [Explanation of Symbols]

[0069] 100: Server device 110: Client device 150: Network 210: Control Unit 220: Storage section 230: Communications Department 1000: Information Processing System

Claims

1. An information processing method performed by an information processing system, Retrieve text data containing event information about the event, Based on the aforementioned text data, multiple illustrations corresponding to each of the multiple events are generated. The generated illustration is placed at the location of each of the multiple events in the process chart. Output the aforementioned process schedule. Information processing methods.

2. The information processing method according to claim 1, In the aforementioned process chart, the illustrations are arranged so that they do not overlap with each other. Information processing methods.

3. The information processing method according to claim 1, Multiple candidate illustrations are displayed to be placed at the corresponding event location in the aforementioned process chart. The user selects an illustration from a list of multiple options displayed based on their selection. Information processing methods.

4. The information processing method according to claim 1, Based on the aforementioned text data, a prompt for illustration generation is generated. The aforementioned prompt is a prompt requesting the generation of an illustration, Based on the aforementioned prompts, obtain multiple illustrations corresponding to each of the multiple events, which are generated by the illustration generation model. Information processing methods.

5. The information processing method according to claim 1, Keywords related to the event are extracted from the aforementioned text data, Based on the aforementioned text data and keywords, multiple illustrations corresponding to each of multiple events and illustrations corresponding to the aforementioned keywords are generated. Information processing methods.

6. There is an information processing system, Having at least one control unit, The control unit, Retrieve text data containing event information about the event, Based on the aforementioned text data, multiple illustrations corresponding to each of the multiple events are generated. The generated illustration is placed at the location of each of the multiple events in the process chart. Output the aforementioned process schedule. Information processing system.

7. It is a program, On the computer, A program for causing the information processing method described in any one of claims 1 to 5.

Images