Information processing methods, programs, and information processing systems

Abstract

It is possible to create reports based on surveys. [Solution] According to an information processing method according to one embodiment, the method includes: a questionnaire acquisition step of acquiring completed questionnaires; an image acquisition step of acquiring images over a predetermined period; a text conversion step of converting the content of the images acquired in the image acquisition step into text; a selection step of selecting images acquired in the image acquisition step based on the completed questionnaires and the text converted; and a generation step of generating a report based on the completed questionnaires, the text converted, and the images selected in the selection step.

Description

Technical Field

[0004] ,

[0006] , , ,

[0005] , , ,

[0001] The present invention relates to an information processing method, a program, and an information processing system.

Background Art

[0002] Patent Document 1 discloses a system that provides a guardian of an elderly person with a life report in which the content of services provided by staff of a support company to the elderly person is organized in association with life scenes and dates and provided to the guardian of the elderly person who lives far away from the elderly person.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the case of the above prior art, a report is created by associating service contents with life scenes and dates, and the opinions of family members (guardians, etc.) are not taken into account, and the information that family members (guardians, etc.) want to know may not be satisfied.

[0005] In consideration of the above facts, an object of the present invention is to obtain an information processing method, a program, and an information processing system capable of creating a report based on a questionnaire.

Means for Solving the Problems

[0006] According to an information processing method according to one embodiment, the method includes: a questionnaire acquisition step of acquiring completed questionnaires; an image acquisition step of acquiring images over a predetermined period; a text conversion step of converting the content of the images acquired in the image acquisition step into text; a selection step of selecting images acquired in the image acquisition step based on the completed questionnaires and the text converted; and a generation step of generating a report based on the completed questionnaires, the text converted, and the images selected in the selection step.

[0007] According to a program of one embodiment, the information processing device is made to execute an information processing method that includes: a questionnaire acquisition step of acquiring completed questionnaires; an image acquisition step of acquiring images over a predetermined period; a text conversion step of converting the contents of the images acquired in the image acquisition step into text; a selection step of selecting images acquired in the image acquisition step based on the completed questionnaires and the text converted; and a generation step of generating a report based on the completed questionnaires, the text converted, and the images selected in the selection step.

[0008] According to an information processing system according to one embodiment, the information processing system includes an information processing device and performs the following steps: a questionnaire acquisition step of acquiring completed questionnaires; an image acquisition step of acquiring images over a predetermined period; a text conversion step of converting the contents of the images acquired in the image acquisition step into text; a selection step of selecting images acquired in the image acquisition step based on the completed questionnaires and the text converted; and a generation step of generating a report based on the completed questionnaires, the text converted, and the images selected in the selection step. [Effects of the Invention]

[0009] According to one embodiment, a report can be created based on a questionnaire. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an example of the configuration of an information processing system according to the embodiment. [Figure 2] This figure shows an example of the server hardware configuration according to the embodiment. [Figure 3] This figure shows an example of the hardware configuration of a parental device according to the embodiment. [Figure 4] This figure shows an example of the functional configuration of the server according to the embodiment. [Figure 5] This figure shows an example of the functional configuration of a parental device according to the embodiment. [Figure 6] This figure shows an example of the functional configuration of an employee terminal according to the embodiment. [Figure 7] This flowchart shows an example of the process of generating a report performed by the information processing system according to the embodiment. [Figure 8] This figure shows an example of a questionnaire screen used by parents according to the embodiment. [Figure 9] This figure shows an example of a report screen according to the embodiment. [Modes for carrying out the invention]

[0011] An embodiment of the information processing system according to the present invention will be described below with reference to Figures 1 to 6. In each figure, identical or equivalent components and parts are given the same reference numerals. Furthermore, the dimensional ratios in the drawings are exaggerated for illustrative purposes and may differ from actual ratios.

[0012] (System Overview) First, an overview of the information processing system 10 according to this embodiment will be described. The information processing system 10 according to this embodiment is a system that can create reports based on questionnaires. In this embodiment, the information processing system 10 is a system that can create reports by combining questionnaires answered by parents on parent terminal 2 and photographs taken by staff terminal 3 at a childcare facility over a predetermined period of time. Specifically, based on questionnaires answered in advance by parents on parent terminal 2, the system can select corresponding photographs from photographs taken by staff terminal 3 over a predetermined period of time, and generate a report corresponding to the questionnaire along with the selected photographs. Furthermore, in this embodiment, the system will be described as a system in which childcare workers take photographs of children at facilities such as kindergartens and nursery schools, and presents parents with a report on the children over a predetermined period of time based on the results of questionnaires from the children's parents. However, it is not limited to this, and for example, it may be used in nursing care facilities to take photographs of the care recipients and present reports to families on a regular basis, or in facilities for extracurricular activities to take photographs of the students and present reports to parents on a regular basis.

[0013] (System Configuration) Figure 1 shows an example of the configuration of the information processing system 10 according to this embodiment. As shown in Figure 1, the information processing system 10 according to this embodiment comprises a server device 1, a parent terminal 2, and an employee terminal 3, all of which are connected to each other via a network N so as to be able to communicate with each other. The network N is, for example, a wired LAN (Local Area Network), a wireless LAN, the Internet, a public telephone network, a mobile data communication network, or a combination thereof. In the example in Figure 1, the information processing system 10 comprises one server device 1, one parent terminal 2, and one employee terminal 3, but it may also comprise multiple units of each.

[0014] The server device 1 is an example of an information processing device that acquires the questionnaire answered by the guardian on the guardian terminal 2 and the photos taken on the staff terminal 3, and creates a report for the guardian. The server device 1 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, or a combination thereof. The specific configuration and operation of the server device 1 will be described later.

[0015] The guardian terminal 2 is an example of an information processing device that performs operations for input and display of various information. The guardian terminal 2 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, a wearable device, or a combination thereof. In the present embodiment, an information processing device that can answer a questionnaire and display the report created by the server device 1 is taken as an example.

[0016] The staff terminal 3 is an example of an information processing device that performs operations for input and display of various information. The staff terminal 2 may be a PC (Personal Computer), a smartphone, a tablet terminal, a server device, a microcomputer, a wearable device, or a combination thereof. In the present embodiment, an information processing device that can take photos of children attending a facility (such as a nursery or kindergarten) and display the report created by the server device 1 is taken as an example.

[0017] (Hardware Configuration - Server) FIG. 2 is a block diagram showing the hardware configuration of the server device 1. The server device 1 includes a processor 101, a memory 102, a storage 103, and a communication I / F 104, which are communicably connected to each other via a bus B.

[0018] The processor 101 controls each component of the server device 1 and realizes the functions of the server device 1 by expanding and executing various programs stored in the storage 103 in the memory 102. The programs executed by the processor 101 include, but are not limited to, the OS (Operating System) and various programs described later. By executing these programs by the processor 101, a part of the state visualization method according to the present embodiment is realized. The processor 101 is, for example, a CPU (Central Processing Unit), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), DSP (Digital Signal Processor), or a combination thereof.

[0019] The memory 102 is, for example, a ROM (Read Only Memory), RAM (Random Access Memory), or a combination thereof. The ROM is, for example, a PROM (Programmable ROM), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), or a combination thereof. The RAM is, for example, a DRAM (Dynamic RAM), SRAM (Static RAM), MRAM (Magnetoresistive RAM), or a combination thereof.

[0020] The storage 103 stores the OS, various programs described later, and various data. The storage 103 is, for example, a flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), SCM (Storage Class Memories), or a combination thereof.

[0021] The communication interface 104 is an interface for connecting the server device 1 to external devices, including parent terminals 2 and staff terminals 3, via the network N, and for controlling communication. The communication interface 104 is, for example, an adapter compliant with Bluetooth®, Wi-Fi®, ZigBee®, Ethernet®, or optical communication (e.g., Fibre Channel).

[0022] (Hardware configuration - User terminal) Figure 3 is a block diagram showing the hardware configuration of the parent terminal 2. The parent terminal 2 comprises a processor 201, memory 202, storage 203, communication I / F 204, input / output I / F 205, input device 206, and output device 207, all of which are connected to each other via bus B for communication. The communication I / F 204 is an interface for connecting the parent terminal 2 to an external device, including the server device 1, via network N, and for controlling communication. The communication I / F 204 is, for example, an adapter compliant with Bluetooth®, Wi-Fi®, ZigBee®, Ethernet®, or optical communication (e.g., Fibre Channel).

[0023] The input / output interface 205 is an interface for connecting the input device 206 and the output device 207 to the parent terminal 2. The input device 206 is, for example, a mouse, keyboard, touch panel, microphone, scanner, camera, various sensors, operation buttons, or a combination thereof. The output device 207, as a user interface, is, for example, a display, projector, printer, speaker, vibrator, or a combination thereof. In this embodiment, as an example, the output device 207 and the input device 206 are configured as an integrated touch panel display.

[0024] In this embodiment, the program may be written to memory 202 or storage 203 during the manufacturing stage of the server device 1, or it may be provided to the server device 1 via network N. It may also be provided to the server device 1 via a non-temporary, computer-readable recording medium such as a disk medium (not shown). Furthermore, since the staff terminal 3 has the same configuration as the parent terminal 2, its description is omitted.

[0025] (Functional Configuration - Server) Next, the functional configuration of server device 1 will be described. Figure 4 shows an example of the functional configuration of server device 1. When executing various programs, server device 1 uses the above hardware resources to realize various functions. Server device 1 has a communication unit 11, a storage unit 12, and a control unit 13 as the functional configuration realized by server device 1. Each functional configuration is realized when the processor 101 reads a program stored in memory 102 or storage 103 and executes it.

[0026] Furthermore, the memory unit 12 stores an image analysis model 121 that analyzes images taken by the staff terminal 3 and converts the image content into text, and an image selection model 122 that selects images taken by the staff terminal 3 for use in report generation. The memory unit 12 also stores a report generation model 123 that generates a report based on the results of the parent questionnaire, the content converted into text by the image analysis model 121, and the images selected by the image selection model 122.

[0027] The control unit 13 includes a questionnaire acquisition unit 131 that acquires questionnaire results answered by the parent on the parent terminal 2, and an image acquisition unit 132 that acquires images taken by the staff terminal 3. It also includes a text unit 133 that uses an image analysis model 121 to convert the content of images into text and save it together with the images, a selection unit 134 that uses an image selection model 122 to select images to be used when generating reports, and a generation unit 135 that generates reports using a report generation model 123.

[0028] In this embodiment, the generation unit 135 generates a report using the report generation model 123, based on the results of the questionnaire answered by the parent on the parent terminal 2 and the images of the child taken by the staff terminal 3. In other words, three learning models are used to generate the report. However, this is not limited to this; for example, any model equivalent to the above three learning models can be used, and the type of model is not limited. Further details will be described later.

[0029] (Functional Configuration - Parental Device) Next, the functional configuration of the parent terminal 2 will be described. Figure 5 is a diagram showing an example of the functional configuration of the parent terminal 2. When executing various programs, the parent terminal 2 uses the above hardware resources to realize various functions. The parent terminal 2 has a communication unit 21, a storage unit 22 in which the program 221 is stored, and a control unit 23 as the functional configuration realized by the parent terminal 2. Each functional configuration is realized by the processor 201 reading and executing the program 221 stored in the memory 202 or storage 203. The control unit 23 includes an information acquisition control unit 231 that acquires information sent from the server device 1, and a display unit 232 that displays the acquired information on the output device 207. The control unit 23 also includes a questionnaire unit 233 that presents questionnaires sent from the server device 1 and transmits the answers to the questionnaires from the parent to the server device 1.

[0030] (Functional Configuration - Staff Terminals) Next, the functional configuration of the staff terminal 3 will be described. Figure 6 is a diagram showing an example of the functional configuration of the staff terminal 3. When executing various programs, the staff terminal 3 uses the above hardware resources to realize various functions. The functional configuration realized by the staff terminal 3 includes a communication unit 31, a storage unit 32 in which the program 321 is stored, and a control unit 33. Each functional configuration is realized when the processor 201 reads and executes the program 221 stored in the memory 202 or storage 203. The control unit 33 includes an information acquisition control unit 331 that acquires information sent from the server device 1, and a display unit 332 that displays the acquired information on the output device 207. The control unit 33 also includes a shooting unit 333 that photographs the children of the facility and transmits it to the server device 1. Note that the photographs of the children may be taken by staff operating the terminal, or they may be taken automatically by the staff terminal 3.

[0031] Next, the processing flow of the information processing system according to this embodiment will be explained using Figures 7 to 9. Figure 7 is a flowchart showing an example of the process of generating a report executed by the information processing system 10 according to this embodiment. Figure 8 is an example of the questionnaire screen entered by the parent according to this embodiment. Figure 9 is an example of the report screen according to this embodiment.

[0032] First, in step S101, the questionnaire acquisition unit 131 acquires the questionnaire that the parent has answered on the questionnaire unit 233 of the parent terminal 2. Here, questionnaire A1 in Figure 8 is displayed on the display unit 232 of the parent terminal 2 and includes question A2, which shows the questions to the parent, and answer A3, which the parent answers. The parent writes their answer to answer A3 on the questionnaire unit 233. At this time, the parent can input in natural language. In this embodiment, the parent inputs answer A3 to questionnaire A1 in natural language, but this is not limited to this, and answer A3 to questionnaire A1 may be created by selecting from multiple-choice items, for example, or it may be a combination of natural language and selection, and may be changed as needed. Also, in this embodiment, there are four questions for question A2, but this is not limited to this, and the number of questions may be changed as needed. Furthermore, questionnaire A1 may be requested from parents periodically, for example, once every three months, once every six months, once a year, etc., and the content of the questions may be changed at each timing. In addition, new questions may be added or modified by referring to answers to past questions. In step S101 of Figure 7, the questionnaire acquisition unit 131 acquires the completed questionnaire A1 from the parent terminal 2.

[0033] Next, in step S102, the image acquisition unit 132 acquires images of the children taken by the shooting unit 333 of the staff terminal 3 during a predetermined period. The images acquired at this time are, for example, images taken during a predetermined period such as once every three months, six months, or once a year. In this embodiment, images are acquired within a limited period in order to generate a report on the children's situation during the predetermined period. The predetermined period may be changed as appropriate depending on which period the report to be generated covers.

[0034] Next, in step S103, the text unit 133 uses the image analysis model 121 to analyze the image acquired in step S102, converts the image content into text, and saves it along with the image. Here, we will explain using image R3 in Figure 9 as an example. When converting the content of image R3 into text, we list the items. That is, we express in text the three children, one balloon, the balloon flying, each child's clothing, actions, position, facial expression, etc., that are in image R3, and save the text along with the image. At this time, for example, by registering individual faces in advance, it is possible to determine who is in the image using facial recognition. Therefore, in this case, individuals may be identified during the text conversion process. This text conversion process is performed on all images acquired in step S102 for a predetermined period.

[0035] Next, in step S104, the selection unit 134 uses the image selection model 122 to select images to be used for report generation. Specifically, the questionnaire responses obtained in step S101 are analyzed, for example, using an LLM (Large Language Model), and images to be used for report generation are selected from the text converted in step S103. In addition to the concerns expressed in the questionnaire, images are selected from the text of the images that are judged to be "appropriate as visual information to convey the developmental status, cognitive aspects, and psychological individuality of the child."

[0036] Next, in step S105, the generation unit 135 generates a report using the parent questionnaire obtained in step S101, the text of the images analyzed in step S103, and the images selected in step S104. Here, Report R1 in Figure 9 shows an example of a report generated in step S105. Report R1 includes text R2 and images R3. Text R2 sets items based on the answers to the parent questionnaire obtained in step S101 and generates corresponding text. In other words, since the answers to the parent questionnaire are what the parents want to know about the children's situation at the facility, a report is generated based on the answers to the questionnaire.

[0037] Specifically, text R2 is generated using the text obtained by converting images to text in step S103. In other words, the responses to the parent questionnaire are analyzed, for example, using LLM to set the items for text R2, extract the text, and generate text R2. Then, image R3 is attached to the image selected in step S104. At this time, for example, when selecting images to be used in the report, predetermined conditions may be set in advance, such as the child's face being in the center and the face being clearly visible. Furthermore, the items and content of text R2 are set and the content is generated based on the questionnaire responses, the content of the text obtained by converting images to text in step S103, and perspectives from general cognitive science, infant developmental psychology, and educational science / childcare science.

[0038] In this embodiment, Report R1 shown in Figure 9 was described as an example of a report, but it is not limited to this. For example, text may be generated with more items, and many images may also be used. For example, two types of reports may be generated: a digest version as an overview of the report and a detailed version showing the details of each item. Also, when analyzing images, if feature parts are extracted even if they were not present in the questionnaire responses, those feature parts may be added to generate the report. In this case, it is possible to learn about characteristics of the child that parents may not have noticed.

[0039] Thus, according to this embodiment, it is possible to create a report based on a questionnaire. In other words, based on the questionnaire answered by the parent, it is possible to generate a report document and select images to attach to the report. As a result, since the report is generated based on the information that the parent wants to know, a customized report tailored to each individual can be generated.

[0040] In this embodiment, images captured by the camera unit 333 of the staff terminal 3 are used, but the embodiment is not limited to this. For example, a camera may be installed within the facility, and images from that camera may be used. Alternatively, both images captured by a camera within the facility and images captured by the camera unit 333 of the staff terminal 3 may be used. Furthermore, in this embodiment, a report was generated using three types of models: an image analysis model 121, an image selection model 122, and a report generation model 123. However, the embodiment is not limited to this. Any type of learning model with similar functionality is acceptable.

[0041] Furthermore, in this embodiment, a report for parents was generated using completed questionnaires from parents, text generated from images taken with the staff terminal 3, and selected images, but it is not limited to this. In addition to questionnaires and images, reports may also be generated using, for example, daily records of children written by childcare workers, video and audio data of children's behavior during childcare, or audio data of staff meetings. In this case, each piece of data (audio data or audio data) is converted into text using speech recognition (existing technology may be used), and the report is generated using LLM.

[0042] As described above, according to this embodiment, a report can be generated based on the questionnaire answered by the guardian, and images to be attached to the report can be selected. However, this embodiment is not limited to facilities such as nurseries and kindergartens. For example, in a nursing home, the care recipient's condition may be photographed and a report may be presented to the family on a regular basis, or in a facility for extracurricular activities, the student's condition may be photographed and a report may be presented to the guardian on a regular basis.

[0043] Although one embodiment of the present invention has been described above, the present invention is not limited to the embodiments described above, and any modifications, improvements, etc. that can achieve the objectives of the present invention are included in the present invention.

[0044] Furthermore, the series of processes described above can be executed by hardware or by software. In other words, the functional configuration is merely illustrative and not particularly limited. That is, it is sufficient for the information processing system to have the functionality to execute the series of processes described above as a whole, and there is no particular limitation on what functional blocks are used to realize this functionality. Also, the location of the functional blocks is not particularly limited and can be arbitrary. For example, the functional blocks of a server may be delegated to a user terminal, etc. Conversely, the functional blocks of a user terminal may be delegated to a server, etc. Moreover, a single functional block may be composed of hardware alone, software alone, or a combination of both.

[0045] For example, when a series of processes are executed by software, the programs that make up that software are installed on a computer or other device from a network or storage medium. The computer may be a computer built into dedicated hardware. Alternatively, the computer may be a computer capable of performing various functions by installing various programs, such as a server, a general-purpose smartphone, or a personal computer.

[0046] Furthermore, for example, a recording medium containing such a program may consist not only of removable media (not shown) distributed separately from the main unit of the device to provide the program to the user, but also of a recording medium provided to the user in a state where it is pre-installed in the main unit of the device.

[0047] In this specification, the step of describing a program to be recorded on a recording medium includes not only processes that are performed chronologically in that order, but also processes that are not necessarily performed chronologically, but are executed in parallel or individually. Furthermore, in this specification, the term "system" refers to an overall system composed of multiple devices, means, etc. [Explanation of Symbols]

[0048] 10. Information Processing Systems 1 Server Device 2 Parent / Guardian Devices 3 staff terminals 131 Questionnaire Acquisition Department 132 Image acquisition unit 133 Text Section 134 Selection Department 135 generation part

Claims

1. An information processing method performed by an information processing device, Steps for obtaining completed questionnaires, An image acquisition step to acquire images over a predetermined period, A text conversion step, which converts the content of the image acquired in the image acquisition step into text, A selection step in which images acquired in the image acquisition step are selected based on the completed questionnaire and the transcribed text, Based on the completed questionnaire, a generation step is performed to generate a report based on the text that has been converted into text and the images selected in the selection step. Information processing methods including

2. In the text conversion step, the content of the image acquired in the image acquisition step is expressed in text and saved in association with the image. In the generation step, a report is generated based on the text. The information processing method according to claim 1.

3. In the selection step, images are selected from the completed questionnaires based on the areas of interest and the transcribed text. The information processing method according to claim 2.

4. In the generation step, the report for the predetermined period is generated. The information processing method according to any one of claims 1 to 3.

5. In an information processing device, Steps for obtaining completed questionnaires, An image acquisition step to acquire images over a predetermined period, A text conversion step, which converts the content of the image acquired in the image acquisition step into text, A selection step in which images acquired in the image acquisition step are selected based on the completed questionnaire and the transcribed text, Based on the completed questionnaire, a generation step is performed to generate a report based on the text that has been converted into text and the images selected in the selection step. A program for executing an information processing method that includes this method.

6. An information processing system equipped with an information processing device, Steps for obtaining completed questionnaires, An image acquisition step to acquire images over a predetermined period, A text conversion step, which converts the content of the image acquired in the image acquisition step into text, A selection step in which images acquired in the image acquisition step are selected based on the completed questionnaire and the transcribed text, Based on the completed questionnaire, a generation step is performed to generate a report based on the text that has been converted into text and the images selected in the selection step. An information processing system that performs this task.

Images