Information processing system, information processing method and program

Abstract

To provide an information processing system capable of evaluating a degree of cognitive function of a user.SOLUTION: In an information processing system 1, an information processing device 2, a medical person terminal 3, and a sound collecting device 4 are constituted so as to communicate through an electric telecommunication line. A processor of the information processing device includes: a presentation step of presenting an instruction for a presentation part to urge a predetermined action to a user; a reception step of a reception part receiving action data regarding the action from the user; an evaluation step of an evaluation part evaluating the degree of a cognitive function of a user in a predetermined index, based on a learned model which has learned beforehand a relation between feature quantity of the predetermined action and the degree of the cognitive function in a predetermined index; and a display control step of the presentation part displaying the score corresponding to the evaluated degree of cognitive function of the user on a screen in a manner of being displayed together with the predetermined index.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses a device for evaluating the degree of cognitive function from the user's speech. The device includes an instruction unit 160 that instructs the subject to quickly pronounce a phrase in which a predetermined syllable is repeated, an acquisition unit 110 that acquires auditory information indicating the speech of the subject who received the instruction, a calculation unit 120 that calculates a feature amount from the auditory information acquired by the acquisition unit 110, an evaluation unit 130 that evaluates the cognitive function of the subject from the feature amount calculated by the calculation unit 120, and an output unit 140 that outputs the evaluation result evaluated by the evaluation unit 130. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-083903 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is still room for improvement in the functions of the above-mentioned known techniques. In particular, since the evaluation results regarding cognitive function may contain sensitive information in some cases, a technique that allows a person who comes into contact with the evaluation results to calmly respond to the results is desired.

[0005] In view of the above circumstances, the present disclosure provides an information processing system etc. that enables a person who comes into contact with an evaluation result to calmly view the evaluation result. [Means for solving the problem]

[0006] According to one aspect of the present invention, an information processing system is provided. The information processing system includes a processor capable of executing a program to perform the following steps: In the presenting step, instructions for encouraging the user to perform a predetermined action are presented to the user. In the receiving step, action data relating to the action from the user is received. In the evaluating step, the degree of the user's cognitive function in a predetermined index is evaluated based on the action data and a previously set trained model. The trained model is a model that has previously trained the relationship between the feature amount of the predetermined action and the degree of cognitive function in the predetermined index. In the display control step, a score corresponding to the evaluated degree of the user's cognitive function is displayed on the screen in a form that is written alongside the predetermined index.

[0007] According to the present disclosure, it is possible to provide an information processing system etc. that allows a person who comes into contact with an evaluation result to calmly view the evaluation result. [Brief description of the drawings]

[0008] [Figure 1] 1 is a configuration diagram illustrating an information processing system 1. [Diagram 2] 2 is a block diagram showing a hardware configuration of an information processing device 2. FIG. [Diagram 3] FIG. 2 is a block diagram showing the hardware configuration of a user terminal. [Figure 4] 2 is a functional block diagram showing functions of an information processing device 2. FIG. [Diagram 5] 2 is an activity diagram showing an example of the flow of information processing executed in the information processing system 1. FIG. [Figure 6] 1 shows an example of a visual impairment check screen 6 displayed on the display unit 34 of the medical professional terminal 3. [Figure 7] 1 shows an example of a voice input instruction screen 7 displayed on the display unit 34 of the medical professional terminal 3. [Figure 8] 1 shows an example of a recording screen 8 displayed on the display unit 34 of the medical professional terminal 3. [Figure 9]FIG. 1 is a schematic diagram of a neural network NN, which is an example of a trained model. [Figure 10] 1 shows an example of an evaluation result display screen 9 displayed on the display unit 34 of the medical professional terminal 3. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings. Various characteristic features shown in the following embodiments can be combined with each other.

[0010] Incidentally, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable recording medium, or may be provided so as to be downloadable from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0011] In this embodiment, the term "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, this embodiment handles various information, which may be represented by, for example, physical values ​​of signal values ​​representing voltage and current, high and low signal values ​​as a binary bit set consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculation may be performed on a circuit in the broad sense.

[0012] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. In other words, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0013] 1. Hardware Configuration This section describes the hardware configuration.

[0014] <Information Processing System 1> FIG. 1 is a configuration diagram showing an information processing system 1. The information processing system 1 includes an information processing device 2, a medical practitioner terminal 3, and a sound collection device 4. The information processing device 2, the medical practitioner terminal 3, and the sound collection device 4 are configured to be able to communicate with each other via an electric communication line. In one embodiment, the information processing system 1 is made up of one or more devices or components. For example, if the information processing system 1 is made up of only the information processing device 2, the information processing system 1 can be the information processing device 2. These components will be described below.

[0015] <Information processing device 2> 2 is a block diagram showing a hardware configuration of the information processing device 2. The information processing device 2 includes a communication bus 20, a communication unit 21, a storage unit 22, and a processor 23. The communication unit 21, the storage unit 22, and the processor 23 are electrically connected via the communication bus 20 inside the information processing device 2.

[0016] <Communications Division 21> The communication unit 21 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), wired LAN network communication, etc., but may also include wireless LAN network communication, mobile communication such as 3G / LTE / 5G, BLUETOOTH (registered trademark) communication, etc. as necessary. In other words, it is more preferable to implement it as a collection of multiple communication means. In other words, the information processing device 2 may communicate various information from the outside via the communication unit 21 and the network.

[0017] <Storage section 22> The storage unit 22 stores various information defined by the above description. This can be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs and the like related to the information processing device 2 executed by the processor 23, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to the program calculations. The storage unit 22 stores various programs, variables, etc. related to the information processing device 2 executed by the processor 23.

[0018] <Processor 23> The processor 23 processes and controls the overall operations related to the information processing device 2. The processor 23 is, for example, a central processing unit (CPU) not shown. The processor 23 realizes various functions related to the information processing device 2 by reading out a predetermined program stored in the storage unit 22. That is, information processing by software stored in the storage unit 22 can be specifically realized by the processor 23, which is an example of hardware, and executed as each functional unit included in the processor 23. These will be described in more detail in the next section. The processor 23 is not limited to being single, and may be implemented with multiple processors 23 for each function. Also, a combination of these may be used.

[0019] <User device> The user terminal is a terminal possessed by a user. In this embodiment, a case will be described in which the user terminal is a terminal 3 (hereinafter referred to as medical professional terminal 3) operated by a user who is a medical professional such as a doctor (hereinafter referred to as a medical professional). The user terminal may further include a terminal (hereinafter referred to as a subject terminal) operated by a user who is a patient whose cognitive function is to be evaluated (hereinafter referred to as a subject terminal). Details will be described later. The user terminal may be in any form, such as a smartphone, tablet terminal, computer, or any other device that can access the information processing device 2 via an electric communication line.

[0020] 3 is a block diagram showing the hardware configuration of a user terminal. The following description will be given taking the medical professional terminal 3 as an example. The medical professional terminal 3 includes a communication bus 30, a communication unit 31, a storage unit 32, a processor 33, a display unit 34, and an input unit 35. The communication unit 31, the storage unit 32, the processor 33, the display unit 34, and the input unit 35 are electrically connected via the communication bus 30 inside the medical professional terminal 3. The description of the communication unit 31, the storage unit 32, and the processor 33 will be omitted as they are the same as the descriptions of the respective units in the information processing device 2.

[0021] <Display section 34> The display unit 34 displays a screen of a graphical user interface (GUI) that can be operated by the user. The display unit 34 may be included in the housing of the medical professional terminal 3, or may be attached externally. Specifically, the display unit 34 may be implemented as a display device such as a CRT display, a liquid crystal display, an organic EL display, or a plasma display. These display devices are preferably implemented by selectively using them according to the type of the medical professional terminal 3.

[0022] <Input section 35> The input unit 35 accepts an operation input made by a medical professional or a subject. The operation input is transferred as a command signal to the processor 33 via the communication bus 30. The processor 33 can execute a predetermined control or calculation based on the transferred command signal as necessary. The input unit 35 may be included in the housing of the medical professional terminal 3, or may be externally attached. For example, the input unit 35 may be implemented as a touch panel integrated with the display unit 34. When the input unit 35 is implemented as a touch panel, the medical professional or a user can input a tap operation, a swipe operation, or the like to the input unit 35. As the input unit 35, a switch button, a mouse, a QWERTY keyboard, or the like can be adopted instead of a touch panel.

[0023] <Sound collection device 4> The sound collector 4 is a so-called microphone that is configured to convert external sounds into signals. The sound collector 4 may be provided by directly connecting the sound collector 4 to the information processing device 2, but is provided in or connected to the medical professional terminal 3.

[0024] The sound collection device 4 is configured to generate voice data by collecting the user's speech. The voice data is temporarily stored in a memory in the user terminal, and does not have to be stored in a non-volatile manner in the storage unit 32. The voice data generated by the sound collection device 4 is configured to be transferable to the information processing device 2 via a network.

[0025] The sound collection device 4 collects, but is not limited to, at least sounds in the human audible range, sounds with frequencies between 20 Hz and 20,000 Hz, and converts them into electrical signals. The sounds may be recorded in monaural or stereo. The sampling rate for digitally processing the sound data may be, for example, 48000 Hz, 44100 Hz, 32000 Hz, 22050 Hz, 16000 Hz, 11025 Hz, 11000 Hz, 8000 Hz, etc. The sampling rate may be within any of the ranges of values ​​exemplified here. By increasing the sampling rate, the temporal timing of the sound can be discretized more precisely, and the accuracy of voice recognition can be improved.

[0026] The data collected by the sound collection device 4 may be appropriately compressed by the processor 33 of the medical professional terminal 3, and the compression format may be any of MP3, AAC, WMA, Vorbis, AC3, MP2, FLAC, TAK, etc. By compression, it is possible to reduce communication traffic caused by data transfer from the user terminal to the information processing device 2.

[0027] 2. Functional configuration In this section, the functional configuration will be described. Fig. 4 is a functional block diagram showing the functions of the information processing device 2. As described above, information processing by software stored in the storage unit 22 can be specifically realized by hardware (specifically, the processor 23) and executed as each functional unit included in the processor 23. In other words, the information processing system 1 includes each functional unit in a program.

[0028] Specifically, the processor 23 includes a receiving unit 231, an acquiring unit 232, a specifying unit 233, an evaluating unit 234, and a presenting unit 235 as each functional unit.

[0029] The receiving unit 231 is configured to receive various information as a receiving step. Specifically, the receiving unit 231 is configured to receive information via the communication unit 21 or the storage unit 22 and to be able to read the information into the working memory. Preferably, the receiving unit 231 receives motion data relating to motion from the subject. In a more typical example, the receiving unit 231 receives input of voice data relating to a speech voice from the subject via the sound collection device 4.

[0030] The acquiring unit 232 is configured to acquire various pieces of information as an acquiring step. Specifically, the acquiring unit 232 acquires user information, which is information related to the subject.

[0031] The specifying unit 233 is configured to specify various pieces of information as a specifying step. Specifically, the specifying unit 233 specifies a presentation mode related to an instruction for prompting the subject to perform a predetermined action, based on the user information acquired by the acquiring unit 232. Details will be described later.

[0032] The evaluation unit 234 is configured to evaluate various characteristics as an evaluation step. Preferably, the evaluation unit 234 evaluates the degree of cognitive function, etc., based on the motion data of the subject accepted by the acceptance unit 231. Details will be described later.

[0033] As a presentation step, the presentation unit 235 presents various pieces of information stored in the storage unit 32 in a form that can be understood by the subject or the medical practitioner. For example, the presentation unit 235 may display a screen or the like including various pieces of information in a form that can be viewed on a terminal such as the medical practitioner terminal 3. As a result, various pieces of information are presented to various users who operate terminals such as the medical practitioner terminal 3. Since the presentation unit 235 can display various pieces of information on the screen in this manner, the presentation unit 235 may be referred to as being configured to be able to execute a display control step. Specifically, the presentation unit 235 controls visual information such as a screen, an image, an icon, and a message to be displayed on the display of the above-mentioned terminal. The presentation unit 235 may generate only rendering information for displaying visual information on the above-mentioned terminal. The presentation unit 235 may also present various pieces of information as auditory information to the subject or the medical practitioner. Furthermore, the presentation unit 235 may present various pieces of information to the subject as unevenness information that presents an uneven tactile sensation to a finger. Details will be described later.

[0034] 3. Operation of Information Processing System 1 In this section, the flow of the information processing method executed by the information processing system 1 will be described with reference to the drawings. As shown below, the information processing method includes the following steps. The order of the processes can be changed as appropriate, a plurality of processes may be executed simultaneously, or some processes may be omitted. In the following, a process of presenting an instruction to the subject (user) by auditory information to prompt the subject (user) to make a predetermined speech sound is shown as a presentation step, but this instruction may prompt an action other than the speech sound. The instruction may also be presented by information other than auditory information. In addition, the object received in the reception step is sometimes called voice data, but the object received in the reception step may be action data related to the action of the subject (user), and is not necessarily limited to voice data.

[0035] 3.1. Overview of information processing FIG. 5 is an activity diagram showing an example of the flow of information processing executed in the information processing system 1. As an example, a case will be described in which the degree of cognitive function of a subject is examined in a medical institution by analyzing the subject's speech voice data. In this embodiment, a case will be described in which both instructions in the form of auditory information IF2 and instructions in the form of visual information IF1 are presented to the subject as instructions to encourage the subject to speak. Moreover, speaking in this embodiment refers to speaking a fixed word or phrase.

[0036] First, as an acquisition step, the acquisition unit 232 accepts and acquires the input of the subject's personal ID from a medical professional (A101). Specifically, for example, the acquisition unit 232 accepts and acquires information indicating the subject's personal ID from the medical professional terminal 3 via the communication unit 21. Furthermore, the identification unit 233 may identify the personal information of the subject from the information stored in the storage unit 22 by authenticating the user based on the personal ID accepted in the activity A101.

[0037] Authentication of the user does not necessarily have to be performed. The acquisition unit 232 may accept and acquire user information of the subject instead of the subject's personal ID. The acquisition unit 232 may accept and acquire personal information of the subject, such as the subject's name, age, or sex, as the user information. The user information may be input from the subject himself / herself.

[0038] Next, as a specification step, the specification unit 233 specifies a presentation mode of instructions for encouraging the subject to speak (A102). Preferably, the specification unit 233 specifies the presentation mode based on the subject's personal information (name, age, sex) associated with the subject's personal ID acquired in activity A101. Specifically, for example, when age information is acquired as the subject's personal information, the specification unit 233 specifies the presentation mode so that the auditory information IF2 is presented at a slower speed than normal for older subjects. Also, for example, when gender information "female" is acquired as the subject's personal information, the specification unit 233 specifies the presentation mode so that audio data giving instructions in a female voice is presented. In particular, it is considered that instructions given by a voice of the same sex can give the subject a sense of security.

[0039] Meanwhile, a process of identifying the degree of the disability of the subject is executed. As an example of a presentation step, the presentation unit 235 displays a screen for determining the degree of disability on the display unit 34 of the medical practitioner terminal 3 (A103). Specifically, the presentation unit 235 displays a question for identifying the degree of visual impairment or hearing impairment on the display unit 34 of the medical practitioner terminal 3 so that the question can be presented to the subject. Details will be described later.

[0040] Next, the receiving unit 231 receives an input of an answer from the subject to the question displayed in activity A103 (A104). Furthermore, the acquiring unit 232 acquires information indicating the degree of the subject's disability based on the received answer. In other words, the acquiring unit 232 acquires specific user information that is information about a specific user. Here, the specific user is a person (subject) who corresponds to a subject among users involved in the information processing system 1. Furthermore, the specific user information includes information that the specific user has a visual impairment or a hearing impairment. Details will be described later.

[0041] Next, the identification unit 233 identifies the degree of the disability of the subject (A105) as an identification step, as will be described in detail later.

[0042] Here, a case where the subject's disability is identified in activity A105 will be described. As an identification step, the identification unit 233 identifies a change in the mode of the screen to be displayed on the display unit 34 of the medical professional terminal 3 based on the degree of the subject's disability identified in activity A105 (A106). Here, the screen to be displayed on the display unit 34 is a screen that displays instructions to encourage speaking. For example, when the subject is identified as having color vision deficiency, the identification unit 233 may change the color of the display mode. Also, for example, when the subject is identified as having age-related deterioration of eyesight, the identification unit 233 may change the brightness and size of characters to be displayed on the screen. Details will be described later.

[0043] Next, the presentation unit 235 presents an instruction to the user, who is the subject, to prompt the user to make a predetermined speech sound by the auditory information IF2 (A107). That is, the instruction information to prompt the subject to make an action (speech) may include the auditory information IF2. The auditory information IF2 is audio information presented to the subject. In this embodiment, the presentation unit 235 may present the instruction to the user by the visual information IF1. In this case, the visual information IF1 is composed of at least one of a character, a figure, and a photo. Hereinafter, the explanation will be continued by giving an example in which the instruction to prompt the subject to make an action includes both the auditory information IF2 and the visual information IF1. Specifically, the presentation unit 235 presents an instruction to prompt the user to make a speech sound based on the presentation mode specified in the activity A102 and the change screen specified in the activity A106. In other words, the presentation unit 235 presents the instruction information including an instruction to prompt the specific user to make a predetermined speech to the specific user, which is variable according to the specific user information including information regarding the subject's disability. Details will be provided below.

[0044] On the other hand, if the subject's disability is not identified in activity A105, the process proceeds to activity A107. In this case, the presenting unit 235 presents the user with an instruction to prompt an action (utterance) based on the presentation mode identified in activity A102 (A107). In other words, the acquiring unit 232 acquires user information, which is information about the user, in activity A101. Furthermore, the presenting unit 235 controls the content of the instruction based on the user information and presents it.

[0045] Next, the reception unit 231 receives voice data related to a speech voice from the user who is the subject (A108). The voice data is data generated by the sound collection device 4 based on the speech voice of the subject. Note that, although the reception unit 231 has been described above as receiving voice data related to a speech voice as the motion data, the reception unit 231 may receive various information according to the contents of the motion data. For example, when receiving motions such as a user's gestures and eye movements as the motion data, the motion data related to the motion of the user may be received via a predetermined imaging device.

[0046] At the same time as receiving the voice data of activity A108, the presenting unit 235 updates the screen on which the instructions are displayed (A109). Here, the screen is a screen on which instructions are displayed to enable the subject to understand the timing at which he or she should speak. In other words, the reception of the motion data in the reception unit 231 starts before the presentation of the instructions in the presentation unit 235 is completed.

[0047] Next, the evaluation unit 234 evaluates the degree of the user's cognitive function in a predetermined index based on the action data (voice data) and a pre-established trained model (step A110). Here, the trained model is a model that has previously trained the relationship between the feature amount of a predetermined action and the degree of the cognitive function in a predetermined index. Details will be described later.

[0048] The presentation unit 235 then presents the evaluation results to the subject and the medical practitioner. In this embodiment, a score corresponding to the degree of the evaluated cognitive function of the user is displayed on the screen in a manner in which it is written alongside a predetermined index. Details will be described later.

[0049] In summary, the information processing system 1 according to this embodiment includes a processor 23 capable of executing a program so that each of the following units is operated. The presenting unit 235 presents to the user an instruction to prompt the user to perform a predetermined action. The accepting unit 231 accepts action data related to the action from the user. The evaluating unit 234 evaluates the degree of the user's cognitive function in a predetermined index based on the action data and a pre-set trained model. The trained model is a model that has previously trained the relationship between the feature amount of the predetermined action and the degree of the cognitive function in the predetermined index. The presenting unit 235 displays a score corresponding to the evaluated degree of the user's cognitive function on the screen in a manner in which the score is displayed alongside the predetermined index.

[0050] For example, Patent Document 1 shows an image showing an evaluation result such as "You are showing signs of dementia" in FIG. 5. In the conventional technology, it is common to show only the presence or absence of cognitive decline like this, and a user who comes across such an image (screen) may feel uneasy because he or she does not know on what basis the evaluation is made. In contrast, according to the above-mentioned embodiment, a score showing the degree is shown together with a predetermined index, so that a person who comes across the evaluation result can calmly view the evaluation result.

[0051] 3.2 Details of data processing The above-mentioned information processing will be described in detail with reference to FIGS.

[0052] (Visual impairment check screen 6) 6 shows an example of a visual impairment check screen 6 displayed on the display unit 34 of the medical professional terminal 3. In this embodiment, an example of determining the degree of color vision deficiency as a visual impairment will be described. The visual impairment check screen 6 has an area 61 in which input instructions for the subject are displayed, an area 62 in which selection input is received from the subject, and a button 63.

[0053] As shown in FIG. 6, area 61 displays an input instruction to the subject, "Please select a box that is a different color." Area 62 is an area that accepts a selective input of an answer to the input instruction from the subject. Area 62 has multiple checkboxes including checkbox 621 and checkbox 622. As shown in FIG. 6, among the multiple checkboxes, checkbox 621 is displayed in a color different from the other checkboxes. Meanwhile, checkbox 622 is displayed in the same color as the other checkboxes except for checkbox 621.

[0054] 6, the subject can answer the color vision deficiency check by selecting any check box in area 62 as a response to the instruction displayed in area 61. The acquisition unit 232 can acquire information indicating the degree of the subject's color vision deficiency by receiving a check box selection input from the subject. In other words, the acquisition unit 232 acquires user information by receiving an input operation from the user who is the subject.

[0055] As an example, when a selection input of the check box 621 is received from the subject, the identifying unit 233 may determine that the subject is unlikely to have a color vision deficiency. On the other hand, when a selection input of a check box other than the check box 621 is received from the subject, the identifying unit 233 may determine that the subject is likely to have a color vision deficiency.

[0056] Furthermore, the identification unit 233 may determine the degree of color vision deficiency of the test subject based on the results of performing such a check multiple times. The button 63 is a button for displaying check content different from the check content displayed on the display unit 34. For example, by receiving an input to the button 63 from the test subject, the presentation unit 235 transitions the content displayed on the display unit 34 from the check screen 6 in FIG. 6 to another check screen.

[0057] In the present embodiment, an example has been described in which the degree of color vision impairment is determined as a visual impairment, but this is not limiting. For example, the degree of visual impairment may be determined as a visual impairment. Also, for example, the degree of accommodation disorder may be determined as a visual impairment. Furthermore, the degree of hearing impairment may be determined.

[0058] In this embodiment, a case will be described in which the subject is identified as having color vision deficiency based on the result of a selection input received from the subject. The identification unit 233 determines that the color scheme displayed on the display unit 34 of the medical professional terminal 3 should be changed to a color scheme that can be recognized by the subject. Specifically, the identification unit 233 changes the color scheme displayed on the instruction screen 7 in Fig. 7, which will be described later. This allows the subject to recognize the instruction to encourage speech without the need for assistance from a third party.

[0059] (Instruction screen 7) Fig. 7 shows an example of a voice input instruction screen 7 displayed on the display unit 34 of the medical professional terminal 3. The voice input instruction screen 7 is an example of a screen displayed in activity A107 of Fig. 5. The voice input instruction screen 7 has a character 71, an area 72 in which instructions for the subject are displayed, and a button 73.

[0060] As shown in FIG. 7, the character 71 has an appearance display that imitates, for example, the appearance of a female medical professional. The character 71 is used as a character that instructs the subject on the specific content of the voice input. The appearance of the character 71 may be changed according to the presentation mode specified in the activity A102. For example, if the subject is a woman, the character 71 having a female appearance display may be selected. Furthermore, the appearance of the character 71 is not limited to a person, and may be an animal, a fictional character, or the like.

[0061] Also, as shown in FIG. 7, the area 72 displays a speech instruction to the subject, "Please say 'pataka' five times in a row." Here, the instruction displayed in the area 72 is also simultaneously presented to the subject by the auditory information IF2. In this manner, the subject can record the voice after recognizing the intonation of the speech. Also, the character 71 may be displayed so as to change in conjunction with the auditory information IF2 presented to the subject. In other words, the presentation unit 235 further presents the character 71 to the user, who is the subject, in a visually recognizable manner. The character 71 is configured to change according to the presentation state of the auditory information IF2, which changes along a time series.

[0062] Furthermore, the visual information IF1 that changes in conjunction with the auditory information IF2 is not limited to the character 71. As an example, the text information IF11 displayed in the area 72 may be displayed in a manner that changes in conjunction with the auditory information IF2. That is, the visual information IF1 is configured to change according to the presentation state of the auditory information IF2 that changes over time. According to such an embodiment, the subject can further grasp the contents of the test. Therefore, the psychological anxiety of the subject can be reduced when testing the cognitive function of the subject.

[0063] 7, the button 73 is a button for starting the subject's voice input. For example, the presentation unit 235 receives an input to the button 73 from the subject or the medical professional, and transitions the content displayed on the display unit 34 from the instruction screen 7 in FIG. 7 to the recording screen 8 in FIG. 8.

[0064] (Recording screen 8) Fig. 8 shows an example of a recording screen 8 displayed on the display unit 34 of the medical professional terminal 3. The recording screen 8 is an example of a screen displayed in activity A108 of Fig. 5. The recording screen 8 includes an area 81, an area 82, a record button 83, a stop button 84, an area 85, an area 86, and an area 87.

[0065] Area 81 is an area where instructions regarding speech are displayed to the subject. As shown in FIG. 8, area 81 displays the instruction "After tapping the record button, please read the following out loud." Area 82 is an area where the specific speech content to be spoken by the subject is displayed. As shown in FIG. 8, area 82 displays the speech content "Pataka (5 times)."

[0066] The record button 83 is a button for starting recording. The acquisition unit 232 acquires voice data of the subject's speech by receiving an input to the record button 83 from the subject or medical professional. The stop button 84 is a button for stopping recording. The acquisition unit 232 stops acquiring the voice data by receiving an input to the stop button 84 from the subject or medical professional.

[0067] The area 85 is an area where it is visually displayed that the subject's voice is being recorded. The presentation unit 235 displays "REC" in the area 85 by receiving an input to the record button 83 from the subject or medical professional. The presentation unit 235 also stops displaying "REC" in the area 85 by receiving an input to the stop button 84 from the subject or medical professional.

[0068] Area 86 is an area where the elapsed time from the start of recording is visibly displayed. As shown in Fig. 8, area 86 may display the elapsed time by object IF14. Area 86 may also display the elapsed time by text information IF15 "00:02.00". Area 86 may also visibly display the remaining time.

[0069] The area 87 is an area where information indicating the subject's speech timing is displayed. In other words, the presentation unit 235 further presents information including an instruction indicating the speech timing to the user who is the subject in a graspable manner. In further other words, the presentation unit 235 further presents guide information including information guiding the speech timing to the specific user who is the subject in a graspable manner. As shown in FIG. 8, the area 87 displays information "patakapataka" indicating the content to be spoken by the subject. Furthermore, the character information displayed in the area 87 includes blackened character information IF12 and whitened character information IF13. As an example, the character form indicating the speech content is changed from the whitened character information IF13 to the blackened character information IF12 in response to the arrival of the speech timing. In the example of FIG. 8, the word to be spoken by the subject next is "ka", and the whitened character information IF13 "ka" changes to a blackened character form in response to the arrival of the speech timing.

[0070] Here, the manner in which the speech timing can be grasped is not limited to this. For example, the manner of characters indicating the speech content may change so as to be emphasized when the speech timing arrives. Furthermore, the manner of characters may change so as to return to the original manner when the speech timing ends.

[0071] Furthermore, the instruction indicating the speaking timing is not limited to a change in the appearance of the characters. For example, an object having the appearance of a metronome may be displayed in the area 87. The object displayed in the area 87 may be presented to the user in a manner in which a pendulum needle moves left and right in accordance with the speaking timing of the subject. Also, for example, an object having the appearance of clapping hands may be displayed in the area 87. The object displayed in the area 87 may be presented to the user in a manner in which hands are clapping in accordance with the speaking timing of the subject. According to such an embodiment, the subject can speak while grasping the speaking timing and rhythm. Therefore, it is possible to obtain voice data that reflects the subject's original speaking state.

[0072] Here, a method for evaluating the degree of a user's cognitive function from motion data such as voice data will be described. FIG. 9 is a schematic diagram of a neural network NN, which is an example of a trained model. An input signal defined by various parameters is input to the first layer L1. The input signal here is input voice data. The input signal is output from the computation nodes N_11 to N_13 of the first layer L1 to the computation nodes N_21 to N_25 of the second layer L2, respectively. At this time, the value output from the computation nodes N_11 to N_13 is multiplied by the weight w set between each computation node N, and the resulting value is input to the computation nodes N_21 to N_25.

[0073] The calculation nodes N_21 to N_25 add up the input values ​​from the calculation nodes N_11 to N_13, and input this value (or a value obtained by adding a predetermined bias value to this value) to a predetermined activation function.

[0074] Then, the output value of the activation function is propagated to the next node, the calculation node N_31. At this time, a value obtained by multiplying the weight w set between the calculation nodes N_21 to N_25 and the calculation node N_31 by the output value is input to the calculation node N_31. The calculation node N_31 adds up the input values ​​and outputs the sum as an output signal. At this time, the calculation node N_31 may add up the input values, input the sum to a bias value, and input the result to the activation function, and output the output value as an output signal. This outputs the degree of cognitive function.

[0075] It should be noted that what is shown in FIG. 9 is for illustrative purposes only and is not limiting.

[0076] In this embodiment, the information indicating the degree of cognitive function includes an evaluation value (score) in a predetermined index. The evaluation unit 234 may perform calculation processing (conversion) based on the received motion data, etc., so that the degree of cognitive function of the user matches the score in the predetermined index. In addition, the evaluation value (score) in this embodiment does not necessarily indicate a numerical value of one point, but may specify a numerical range (numerical range), for example, "10 to 12". In addition, the evaluation value (score) in this embodiment may specify an upper limit value or a lower limit value, such as "20 or more" or "less than 15". On the other hand, examples of the "predetermined index" include various indexes (scales) that can evaluate the degree of cognitive function. Examples of such measures include, but are not limited to, the Montreal Cognitive Assessment (MoCA; the Japanese version is sometimes referred to as MoCA-J), the Behavior Rating Inventory of Executive Function (BRIEF), the Mini-Menta State Examination (MMSE), the Hasegawa Brief Dementia Scale, and the Wechsler Memory Scale. In one embodiment, a display may be further provided that classifies the patient into "healthy (normal)", "cognitive decline", "risk of cognitive decline", or "risk of dementia" according to the stage based on the above-mentioned evaluation value (score).

[0077] (Evaluation result display screen 9) 10 shows an example of the evaluation result display screen 9 displayed on the display unit 34 of the medical professional terminal 3. The evaluation result display screen 9 includes an area 91, an area 92, an area 93, and a remeasurement button 94.

[0078] Preferably, the area 91 is an area where information on the cognitive function to be evaluated is displayed. As shown in FIG. 10, the area 91 displays "audio evaluation result of cognitive function" as the evaluation target. Also preferably, the area 92 is an area where the measurement result 921 is displayed. In the example of FIG. 10, the area 92 displays information "28 (MoCA-J standard; healthy)" as the measurement result 921. That is, in this embodiment, a score corresponding to the degree of the evaluated cognitive function of the user is displayed on the screen (evaluation result display screen 9) in a manner in which it is displayed together with a predetermined index (MoCA-J). In this way, a user who is exposed to a quantitative evaluation can easily visually recognize the evaluation result calmly.

[0079] In addition, when the degree of cognitive function of the user has been evaluated in the past, the presentation unit 235 may also display the previously evaluated degree of cognitive function on the screen. That is, in FIG. 10, in the case where the subject has previously received the same evaluation, a comparison result 931 showing a comparison between the current evaluation result ("score: 28") and the previous evaluation result ("score: 29") is displayed as the area 93. For example, the comparison result 931 is evaluated based on the amount of change of the current evaluation value 934 with respect to the previous evaluation value 935 described later. In other words, the acquisition unit 232 may further acquire evaluation information including the past evaluation result of the specific user who is the subject. The evaluation unit 234 may further evaluate the amount of change of the degree of cognitive function of the specific user based on the evaluation information. In the example of FIG. 10, the area 93 displays information "no change" as the comparison result 931 showing the comparison with the previous time.

[0080] Preferably, in the area 93, an item 932 “this time”, an item 933 “last time”, a current evaluation value 934, and a previous evaluation value 935 are further displayed so as to be visible to the subject. In the example of Fig. 10, for example, the evaluation unit 234 evaluates the speech of the subject in both the previous measurement and the current measurement and judges it to be “healthy (normal)”.

[0081] This allows the subject to understand the degree of their own cognitive function, and further allows the progress of the user's cognitive function to be quantitatively evaluated.

[0082] [others] The information processing system 1 may have the following configuration.

[0083] In the above embodiment, the screen on which the speaking instruction is displayed is updated at the same time as the voice data is received by the activity A108, but this is not limited thereto. The reception unit 231 may start receiving the voice data after the presentation of the speaking instruction is completed. In other words, the reception unit 231 may start receiving the voice data after the presentation of the instruction is completed by the presentation unit 235. According to this aspect, if the subject knows the timing of speaking, the subject can concentrate on speaking. Therefore, it is possible to obtain voice data that reflects the subject's original speaking state.

[0084] In the above embodiment, the presenting unit 235 presents the instruction to encourage the subject to speak using the visual information IF1 and the audio information IF2, but this is not limited to the above. For example, if the subject has weak eyesight, the presenting unit 235 may present the instruction to encourage the subject to speak using Braille information. In other words, the instruction information, which is an instruction to encourage the subject to speak, includes uneven information for presenting an uneven tactile sensation to the finger of the specific user who is the subject. According to this aspect, even if the subject has a visual impairment such as weak eyesight, psychological anxiety about the test can be reduced. Therefore, it is possible to obtain voice data that reflects the subject's original speech state. In addition, when presenting Braille information to the subject, it is preferable to use a display compatible with Braille as the display unit 34 of the medical professional terminal 3. The presenting unit 235 converts the character information, which is the instruction information, into Braille information and presents it to the subject.

[0085] In the above embodiment, the evaluation unit 234 evaluates the degree of cognitive function based on the voice data in which the subject speaks about fixed words or phrases, but this is not limited to the above. For example, the evaluation unit 234 may evaluate the degree of cognitive function based on the voice data in which the subject speaks in a conversational format. Specifically, in activity A108 in FIG. 5, the reception unit 231 receives voice data of the subject who speaks about an answer to a question. For example, the presentation unit 235 may present the subject with a question such as "What did you do yesterday?" or "What was the most memorable event in the past week?" as a question for the subject. Furthermore, the presentation unit 235 may further present the subject with a question related to the content of the answer received from the subject. For example, the evaluation unit 234 may evaluate the degree of cognitive function of the subject based on the amount of speech acquired from the received voice data. In such a case, a trained model that has been trained in advance on conversational exchanges may be used.

[0086] When evaluating the degree of cognitive function in a conversational format, the presentation unit 235 may present the subject with the timing to speak a response to a question. In other words, the presentation unit 235 may further present guide information including information guiding the timing of speaking to a specific user who is the subject in a manner that the user can grasp. According to such a manner, the subject can grasp the timing to respond to a question, and the tension and psychological anxiety regarding the test can be reduced.

[0087] In the above embodiment, the identification unit 233 identifies the degree of the disability of the subject by a simple check, but this is not limited to the above. For example, the reception unit 231 may receive the degree of disability from the subject through self-reporting by the subject. The identification unit 233 may identify the degree of disability according to the information received from the subject.

[0088] In the above embodiment, the subject is examined at a medical institution, but this is not the only possible case. For example, the subject may be examined using his / her own subject terminal. Specifically, the information processing system 1 may further include a subject terminal as a user terminal in addition to the medical practitioner terminal 3. The subject terminal may include, for example, a hardware configuration substantially similar to that of the information processing device 2 and the medical practitioner terminal 3. The communication unit of the subject terminal may transmit the subject's voice data generated by the sound collection device 4 to the information processing device 2 or the medical practitioner terminal 3. This allows the subject to easily undergo an examination of the degree of cognitive function at home or the like, and to receive a diagnosis from a medical practitioner remotely.

[0089] The information processing device 2 may be an on-premise type or a cloud type. As the information processing device 2 in the cloud type, the above functions and processes may be provided in the form of, for example, SaaS (Software as a Service) or cloud computing.

[0090] In the above embodiment, the information processing device 2 performs various storage and control, but multiple external devices may be used instead of the information processing device 2. That is, various information and programs may be distributed and stored in multiple external devices using block chain technology or the like.

[0091] Another aspect of the embodiment may be a program that causes a computer to execute each step.

[0092] It may be provided in any of the following ways:

[0093] (1) An information processing system comprising a processor capable of executing a program to perform each of the following steps: in a presentation step, instructions for encouraging the user to perform a predetermined action are presented to the user; in a reception step, action data relating to the action from the user is received; in an evaluation step, a degree of the user's cognitive function in a predetermined index is evaluated based on the action data and a predetermined trained model, wherein the trained model is a model that has been trained in advance regarding the relationship between features of the predetermined action and the degree of cognitive function in the predetermined index; and in a display control step, a score corresponding to the evaluated degree of the user's cognitive function is displayed on a screen in a manner that is shown alongside the predetermined index.

[0094] (2) In the information processing system described in (1) above, in the presenting step, the instruction is presented to the user by auditory information.

[0095] (3) In the information processing system described in (1) or (2) above, in the presentation step, the instruction is presented to the user by visual information, and the visual information is composed of at least one of text, figures, and photographs.

[0096] (4) In the information processing system according to any one of (1) to (3) above, the motion data relating to the motion is voice data relating to a speech sound.

[0097] (5) In the information processing system described in any one of (1) to (4) above, in the display control step, if the user has had the degree of cognitive function evaluated in the past, the previously evaluated degree of cognitive function is also displayed on the screen.

[0098] (6) In the information processing system described in any one of (1) to (5) above, the system further includes, in the acquisition step, acquiring user information which is information about the user, and in the presentation step, controlling the content of the instruction based on the user information and presenting the instruction.

[0099] (7) An information processing method, comprising the steps of the information processing system according to any one of (1) to (6) above.

[0100] (8) A program that causes a computer to execute each of the steps described in any one of (1) to (6) above. Of course, this is not the case.

[0101] Finally, although various embodiments according to the present disclosure have been described, these are presented as examples and are not intended to limit the scope of the invention. The novel embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. The embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0102] 1: Information processing system 2: Information processing equipment 3: Medical device 4: Sound collection device 6: Check screen 7: Instruction screen 8: Recording screen 9: Evaluation result display screen 20: Communication bus 21: Communications Department 22: Storage section 23: Processor 30: Communication bus 31: Communications Department 32: Storage section 33: Processor 34:Display section 35: Input section 61 :Area 62 :Area 63: Button 71: Character 72 :Area 73: Button 81 :Area 82 :Area 83: Record button 84: Stop button 85 :Area 86 :Area 87 :Area 91: area 92: area 93 :Area 94: Remeasurement button 110: Acquisition section 120: Calculation section 130: Evaluation section 140: Output section 160: Instruction section 231: Reception 232: Acquisition Department 233: Specific part 234: Evaluation section 235: Presentation section 621: Checkbox 622: Checkbox 921: Measurement result 931 :Comparison result 932 :Item 933 :Item 934: Rating value 935: Rating

Claims

1. An information processing system, A processor capable of executing a program to perform the following steps: In the presenting step, an instruction for prompting the user to perform a predetermined action is presented to the user; In the receiving step, motion data relating to the motion is received from the user; In the evaluation step, a degree of the cognitive function of the user in a predetermined index is evaluated based on the motion data and a previously set trained model, and the trained model is a model that has previously trained a relationship between a feature amount of a predetermined motion and a degree of the cognitive function in the predetermined index; In the display control step, the system displays a score corresponding to the degree of the evaluated cognitive function of the user on a screen in a manner that is displayed alongside the specified indicator.

2. 2. The information processing system according to claim 1, In the presenting step, the system presents the instructions to the user by auditory information.

3. 2. The information processing system according to claim 1, The presenting step presents the instruction to the user by visual information; A system in which the visual information is composed of at least one of text, graphics, and photographs.

4. 2. The information processing system according to claim 1, A system, wherein the motion data relating to the motion is voice data relating to a speech sound.

5. 2. The information processing system according to claim 1, In the display control step, if the user has had the degree of cognitive function evaluated in the past, the system also displays the degree of cognitive function previously evaluated on the screen.

6. 2. The information processing system according to claim 1, Furthermore, in the acquisition step, user information that is information about the user is acquired, In the presenting step, the system controls the content of the instruction based on the user information and presents the instruction.

7. 1. An information processing method, comprising: A method comprising the steps in an information processing system according to any one of claims 1 to 6.

8. A program, A program causing a computer to execute each of the steps according to any one of claims 1 to 6.

Images