Information processing system, information processing method, and program

Abstract

According to one embodiment of the present invention, provided is an information processing system. This information processing system is provided with a processor that can execute a program so that the following steps are performed. In an acquisition step, specific user information is acquired which is information pertaining to a specific user. The specific user is a person corresponding to a subject among users engaged in the information processing system. The specific user information includes information indicating that the specific user has a visual impairment or an auditory impairment. In a presentation step, instruction information including an instruction for urging the specific user to perform prescribed speech is variably presented to the specific user in accordance with the specific user information. In a reception step, speech data pertaining to the speech from the specific user is received. In an evaluation step, the presence / absence or the degree of cognitive function deterioration of the specific user is evaluated on the basis of the speech data and a preset trained model.

Description

Information processing system, information processing method and program

[0001] The present invention relates to an information processing system, an information processing method, and a program.

[0002] Patent Literature 1 discloses a device for evaluating the level of cognitive function from a user's speech. The device includes an instruction unit 160 that instructs the user to quickly pronounce a phrase in which predetermined syllables are repeated, an acquisition unit 110 that acquires speech data representing the speech uttered by the user in response to the instruction, a calculation unit 120 that calculates feature values ​​from the speech data acquired by the acquisition unit 110, an evaluation unit 130 that evaluates the cognitive function of the user from the feature values ​​calculated by the calculation unit 120, and an output unit 140 that outputs the evaluation results obtained by the evaluation unit 130.

[0003] Japanese Patent Application Laid-Open No. 2019-083903

[0004] However, there is still room for improvement in the functions of the above-mentioned known technologies. In particular, there is a need for a technology that can assist in the evaluation of the degree of cognitive decline according to the level of the user's audiovisual ability.

[0005] In consideration of the above circumstances, the present disclosure provides an information processing system etc. that can assist in evaluating the degree of cognitive decline based on the level of auditory and visual acuity of the user being evaluated.

[0006] According to one aspect of the present invention, an information processing system is provided. The information processing system includes a processor capable of executing a program to perform the following steps: In the acquisition step, specific user information, which is information about a specific user, is acquired. The specific user is a subject among users involved in the information processing system. The specific user information includes information that the specific user has a visual impairment or a hearing impairment. In the presentation step, instruction information including instructions for encouraging the specific user to make a predetermined utterance is presented to the specific user, which is variable depending on the specific user information. In the reception step, voice data related to the utterance from the specific user is received. In the evaluation step, the presence or degree of cognitive decline of the specific user is evaluated based on the voice data and a predetermined trained model.

[0007] According to the present disclosure, it is possible to provide an information processing system etc. that can assist in evaluating the degree of cognitive decline based on the level of auditory and visual acuity of the user being evaluated.

[0008] 1 is a configuration diagram showing an information processing system 1. FIG. 2 is a block diagram showing the hardware configuration of an information processing device 2. FIG. 3 is a block diagram showing the hardware configuration of a user terminal. FIG. 4 is a functional block diagram showing the functions of the information processing device 2. FIG. 5 is an activity diagram showing an example of the flow of information processing executed in the information processing system 1. FIG. 6 shows an example of a visual impairment check screen 6 displayed on the display unit 34 of the medical practitioner terminal 3. FIG. 7 shows an example of a voice input instruction screen 7 displayed on the display unit 34 of the medical practitioner terminal 3. FIG. 8 shows an example of a recording screen 8 displayed on the display unit 34 of the medical practitioner terminal 3. FIG. 9 is a schematic diagram of a neural network NN, which is an example of a trained model. FIG. 10 shows an example of an evaluation result display screen 9 displayed on the display unit 34 of the medical practitioner terminal 3.

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the accompanying drawings. Various features shown in the following embodiments can be combined with each other.

[0010] Incidentally, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable recording medium, or may be provided so as to be downloadable from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0011] In this embodiment, the term "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, various types of information are handled in this embodiment, and this information may be represented by, for example, physical values ​​of signal values ​​representing voltages and currents, high and low signal values ​​as a binary bit set consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculations may be performed on a circuit in the broad sense.

[0012] Furthermore, a circuit in the broad sense is a circuit realized by at least an appropriate combination of a circuit, circuitry, a processor, a memory, etc. That is, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0013] 1. Hardware Configuration This section explains the hardware configuration.

[0014] <Information Processing System 1> Fig. 1 is a configuration diagram showing an information processing system 1. The information processing system 1 comprises an information processing device 2, a medical professional terminal 3, and a sound collection device 4. The information processing device 2, the medical professional terminal 3, and the sound collection device 4 are configured to be able to communicate with each other via a telecommunications line. In one embodiment, the information processing system 1 is made up of one or more devices or components. For example, if the information processing system 1 is made up of only the information processing device 2, the information processing system 1 can be the information processing device 2. These components will be described below.

[0015] 2 is a block diagram showing the hardware configuration of the information processing device 2. The information processing device 2 includes a communication bus 20, a communication unit 21, a storage unit 22, and a processor 23. The communication unit 21, the storage unit 22, and the processor 23 are electrically connected via the communication bus 20 inside the information processing device 2.

[0016] <Communication Unit 21> The communication unit 21 is preferably a wired communication means such as USB, IEEE 1394, Thunderbolt (registered trademark), or wired LAN network communication, but may also include wireless LAN network communication, mobile communication such as 3G / LTE / 5G, or BLUETOOTH (registered trademark) communication as needed. In other words, it is more preferable to implement the communication unit 21 as a collection of multiple communication means. In other words, the information processing device 2 may communicate various information from the outside via the communication unit 21 and the network.

[0017] <Storage Unit 22> The storage unit 22 stores various pieces of information defined above. This may be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs and the like related to the information processing device 2 executed by the processor 23, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to program calculations. The storage unit 22 stores various programs, variables, etc. related to the information processing device 2 executed by the processor 23.

[0018] <Processor 23> The processor 23 processes and controls the overall operations related to the information processing device 2. The processor 23 is, for example, a central processing unit (CPU) not shown. The processor 23 realizes various functions related to the information processing device 2 by reading out predetermined programs stored in the storage unit 22. In other words, information processing by software stored in the storage unit 22 is specifically realized by the processor 23, which is an example of hardware, and can be executed as each functional unit included in the processor 23. These will be described in more detail in the next section. Note that the processor 23 is not limited to being single, and multiple processors 23 may be provided for each function. A combination of these may also be used.

[0019] <User Terminal> The user terminal is a terminal owned by a user. In this embodiment, a case will be described in which the user terminal is a terminal 3 (hereinafter referred to as a medical professional terminal 3) operated by a user who is a medical professional such as a doctor (hereinafter referred to as a medical professional). The user terminal may also include a terminal (hereinafter referred to as a subject terminal) operated by a user who is a patient whose cognitive function is to be evaluated (hereinafter referred to as a subject). Details will be described later. The user terminal may be in any form, such as a smartphone, tablet terminal, computer, or any other device that can access the information processing device 2 via an electric communication line.

[0020] 3 is a block diagram showing the hardware configuration of a user terminal. The following description uses the medical professional terminal 3 as an example. The medical professional terminal 3 includes a communication bus 30, a communication unit 31, a storage unit 32, a processor 33, a display unit 34, and an input unit 35. The communication unit 31, the storage unit 32, the processor 33, the display unit 34, and the input unit 35 are electrically connected via the communication bus 30 within the medical professional terminal 3. The description of the communication unit 31, the storage unit 32, and the processor 33 is omitted because they are the same as the descriptions of the various units in the information processing device 2.

[0021] <Display Unit 34> The display unit 34 displays a graphical user interface (GUI) screen that can be operated by the user. The display unit 34 may be included in the housing of the medical professional terminal 3 or may be attached externally. Specifically, the display unit 34 may be implemented as a display device such as a CRT display, a liquid crystal display, an organic EL display, or a plasma display. These display devices are preferably used depending on the type of medical professional terminal 3.

[0022] <Input Unit 35> The input unit 35 accepts operational inputs made by a medical professional or a subject. The operational inputs are transferred as command signals to the processor 33 via the communication bus 30. The processor 33 can execute predetermined control or calculations based on the transferred command signals as necessary. The input unit 35 may be included in the housing of the medical professional terminal 3 or may be externally attached. For example, the input unit 35 may be implemented as a touch panel integrated with the display unit 34. When the input unit 35 is implemented as a touch panel, the medical professional or user can input tapping, swiping, and the like to the input unit 35. Instead of a touch panel, a switch button, a mouse, a QWERTY keyboard, or the like can be used as the input unit 35.

[0023] The sound collector 4 is a microphone that can convert external sounds into signals. The sound collector 4 may be directly connected to the information processing device 2, but is also provided in or connected to the medical professional terminal 3.

[0024] The sound collection device 4 is configured to generate voice data by collecting the user's speech. The voice data is temporarily stored in a memory in the user terminal and does not have to be stored non-volatilely in the storage unit 32. The voice data generated by the sound collection device 4 is configured to be transferable to the information processing device 2 via a network.

[0025] The sound collection device 4 collects, but is not limited to, at least sounds in the human audible range, sounds with frequencies between 20 Hz and 20,000 Hz, and converts them into electrical signals. The sounds may be recorded in monaural or stereo. The sampling rate for digitally processing the sound data may be, for example, 48,000 Hz, 44,100 Hz, 32,000 Hz, 22,050 Hz, 16,000 Hz, 11,025 Hz, 11,000 Hz, 8,000 Hz, etc. The sampling rate may be within any of the ranges of values ​​exemplified here. Increasing the sampling rate allows for more precise discretization of the temporal timing of the sound, improving the accuracy of voice recognition.

[0026] Furthermore, the data collected by the sound collection device 4 may be appropriately compressed by the processor 33 of the medical professional terminal 3, and the compression format at this time may be any of MP3, AAC, WMA, Vorbis, AC3, MP2, FLAC, TAK, etc. Compression can reduce communication traffic caused by data transfer from the user terminal to the information processing device 2.

[0027] 2. Functional Configuration In this section, the functional configuration will be described. FIG. 4 is a functional block diagram showing the functions of the information processing device 2. As described above, information processing by software stored in the storage unit 22 can be specifically realized by hardware (specifically, the processor 23), and executed as each functional unit included in the processor 23. In other words, the information processing system 1 includes each functional unit in a program.

[0028] Specifically, the processor 23 includes, as its functional units, a reception unit 231, an acquisition unit 232, an identification unit 233, an evaluation unit 234, and a presentation unit 235.

[0029] The receiving unit 231 is configured to receive various information as a receiving step. Specifically, the receiving unit 231 is configured to receive information via the communication unit 21 or the storage unit 22 and read the information into the working memory. Preferably, the receiving unit 231 receives input of voice data related to the speech of the subject via the sound collection device 4.

[0030] The acquiring unit 232 is configured to acquire various pieces of information as an acquiring step. Specifically, the acquiring unit 232 acquires user information, which is information about the subject.

[0031] The specifying unit 233 is configured to specify various pieces of information as a specifying step. Specifically, the specifying unit 233 specifies a presentation mode for instructions to prompt the subject to speak, based on the user information acquired by the acquiring unit 232. Details will be described later.

[0032] The evaluation unit 234 is configured to evaluate various characteristics as the evaluation step. Preferably, the evaluation unit 234 evaluates the degree of cognitive decline based on the voice data of the subject acquired by the acquisition unit 232. Details will be described later.

[0033] As a presentation step, the presentation unit 235 presents various pieces of information stored in the storage unit 32 to the subject or medical professional in a manner that can be understood by the subject or medical professional. For example, the presentation unit 235 may display a screen containing various pieces of information in a manner that can be viewed on a terminal such as the medical professional terminal 3. This allows various pieces of information to be presented to various users operating the terminal such as the medical professional terminal 3. The presentation unit 235 also controls the display of visual information such as screens, images, icons, and messages on the display of the terminal. The presentation unit 235 may generate only rendering information for displaying the visual information on the terminal. The presentation unit 235 may also present various pieces of information to the subject or medical professional as auditory information. Furthermore, the presentation unit 235 may present various pieces of information to the subject as unevenness information that presents an uneven tactile sensation to the finger. Details will be described later. 3. Operation of the Information Processing System 1 In this section, the flow of the information processing method executed by the information processing system 1 will be described with reference to the figures. As shown below, the information processing method includes the following steps. The order of the processes can be changed as appropriate, multiple processes can be executed simultaneously, or some processes can be omitted.

[0034] 3.1. Overview of Information Processing FIG. 5 is an activity diagram showing an example of the flow of information processing executed in the information processing system 1. As an example, a case will be described in which the cognitive function of a subject is examined at a medical institution by analyzing the subject's speech data. In this embodiment, a case will be described in which both instructions in the form of auditory information IF2 and instructions in the form of visual information IF1 are presented to the subject as instructions to encourage the subject to speak. Furthermore, in this embodiment, "speaking" refers to speaking fixed words or phrases.

[0035] First, as a reception step, the reception unit 231 receives input of the subject's personal ID from a medical professional (A101). Specifically, for example, the reception unit 231 receives information indicating the subject's personal ID from the medical professional terminal 3 via the communication unit 21. Furthermore, the identification unit 233 may identify the subject's personal information from the information stored in the storage unit 22 by authenticating the user based on the personal ID received in activity A101.

[0036] User authentication does not necessarily have to be performed. The receiving unit 231 may also receive user information of the subject instead of the subject's personal ID. The receiving unit 231 may also receive input of personal information of the subject, such as the subject's name, age, or gender, as the user information. The user information may also be received from the subject himself / herself.

[0037] Next, as a specification step, the specification unit 233 specifies a presentation mode of instructions to prompt the subject to speak (A102). Preferably, the specification unit 233 specifies the presentation mode based on the subject's personal information (name, age, gender) linked to the subject's personal ID received in activity A101. Specifically, for example, when age information is received as the subject's personal information, the specification unit 233 specifies the presentation mode so that auditory information IF2 is presented at a slower speed than usual for older subjects. Furthermore, for example, when gender "female" is received as the subject's personal information, the specification unit 233 specifies the presentation mode so that audio data giving instructions in a female voice is presented. In particular, it is considered that instructions given by a voice of the same gender can give the subject a sense of security.

[0038] Meanwhile, a process for identifying the degree of disability of the subject is executed. As an example of a presentation step, the presentation unit 235 displays a screen for determining the degree of disability on the display unit 34 of the medical professional terminal 3 (A103). Specifically, the presentation unit 235 displays questions for identifying the degree of visual impairment or hearing impairment on the display unit 34 of the medical professional terminal 3 so that the questions can be presented to the subject. Details will be described later.

[0039] Next, the receiving unit 231 receives an answer from the subject to the question displayed in activity A103 (A104). Furthermore, the acquiring unit 232 acquires information indicating the degree of disability of the subject based on the received answer. In other words, the acquiring unit 232 acquires specific user information, which is information about a specific user. Here, the specific user is a person (subject) who corresponds to a subject among users involved in the information processing system 1. Furthermore, the specific user information includes information that the specific user has a visual impairment or a hearing impairment. Details will be described later.

[0040] Next, the identification unit 233 identifies the degree of disability of the subject (A105), as an identification step. Details will be described later.

[0041] Here, a case where a disability of a subject is identified in activity A105 will be described. As an identification step, the identification unit 233 identifies a change to the mode of the screen to be displayed on the display unit 34 of the medical professional terminal 3 based on the degree of disability of the subject identified in activity A105 (A106). Here, the screen displayed on the display unit 34 is a screen that displays instructions to encourage speech. For example, if the identification unit 233 identifies the subject as having color vision deficiency, the identification unit 233 may change the color of the display mode. Also, for example, if the identification unit 233 identifies the subject as having age-related visual impairment, the identification unit 233 may change the brightness and size of the characters displayed on the screen. Details will be described later.

[0042] Next, the presentation unit 235 presents instructions to the user (the subject) using auditory information IF2 to prompt the user to make a predetermined speech (A107). That is, the instruction information for prompting the subject to speak includes auditory information IF2. The auditory information IF2 is audio information presented to the subject. In this embodiment, a case will be described in which the presentation unit 235 also presents instructions to the user using visual information IF1. In this case, the visual information IF1 is composed of at least one of text, graphics, and photographs. Specifically, the presentation unit 235 presents instructions to the user to prompt the user to make a predetermined speech based on the presentation format identified in activity A102 and the change screen identified in activity A106. In other words, the presentation unit 235 presents instruction information to the specific user, including instructions to prompt the specific user to make a predetermined speech, which is variable depending on specific user information including information about the subject's disability. Details will be described later.

[0043] On the other hand, if the subject's disability is not identified in activity A105, the process proceeds to activity A107. In this case, the presentation unit 235 presents the user with an instruction to prompt them to speak, based on the presentation mode identified in activity A102 (A107). In other words, the acquisition unit 232 acquires user information, which is information about the user, in activity A101. Furthermore, the presentation unit 235 controls the content of the auditory information IF2 based on the user information and presents it.

[0044] Next, the receiving unit 231 receives voice data relating to the speech of the user who is the subject (A108). The voice data is data generated by the sound collection device 4 based on the speech of the subject.

[0045] At the same time as receiving the voice data of activity A108, the presenting unit 235 updates the screen displaying the instruction to speak (A109). Here, the screen displays instructions to enable the subject to understand the timing at which to speak. In other words, the receiving unit 231 starts receiving the voice data before the presenting unit 235 completes presenting the instructions.

[0046] Next, the evaluation unit 234 evaluates the presence or degree of cognitive decline of the user based on the voice data and a pre-trained model (step A110). Here, the pre-trained model is a model that has been trained in advance to determine the relationship between predetermined features of speech and the presence or degree of cognitive decline. Details will be described later.

[0047] The presentation unit 235 then presents the evaluation results to the subject and medical professionals, as will be described in detail later.

[0048] In summary, the information processing system 1 according to this embodiment includes a processor 23 capable of executing programs to operate the following units. The presentation unit 235 presents instructions to the user via auditory information IF2 to prompt the user to produce a predetermined speech sound. The reception unit 231 receives voice data related to the speech sound from the user. The evaluation unit 234 evaluates the presence or degree of cognitive decline of the user based on the voice data and a pre-set trained model. The trained model is a model that has been trained in advance to determine the relationship between the features of a predetermined speech sound and the presence or degree of cognitive decline.

[0049] According to this aspect, when evaluating a user's cognitive decline, it is possible to obtain voice data that reflects the subject's original speech state. Therefore, it is possible to improve the accuracy of the evaluation of cognitive decline. Furthermore, by presenting the auditory information IF2 and the visual information IF1 when evaluating a user's cognitive decline, it is possible to reduce the user's psychological anxiety.

[0050] The information processing system 1 according to this embodiment further includes an acquisition unit 232, an identification unit 233, and a presentation unit 235. The acquisition unit 232 acquires specific user information, which is information about a specific user. The specific user is a person who corresponds to a subject among users involved in the information processing system 1. The specific user information includes information that the specific user has a visual impairment or a hearing impairment. The presentation unit 235 presents to the specific user instruction information, which includes instructions for encouraging the specific user to make a predetermined utterance, to vary depending on the specific user information.

[0051] According to this aspect, speech support can be provided according to the degree of the user's disability, and speech data that reflects the subject's original speech state can be obtained, thereby improving the accuracy of the assessment of cognitive decline.

[0052] 3.2 Details of Information Processing Details of the information processing described above will be explained with reference to FIGS.

[0053] (Visual Impairment Check Screen 6) Figure 6 shows an example of the visual impairment check screen 6 displayed on the display unit 34 of the medical professional terminal 3. In this embodiment, an example of determining the degree of color vision deficiency as a visual impairment will be described. The visual impairment check screen 6 has an area 61 where input instructions for the subject are displayed, an area 62 where selection inputs from the subject are accepted, and a button 63.

[0054] As shown in FIG. 6 , an input instruction to the test subject, "Please select a box that is a different color," is displayed in area 61. Area 62 is an area that accepts a selective input of an answer to the input instruction from the test subject. Area 62 has multiple check boxes, including check box 621 and check box 622. As shown in FIG. 6 , among the multiple check boxes, check box 621 is displayed in a color different from the other check boxes. On the other hand, check box 622 is displayed in the same color as the other check boxes except for check box 621.

[0055] 6 , the subject can answer the color vision deficiency check by selecting any check box in area 62 in response to the instruction displayed in area 61. The acquiring unit 232 can acquire information indicating the degree of the subject's color vision deficiency by receiving a selection input from the check box from the subject. In other words, the acquiring unit 232 acquires user information by receiving an input operation from the user who is the subject.

[0056] For example, when the identification unit 233 receives a selection input from the subject for the check box 621, the identification unit 233 may determine that the subject is unlikely to have color vision deficiency. On the other hand, when the identification unit 233 receives a selection input from the subject for a check box other than the check box 621, the identification unit 233 may determine that the subject is likely to have color vision deficiency.

[0057] Furthermore, the identification unit 233 may determine the degree of color vision deficiency of the test subject based on the results of performing such a check multiple times. The button 63 is a button for displaying check content different from the check content displayed on the display unit 34. For example, by receiving input from the test subject using the button 63, the presentation unit 235 transitions the content displayed on the display unit 34 from the check screen 6 in FIG. 6 to a different check screen.

[0058] In the present embodiment, an example has been described in which the degree of color vision impairment is determined as a visual impairment, but this is not limiting. For example, the degree of visual impairment may be determined as a visual acuity impairment. Also, for example, the degree of accommodation impairment may be determined as a visual impairment. Furthermore, the degree of hearing impairment may be determined.

[0059] In this embodiment, a case will be described in which the subject is identified as having color vision deficiency based on the result of a selection input received from the subject. The identification unit 233 determines that the color scheme displayed on the display unit 34 of the medical professional terminal 3 should be changed to a color scheme that the subject can recognize. Specifically, the identification unit 233 changes the color scheme displayed on the instruction screen 7 in Figure 7, which will be described later. This allows the subject to recognize instructions to prompt speech without the need for assistance from a third party.

[0060] (Instruction screen 7) Figure 7 shows an example of the voice input instruction screen 7 displayed on the display unit 34 of the medical professional terminal 3. The voice input instruction screen 7 is an example of a screen displayed in activity A107 in Figure 5. The voice input instruction screen 7 has a character 71, an area 72 where instructions to the subject are displayed, and a button 73.

[0061] As shown in FIG. 7 , the character 71 has an appearance that imitates, for example, the appearance of a female medical professional. The character 71 is used as a character that instructs the subject on the specific content of the voice input. The appearance of the character 71 may be changed according to the presentation mode specified in activity A102. For example, if the subject is a woman, the character 71 having a female appearance may be selected. Furthermore, the appearance of the character 71 is not limited to a person, but may also be an animal, a fictional character, or the like.

[0062] As shown in FIG. 7 , an area 72 displays a speech instruction to the subject, "Please say 'pataka' five times in a row." The instruction displayed in area 72 is also simultaneously presented to the subject by auditory information IF2. This allows the subject to recognize the intonation of the speech before recording the audio. The character 71 may also be displayed so as to change in conjunction with the auditory information IF2 presented to the subject. In other words, the presentation unit 235 further presents the character 71 to the user, who is the subject, in a visually recognizable manner. The character 71 is configured to change in accordance with the presentation state of the auditory information IF2, which changes over time.

[0063] Furthermore, the visual information IF1 that changes in conjunction with the auditory information IF2 is not limited to the character 71. As an example, the text information IF11 displayed in the area 72 may be displayed in a manner that changes in conjunction with the auditory information IF2. That is, the visual information IF1 is configured to change in accordance with the presentation state of the auditory information IF2, which changes over time. This configuration allows the subject to better understand the content of the test. Therefore, the psychological anxiety of the subject can be reduced when testing the subject's cognitive decline.

[0064] 7, a button 73 is a button for starting voice input by the subject. For example, when the presentation unit 235 receives input from the subject or a medical professional via the button 73, the presentation unit 235 transitions the content displayed on the display unit 34 from the instruction screen 7 in FIG. 7 to the recording screen 8 in FIG. 8.

[0065] (Recording screen 8) Figure 8 shows an example of the recording screen 8 displayed on the display unit 34 of the medical professional terminal 3. The recording screen 8 is an example of a screen displayed in activity A108 of Figure 5. The recording screen 8 includes an area 81, an area 82, a record button 83, a stop button 84, an area 85, an area 86, and an area 87.

[0066] Area 81 is an area where instructions regarding speech are displayed to the subject. As shown in FIG. 8 , area 81 displays the instruction "After tapping the record button, please read the following aloud." Area 82 is an area where the specific speech content to be spoken by the subject is displayed. As shown in FIG. 8 , area 82 displays the speech content "Pataka (5 times)."

[0067] The record button 83 is a button for starting recording. The acquisition unit 232 acquires voice data of the subject's speech by accepting input to the record button 83 from the subject or medical professional. The stop button 84 is a button for stopping recording. The acquisition unit 232 stops acquiring voice data by accepting input to the stop button 84 from the subject or medical professional.

[0068] Area 85 is an area where a visual indication is displayed that the subject's voice is being recorded. When the presentation unit 235 receives input from the subject or medical professional via the record button 83, it displays "REC" in area 85. When the presentation unit 235 receives input from the subject or medical professional via the stop button 84, it stops displaying "REC" in area 85.

[0069] Area 86 is an area where the elapsed time from the start of recording is visibly displayed. As shown in Fig. 8 , area 86 may display the elapsed time using object IF14. Area 86 may also display the elapsed time using text information IF15 "00:02.00". Area 86 may also visibly display the remaining time.

[0070] Area 87 is an area where information indicating the timing of the subject's speech is displayed. In other words, the presentation unit 235 further presents information including instructions indicating the timing of speech to the user who is the subject in a comprehensible manner. In other words, the presentation unit 235 further presents guide information including information guiding the timing of speech to the specific user who is the subject in a comprehensible manner. As shown in FIG. 8 , area 87 displays information "flip flop" indicating the content of the subject's speech. Furthermore, the text information displayed in area 87 includes black text information IF12 and white text information IF13. As an example, when the timing of speech arrives, the style of the text indicating the content of speech changes from white text information IF13 to black text information IF12. In the example of FIG. 8 , the next word to be spoken by the subject is "ka," and when the timing of speech arrives, the white text information IF13 "ka" changes to black text information IF13.

[0071] Here, the manner in which the timing of speech can be grasped is not limited to this. For example, the manner in which the characters indicating the content of speech are emphasized may change as the timing of speech arrives. Furthermore, the manner in which the characters are emphasized may change to return to their original manner as the timing of speech ends.

[0072] Furthermore, the instruction indicating the timing of speech is not limited to a change in the appearance of characters. For example, an object with the appearance of a metronome may be displayed in area 87. The object displayed in area 87 may be presented to the user in the form of a pendulum needle moving left and right in time with the timing of the subject's speech. Also, for example, an object with the appearance of clapping hands may be displayed in area 87. The object displayed in area 87 may be presented to the user in the form of clapping hands in time with the timing of the subject's speech. According to such an embodiment, the subject can speak while understanding the timing and rhythm of the speech. Therefore, it is possible to obtain voice data that reflects the subject's original speaking state.

[0073] Here, a method for evaluating the degree of cognitive decline of a user from voice data will be described. FIG. 9 is a schematic diagram of a neural network NN, which is an example of a trained model. An input signal defined by various parameters is input to a first layer L1. Here, the input signal is input voice data. This input signal is output from computation nodes N_11 to N_13 in the first layer L1 to computation nodes N_21 to N_25 in the second layer L2. At this time, the values ​​output from computation nodes N_11 to N_13 are multiplied by weights w set between each computation node N, and the resulting values ​​are input to computation nodes N_21 to N_25.

[0074] Computation nodes N_21 to N_25 add up the input values ​​from computation nodes N_11 to N_13, and input this value (or a value obtained by adding a predetermined bias value to this value) to a predetermined activation function.

[0075] The output value of the activation function is then propagated to the next node, computation node N_31. At this time, the value obtained by multiplying the weight w set between computation nodes N_21 to N_25 and computation node N_31 by the output value is input to computation node N_31. Computation node N_31 adds up the input values ​​and outputs the sum as an output signal. At this time, computation node N_31 may add up the input values, add a bias value to the sum, input the result to the activation function, and output the output value as an output signal. This outputs the presence or degree of cognitive decline.

[0076] It should be noted that what is shown in FIG. 9 is for illustrative purposes only and is not limiting.

[0077] Here, the information indicating the presence or absence or degree of cognitive decline is not limited to a particular format, but preferably includes information indicating a stage in numbers as an evaluation value. More preferably, the evaluation value may be classified into "normal," "cognitive decline," "risk of cognitive decline," or "risk of dementia" depending on the stage. In this embodiment, an example will be described in which the information indicating the presence or absence or degree of cognitive decline is information indicating a stage in numbers.

[0078] (Evaluation result display screen 9) Figure 10 shows an example of the evaluation result display screen 9 displayed on the display unit 34 of the medical professional terminal 3. The evaluation result display screen 9 includes an area 91, an area 92, an area 93, and a remeasurement button 94.

[0079] Preferably, area 91 is an area where information on cognitive decline, which is the subject of evaluation, is displayed. As shown in Fig. 9 , area 91 displays "audio evaluation result of cognitive function" as the subject of evaluation. Also preferably, area 92 is an area where measurement result 921 is displayed. In the example of Fig. 9 , area 92 displays information that "cognitive function is at a normal level" as measurement result 921.

[0080] More preferably, the area 93 is an area where a comparison result 931 showing a comparison between the current evaluation result and the previous evaluation result is displayed when the subject has previously received the same evaluation. For example, the comparison result 931 is evaluated based on the amount of change in the current evaluation value 934 relative to the previous evaluation value 935, which will be described later. In other words, the acquisition unit 232 further acquires evaluation information including past evaluation results of the specific user who is the subject. The evaluation unit 234 further evaluates the amount of change in the degree of cognitive decline of the specific user based on the evaluation information. In the example of FIG. 9 , the area 93 displays the information "no change" as the comparison result 931 showing the comparison with the previous time.

[0081] Preferably, an item 932 "this time", an item 933 "last time", a current evaluation value 934, and a previous evaluation value 935 are further displayed so as to be visible to the subject in the area 93. In the example of Fig. 9, for example, the evaluation unit 234 evaluates the speech of the subject in both the previous measurement and the current measurement and determines that the speech is "normal".

[0082] This allows the subject to understand the level of their own cognitive function, and also allows the progress of the user's cognitive decline to be quantitatively evaluated.

[0083] [Others] The information processing system 1 may employ the following aspects.

[0084] In the above-described embodiment, the screen displaying the speech instruction is updated simultaneously with the reception of the speech data by activity A108. However, this is not limited to this. The reception unit 231 may start receiving the speech data after the presentation of the speech instruction is completed. In other words, the reception unit 231 may start receiving the speech data after the presentation of the instruction by the presentation unit 235 is completed. According to this embodiment, if the subject is aware of the timing to speak, the subject can concentrate on speaking. Therefore, it is possible to obtain speech data that reflects the subject's original speech state.

[0085] In the above-described embodiment, the presentation unit 235 presented instructions to the subject to prompt speech using visual information IF1 and auditory information IF2, but this is not limited to this. For example, if the subject is visually impaired, the presentation unit 235 may present instructions to the subject using Braille information. In other words, the instruction information for prompting speech includes texture information for providing a textured feel to the subject's finger. This configuration can reduce psychological anxiety regarding the examination, even for subjects with visual impairments such as visually impaired subjects. Therefore, audio data reflecting the subject's original speech state can be obtained. When presenting Braille information to the subject, it is preferable to use a display compatible with Braille as the display unit 34 of the medical professional terminal 3. The presentation unit 235 converts the text information, which is the instruction information, into Braille information and presents it to the subject.

[0086] In the above-described embodiment, the evaluation unit 234 evaluated the degree of cognitive decline based on speech data in which the subject spoke fixed words or phrases. However, this is not limited to this. For example, the evaluation unit 234 may evaluate the degree of cognitive decline based on speech data in which the subject spoke in a conversational format. Specifically, in activity A108 of FIG. 5 , the reception unit 231 receives speech data in which the subject spoke in response to a question. For example, the presentation unit 235 may present the subject with questions such as, "What did you do yesterday?" or "What was the most memorable event in the past week?". Furthermore, the presentation unit 235 may present the subject with further questions related to the content of the answers received from the subject. For example, the evaluation unit 234 may evaluate the degree of cognitive decline of the subject based on the amount of speech acquired from the received speech data. In such a case, a trained model that has previously trained conversational interactions may be used.

[0087] When the degree of cognitive decline is evaluated in a conversational format, the presenting unit 235 may present the subject with the timing to speak answers to questions. In other words, the presenting unit 235 may further present guide information including information guiding the timing of speaking to the specific user who is the subject in a manner that the user can understand. According to this embodiment, the subject can understand the timing to respond to questions, thereby reducing tension and psychological anxiety regarding the test.

[0088] In the above-described embodiment, the identification unit 233 identifies the degree of disability of the subject through a simple check, but this is not limited to this. For example, the reception unit 231 may receive the degree of disability from the subject through self-reporting by the subject. The identification unit 233 may identify the degree of disability based on the information received from the subject.

[0089] In the above-described embodiment, the subject undergoes testing at a medical institution, but this is not limited thereto. For example, the subject may be able to undergo testing using their own subject terminal. Specifically, the information processing system 1 may include a subject terminal in addition to the medical professional terminal 3 as a user terminal. The subject terminal may have, for example, substantially the same hardware configuration as the information processing device 2 and the medical professional terminal 3. The communication unit of the subject terminal may transmit the subject's voice data generated by the sound collection device 4 to the information processing device 2 or the medical professional terminal 3. This allows the subject to easily undergo testing for the degree of cognitive decline at home, etc., and receive a diagnosis from a medical professional remotely.

[0090] The information processing device 2 may be an on-premise type or a cloud type. As the information processing device 2 in the cloud type, for example, the above functions and processes may be provided in the form of SaaS (Software as a Service) or cloud computing.

[0091] In the above embodiment, the information processing device 2 performs various storage and control operations, but multiple external devices may be used instead of the information processing device 2. That is, various information and programs may be distributed and stored in multiple external devices using blockchain technology or the like.

[0092] Another aspect of the embodiment may be a program that causes a computer to execute each step.

[0093] It may be provided in the following manner.

[0094] (1) An information processing system comprising a processor capable of executing a program to perform each of the following steps: in an acquisition step, specific user information is acquired, which is information about a specific user, wherein the specific user is a subject among users involved in the information processing system, and the specific user information includes information that the specific user has a visual impairment or a hearing impairment; in a presentation step, instruction information including instructions for encouraging the specific user to make a predetermined utterance is presented to the specific user, which information is variable depending on the specific user information; in a reception step, voice data related to the utterance from the specific user is received; and in an evaluation step, the presence or degree of cognitive decline of the specific user is evaluated based on the voice data and a predetermined trained model.

[0095] According to this aspect, speech support can be provided according to the degree of the user's disability.

[0096] (2) In the information processing system described in (1) above, the trained model is a model that has been trained in advance to determine the relationship between the content of a conversation or the features of a predetermined speech voice and the presence or degree of cognitive decline.

[0097] (3) In the information processing system described in either (1) or (2) above, the acquisition step further acquires evaluation information including past evaluation results of the specific user, and the evaluation step further evaluates the amount of change in the degree of cognitive function decline of the specific user based on the evaluation information.

[0098] According to this aspect, the progress of the user's cognitive decline can be accurately evaluated.

[0099] (4) In the information processing system described in any one of (1) to (3) above, the acquisition step acquires the specific user information by accepting an input operation from the user.

[0100] (5) In the information processing system described in any one of (1) to (4) above, in the presentation step, guide information including information guiding the timing of speech is further presented to the specific user in a manner that the user can grasp.

[0101] According to this embodiment, it is possible to reduce the psychological anxiety of the subject, and therefore to obtain high-quality voice data.

[0102] (6) In the information processing system described in any one of (1) to (5) above, the instruction information includes unevenness information for presenting an uneven tactile sensation to the finger of the specific user.

[0103] According to this aspect, even a visually impaired user can obtain high-quality audio data.

[0104] (7) In the information processing system described in any one of (1) to (6) above, the instruction information includes auditory information.

[0105] According to this aspect, even a visually impaired user can obtain high-quality audio data.

[0106] (8) In the information processing system described in any one of (1) to (7) above, the instruction information includes visual information, and the visual information is composed of at least one of characters, figures, and photographs.

[0107] According to this aspect, even a user with a hearing impairment can obtain high-quality voice data.

[0108] (9) An information processing method, comprising the steps of the information processing system according to any one of (1) to (8) above.

[0109] (10) A program that causes a computer to execute each step described in any one of (1) to (8) above. Of course, this is not a limitation.

[0110] Finally, while various embodiments of the present disclosure have been described, they are presented as examples and are not intended to limit the scope of the invention. The novel embodiments may be embodied in various other forms, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. Such embodiments and modifications are intended to be included within the scope and spirit of the invention, as well as within the scope of the inventions and their equivalents as defined in the claims.

[0111] 1: Information processing system 2: Information processing device 3: Medical professional terminal 4: Sound collection device 6: Check screen 7: Instruction screen 8: Recording screen 9: Evaluation result display screen 20: Communication bus 21: Communication unit 22: Memory unit 23: Processor 30: Communication bus 31: Communication unit 32: Memory unit 33: Processor 34: Display unit 35: Input unit 61: Area 62: Area 63: Button 71: Character 72: Area 73: Button 81: Area 82: Area 83: Recording button 84: Stop button 85: Area 86: Area 87: Area 91: Area 92: Area 93: Area 94: Remeasurement button 110: Acquisition unit 120: Calculation unit 130: Evaluation unit 140: Output unit 160: Instruction unit 231: Reception unit 232: Acquisition unit 233: Identification unit 234: Evaluation unit 235: Presentation unit 621: Check box 622: Check box 921: Measurement result 931: Comparison result 932: Item 933: Item 934: Evaluation value 935: Evaluation value IF1: Visual information IF11: Character information IF12: Character information IF13: Character information IF14: Object IF15: Character information IF2: Auditory information L1: First layer L2: Second layer N: Computation node NN: Neural network N_11: Computation node N_21: Computation node N_31: Computation node w: Weight

Claims

1. An information processing system comprising a processor capable of executing a program to perform each of the following steps: an acquisition step acquires specific user information, which is information about a specific user, wherein the specific user is a subject among users involved in the information processing system, and the specific user information includes information that the specific user has a visual impairment or a hearing impairment; a presentation step presents to the specific user instruction information, the instruction information including instructions for encouraging the specific user to speak in a predetermined manner, which is variable depending on the specific user information; a reception step accepts voice data related to the speech from the specific user; and an evaluation step evaluates the presence or degree of cognitive decline of the specific user based on the voice data and a predetermined trained model.

2. An information processing system according to claim 1, wherein the trained model is a model that has been trained in advance to determine the relationship between the content of a conversation or a specified speech feature and the presence or degree of cognitive decline.

3. An information processing system according to any one of claims 1 or 2, wherein the acquisition step further comprises acquiring evaluation information including past evaluation results of the specific user, and the evaluation step further comprises evaluating the amount of change in the degree of cognitive decline of the specific user based on the evaluation information.

4. An information processing system according to any one of claims 1 to 3, wherein in the acquisition step, the specific user information is acquired by accepting an input operation from the user.

5. An information processing system according to any one of claims 1 to 4, wherein the presenting step further comprises presenting to the specific user guide information including information guiding on the timing of speech in a comprehensible manner.

6. An information processing system according to any one of claims 1 to 5, wherein the instruction information includes unevenness information for presenting an uneven tactile sensation to the finger of the specific user.

7. An information processing system according to any one of claims 1 to 6, wherein the instruction information includes auditory information.

8. An information processing system according to any one of claims 1 to 7, wherein the instruction information includes visual information, and the visual information is composed of at least one of characters, figures, and photographs.

9. An information processing method comprising the steps of an information processing system according to any one of claims 1 to 8.

10. A program causing a computer to execute each step according to any one of claims 1 to 8.

Images