Information processing systems, information processing methods, and programs

Abstract

This invention provides an information processing system, method, and program that support the evaluation of the degree of cognitive decline in accordance with the degree of hearing and vision of the user being evaluated. [Solution] An information processing system comprising an information processing device, a medical worker terminal, and a sound collection device that can communicate via a telecommunications line, wherein the information processing device includes a processor capable of executing a program so that the following steps are performed. The processor includes an acquisition unit that acquires specific user information, which is information that a specific user among the users involved in the information processing system has a visual impairment or a hearing impairment; a presentation unit that presents instruction information to the specific user, which includes instructions to prompt the specific user to make a predetermined utterance, in a variable manner according to the specific user information; a reception unit that receives voice data relating to the utterance from the specific user; and an evaluation unit that evaluates whether or not there is a decline in the cognitive function of the specific user, or the degree thereof, based on the voice data and a pre-set trained model.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] Patent Document 1 discloses an apparatus for evaluating the degree of a cognitive function from a user's spoken voice. The apparatus includes an instruction unit 160 that instructs to quickly pronounce a phrase in which a predetermined syllable is repeated, an acquisition unit 110 that acquires voice data indicating the voice uttered by the evaluated person who has received the instruction, a calculation unit 120 that calculates a feature amount from the voice data acquired by the acquisition unit 110, an evaluation unit 130 that evaluates the cognitive function of the evaluated person from the feature amount calculated by the calculation unit 120, and an output unit 140 that outputs the evaluation result evaluated by the evaluation unit 130.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, there is still room for improvement in the functions according to the above-known technology. In particular, in the evaluation of the degree of cognitive function decline, a technology that can assist the evaluation according to the degree of the user's visual and auditory abilities as the evaluated person is desired.

[0005] In view of the above circumstances, the present disclosure aims to provide an information processing system and the like that can assist the evaluation of the degree of cognitive function decline according to the degree of the user's visual and auditory abilities as the evaluated person.

Means for Solving the Problems

[0006] According to one aspect of the present invention, an information processing system is provided. This information processing system includes a processor capable of executing a program so as to perform the following steps: In the acquisition step, specific user information, which is information relating to a specific user, is acquired. The specific user is a person who corresponds to a test subject among the users involved in the information processing system. The specific user information includes information indicating that the specific user has a visual impairment or a hearing impairment. In the presentation step, instruction information, which includes instructions to prompt the specific user to make a predetermined utterance, is presented to the specific user in a variable manner according to the specific user information. In the reception step, audio data relating to the utterance from the specific user is received. In the evaluation step, the presence or degree of cognitive decline in the specific user is evaluated based on the audio data and a pre-set trained model.

[0007] According to this disclosure, it is possible to provide an information processing system, etc., that can support the evaluation of the degree of cognitive decline according to the degree of hearing and vision of the user being evaluated. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram showing the configuration of Information Processing System 1. [Figure 2] This is a block diagram showing the hardware configuration of the information processing device 2. [Figure 3] A block diagram showing the hardware configuration of the user terminal. [Figure 4] This is a functional block diagram showing the functions of the information processing device 2. [Figure 5] This is an activity diagram showing an example of the information processing flow performed in Information Processing System 1. [Figure 6] This shows an example of a visual impairment check screen 6 displayed on the display unit 34 of the medical professional terminal 3. [Figure 7] This shows an example of the voice input instruction screen 7 displayed on the display unit 34 of the medical device terminal 3. [Figure 8] This shows an example of the recording screen 8 displayed on the display unit 34 of the medical device terminal 3. [Figure 9] This is an overview diagram of a neural network (NN), which is an example of a pre-trained model. [Figure 10] This shows an example of the evaluation result display screen 9 shown on the display unit 34 of the medical professional terminal 3. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. The various features shown in the embodiments below can be combined with each other.

[0010] Incidentally, the program for implementing the software appearing in this embodiment may be provided as a non-transitory computer-readable medium, or it may be provided so that it can be downloaded from an external server, or it may be provided so that the program is launched on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0011] Furthermore, in this embodiment, "part" may include, for example, hardware resources implemented by circuits in a broad sense, and the information processing of software that can be specifically realized by these hardware resources. In addition, various types of information are handled in this embodiment, and these types of information can be represented, for example, by the physical values ​​of signal values ​​representing voltage and current, the high or low values ​​of signal values ​​as a set of binary bits composed of 0s or 1s, or by quantum superposition (so-called qubits), and communication and calculations can be performed on circuits in a broad sense.

[0012] In addition, a circuit in a broad sense is a circuit realized by appropriately combining at least a circuit, circuitry, a processor, a memory, etc. That is, it includes an application specific integrated circuit (ASIC), programmable logic devices (e.g., simple programmable logic device (SPLD), complex programmable logic device (CPLD), and field programmable gate array (FPGA)), etc.

[0013] 1. Hardware Configuration In this section, the hardware configuration will be described.

[0014] <Information Processing System 1> FIG. 1 is a configuration diagram showing an information processing system 1. The information processing system 1 includes an information processing apparatus 2, a medical staff terminal 3, and a sound collection apparatus 4. The information processing apparatus 2, the medical staff terminal 3, and the sound collection apparatus 4 are configured to be communicable through a telecommunication line. In one embodiment, the information processing system 1 is composed of one or more devices or components. For example, if it consists only of the information processing apparatus 2, the information processing system 1 can be the information processing apparatus 2. Hereinafter, these components will be described.

[0015] <Information Processing Apparatus 2> FIG. 2 is a block diagram showing the hardware configuration of the information processing apparatus 2. The information processing apparatus 2 includes a communication bus 20, a communication unit 21, a storage unit 22, and a processor 23. The communication unit 21, the storage unit 22, and the processor 23 are electrically connected via the communication bus 20 inside the information processing apparatus 2.

[0016] <Communication Unit 21> Although a wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), or wired LAN network communication is preferred for the communication unit 21, wireless LAN network communication, mobile communication such as 3G / LTE / 5G, or BLUETOOTH (registered trademark) communication may be included as needed. That is, it is more preferable to implement it as a collection of these plural communication means. That is, the information processing apparatus 2 may communicate various information from the outside via the communication unit 21 and a network.

[0017] <Storage unit 22> The storage unit 22 stores various information defined as described above. This can be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs and the like related to the information processing apparatus 2 executed by the processor 23, or as a memory such as a random access memory (RAM) that stores information (arguments, arrays, etc.) temporarily necessary for program operations. The storage unit 22 stores various programs, variables, and the like related to the information processing apparatus 2 executed by the processor 23.

[0018] <Processor 23> The processor 23 performs processing and control of the overall operations related to the information processing apparatus 2. The processor 23 is, for example, a central processing unit (CPU) not shown in the figure. The processor 23 realizes various functions related to the information processing apparatus 2 by reading a predetermined program stored in the storage unit 22. That is, the information processing by software stored in the storage unit 22 is specifically realized by the processor 23, which is an example of hardware, and can be executed as each functional unit included in the processor 23. These will be described in more detail in the next section. Note that the processor 23 is not limited to being single, and may be implemented to have a plurality of processors 23 for each function, or a combination thereof may also be used.

[0019] <User terminal> A user terminal is a device owned by the user. In this embodiment, we will describe the case where the user terminal is a terminal 3 operated by a medical professional such as a doctor (hereinafter referred to as "medical professional terminal 3"). In addition, the user terminal may also include a terminal operated by a patient whose cognitive function is being evaluated (hereinafter referred to as "subject terminal"). Details will be described later. The user terminal can be a smartphone, tablet, computer, or any other device that can access the information processing device 2 via a telecommunications line, regardless of its form.

[0020] Figure 3 is a block diagram showing the hardware configuration of a user terminal. The following explanation will use the medical professional terminal 3 as an example. The medical professional terminal 3 comprises a communication bus 30, a communication unit 31, a storage unit 32, a processor 33, a display unit 34, and an input unit 35. The communication unit 31, storage unit 32, processor 33, display unit 34, and input unit 35 are electrically connected within the medical professional terminal 3 via the communication bus 30. The explanation of the communication unit 31, storage unit 32, and processor 33 is the same as the explanation of each part in the information processing device 2, so it will be omitted.

[0021] <Display section 34> The display unit 34 displays a graphical user interface (GUI) screen that can be operated by the user. The display unit 34 may be included in the medical terminal 3 housing or it may be an external component. Specifically, the display unit 34 may be implemented as a display device such as a CRT display, liquid crystal display, organic EL display, or plasma display. It is preferable that these display devices be used according to the type of medical terminal 3.

[0022] <Input section 35> The input unit 35 receives operation inputs made by a medical professional or a patient. The operation input is transmitted to the processor 33 via the communication bus 30 as a command signal. The processor 33 can perform predetermined controls or calculations based on the transmitted command signal as needed. The input unit 35 may be included in the housing of the medical professional terminal 3 or it may be external. For example, the input unit 35 may be implemented as a touch panel integrated with the display unit 34. When the input unit 35 is implemented as a touch panel, the medical professional or user can input tap operations, swipe operations, etc. to the input unit 35. Instead of a touch panel, the input unit 35 can be a switch button, a mouse, a QWERTY keyboard, etc.

[0023] <Sound collection device 4> The sound collection device 4 is a so-called microphone configured to convert external sounds into signals. The sound collection device 4 may be provided by directly connecting the sound collection device 4 to the information processing device 2, or it may be provided with or connected to the medical worker terminal 3.

[0024] The sound collection device 4 is configured to generate voice data by collecting the user's speech. The voice data is temporarily stored in the memory of the user terminal and does not necessarily need to be stored non-volatilely in the storage unit 32. The voice data generated by the sound collection device 4 is configured to be transferable to the information processing device 2 via the network.

[0025] The sound collection device 4 is not particularly limited, but it collects sound in the human audible range, specifically between 20 Hz and 20,000 Hz, and converts it into an electrical signal. The sound may be recorded in mono or stereo. When digitally processing the sound data, the sampling rate may be, for example, 48,000 Hz, 44,100 Hz, 32,000 Hz, 22,050 Hz, 16,000 Hz, 11,025 Hz, 11,000 Hz, 8,000 Hz, etc. Any of the values ​​exemplified here may be used. By increasing the sampling rate, the temporal timing of the sound can be discretized more precisely, thereby improving the accuracy of speech recognition.

[0026] Furthermore, the data collected by the sound collection device 4 may be compressed as appropriate by the processor 33 of the medical staff terminal 3, and the compression format may be any of MP3, AAC, WMA, Vorbis, AC3, MP2, FLAC, TAK, etc. Compression can reduce communication traffic caused by data transfer from the user terminal to the information processing device 2.

[0027] 2. Functional Configuration This section describes the functional configuration. Figure 4 is a functional block diagram showing the functions of the information processing device 2. As mentioned above, the information processing performed by the software stored in the memory unit 22 is concretely realized by the hardware (specifically, the processor 23), and can be executed as each functional unit included in the processor 23. In other words, the information processing system 1 comprises each functional unit in the program.

[0028] Specifically, the processor 23 includes, as functional units, a receiving unit 231, an acquisition unit 232, a identification unit 233, an evaluation unit 234, and a presentation unit 235.

[0029] The reception unit 231 is configured to receive various types of information as a reception step. Specifically, the reception unit 231 is configured to receive information via the communication unit 21 or the storage unit 22 and to read it into the working memory. Preferably, the reception unit 231 receives input of voice data relating to the subject's speech via the sound collection device 4.

[0030] The acquisition unit 232 is configured to acquire various types of information as an acquisition step. Specifically, the acquisition unit 232 acquires user information, which is information about the subject.

[0031] The identification unit 233 is configured to identify various types of information as part of the identification step. Specifically, based on the user information acquired by the acquisition unit 232, the identification unit 233 identifies the presentation method for instructions to prompt the subject to speak. Further details will be described later.

[0032] The evaluation unit 234 is configured to evaluate various characteristics as an evaluation step. Preferably, the evaluation unit 234 evaluates the degree of cognitive decline based on the subject's voice data acquired by the acquisition unit 232. Details will be described later.

[0033] The presentation unit 235, as a presentation step, presents various information stored in the memory unit 32 in a manner that can be understood by the subject or medical personnel. For example, the presentation unit 235 may display a screen containing various information in a manner that can be viewed on a terminal such as the medical personnel terminal 3. This presents various information to various users operating the terminal such as the medical personnel terminal 3. The presentation unit 235 also controls the display of the terminal to show visual information such as screens, images, icons, and messages. The presentation unit 235 may only generate rendering information for displaying visual information on the terminal. The presentation unit 235 may also present various information to the subject or medical personnel as auditory information. Furthermore, the presentation unit 235 may present various information to the subject as tactile information that provides a tactile sensation of bumps and depressions to the fingers. Details will be described later. 3. Operation of Information Processing System 1 This section describes the flow of the information processing method executed by information processing system 1, with reference to the diagram. As shown below, the information processing method comprises the following steps. Note that the order of processing can be changed as appropriate, multiple processes may be executed simultaneously, and some processes may be omitted.

[0034] 3.1. Overview of Information Processing Figure 5 is an activity diagram showing an example of the information processing flow performed in Information Processing System 1. As an example, it describes a case in which the degree of a subject's cognitive function is examined by analyzing the subject's speech data in a medical institution. In this embodiment, it describes a case in which both auditory information IF2 instructions and visual information IF1 instructions are presented to the subject as instructions to encourage speech. In this embodiment, speech refers to uttering fixed words or phrases.

[0035] First, the reception unit 231 receives the subject's personal ID from the medical professional as a reception step (A101). Specifically, for example, the reception unit 231 receives information indicating the subject's personal ID from the medical professional terminal 3 via the communication unit 21. Furthermore, the identification unit 233 may identify the subject's personal information from the information stored in the storage unit 22 by authenticating the user based on the personal ID received in activity A101.

[0036] User authentication is not necessarily required. The reception unit 231 may accept user information of the subject instead of the subject's personal ID. The reception unit 231 may accept input of the subject's personal information, such as the subject's name, age, or gender, as user information. Furthermore, user information may be accepted from the subject themselves.

[0037] Next, the identification unit 233, as a identification step, identifies the presentation method for instructions to prompt the subject to speak (A102). Preferably, the identification unit 233 identifies the presentation method based on the subject's personal information (name, age, gender) linked to the subject's personal ID, which was received in activity A101. Specifically, for example, if age information is received as the subject's personal information, the presentation method is identified to present auditory information IF2 at a slower rate than usual for elderly subjects. Also, for example, if gender "female" information is received as the subject's personal information, the presentation method is identified to present audio data with instructions in a female voice. It is thought that instructions in a voice of the same gender can give the subject a greater sense of security.

[0038] Meanwhile, a process is performed to determine the degree of the subject's disability. As an example of a presentation step, the presentation unit 235 displays a screen for determining the degree of disability on the display unit 34 of the medical staff terminal 3 (A103). Specifically, the presentation unit 235 displays questions for determining the degree of visual or hearing impairment on the display unit 34 of the medical staff terminal 3 so that they can be presented to the subject. Further details will be described later.

[0039] Next, the reception unit 231 receives input from the subject in response to the questions displayed in activity A103 (A104). Furthermore, the acquisition unit 232 acquires information indicating the degree of the subject's disability based on the received responses. In other words, the acquisition unit 232 acquires specific user information, which is information about a specific user. Here, the specific user is a person (subject) who is a user of the information processing system 1 and corresponds to the test subject. Furthermore, the specific user information includes information indicating that the specific user has a visual impairment or a hearing impairment. Details will be described later.

[0040] Next, the identification unit 233, as an identification step, identifies the degree of the subject's disability (A105). Details will be described later.

[0041] Here, we will explain the case where the subject's disability is identified in Activity A105. As a identification step, the identification unit 233 identifies a change in the appearance of the screen displayed on the display unit 34 of the medical terminal 3 based on the degree of the subject's disability identified in Activity A105 (A106). Here, the screen displayed on the display unit 34 is a screen that displays instructions to prompt speech. For example, if the identification unit 233 identifies that the subject has color blindness, it may change the colors of the display appearance. Alternatively, for example, if the identification unit 233 identifies that the subject has age-related vision loss, it may change the brightness and size of the characters displayed on the screen. Further details will be described later.

[0042] Next, the presentation unit 235 presents the user, who is the subject, with instructions via auditory information IF2 to prompt a predetermined speech (A107). In other words, the instruction information to prompt the subject to speak includes auditory information IF2. Auditory information IF2 is audio information presented to the subject. In this embodiment, we will describe the case in which the presentation unit 235 further presents instructions to the user via visual information IF1. In this case, visual information IF1 consists of at least one of characters, figures, and photographs. Specifically, the presentation unit 235 presents the user with instructions to prompt a speech based on the presentation mode identified in activity A102 and the change screen identified in activity A106. In other words, the presentation unit 235 presents the specific user with instruction information, including instructions to prompt a predetermined speech, in a variable manner according to specific user information, including information about the subject's disability. Details will be described later.

[0043] On the other hand, if the subject's disability is not identified in Activity A105, the process proceeds to Activity A107. In this case, the presentation unit 235 presents the user with instructions to prompt speech based on the presentation pattern identified in Activity A102 (A107). In other words, the acquisition unit 232 acquires user information, which is information about the user, in Activity A101. Furthermore, the presentation unit 235 controls the content of auditory information IF2 based on the user information and presents it.

[0044] Next, the reception unit 231 receives audio data related to the speech of the user, who is the subject (A108). The audio data is generated by the sound collection device 4 based on the subject's speech.

[0045] Simultaneously with receiving the voice data for activity A108, the presentation unit 235 updates the screen displaying the speech instruction (A109). Here, the screen displays instructions that allow the subject to understand when to speak. In other words, the reception of voice data by the reception unit 231 begins before the presentation of instructions by the presentation unit 235 is completed.

[0046] Next, the evaluation unit 234 evaluates whether or not the user's cognitive function has declined, or to what extent, based on the voice data and a pre-set trained model (step A110). Here, the trained model is a model that has been pre-trained to recognize the relationship between predetermined speech features and the presence or degree of cognitive decline. Details will be described later.

[0047] The display unit 235 then presents the evaluation results to the subject and the medical staff. Further details will be described later.

[0048] In summary, the information processing system 1 according to this embodiment includes a processor 23 capable of executing a program so that the following functions are performed: The presentation unit 235 presents instructions to the user via auditory information IF2 to prompt the user to produce a predetermined speech sound. The reception unit 231 receives audio data related to the user's speech sound. The evaluation unit 234 evaluates the presence or degree of cognitive decline in the user based on the audio data and a pre-set trained model. The trained model is a model that has been pre-trained to know the relationship between predetermined speech sound features and the presence or degree of cognitive decline.

[0049] This configuration allows for the acquisition of audio data that reflects the subject's original speech state when evaluating a user's cognitive decline. Therefore, the accuracy of the assessment of cognitive decline can be improved. Furthermore, by presenting auditory information IF2 and visual information IF1 when evaluating a user's cognitive decline, the user's psychological anxiety can be reduced.

[0050] Furthermore, the information processing system 1 according to this embodiment further comprises an acquisition unit 232, a specification unit 233, and a presentation unit 235. The acquisition unit 232 acquires specific user information, which is information relating to a specific user. The specific user is a person who corresponds to a test subject among the users involved in the information processing system 1. The specific user information includes information indicating that the specific user has a visual impairment or a hearing impairment. The presentation unit 235 presents instruction information to the specific user, which includes instructions to prompt the specific user to make a predetermined utterance, in a variable manner according to the specific user information.

[0051] This configuration allows for speech support tailored to the user's level of disability. Therefore, it is possible to obtain audio data that reflects the subject's natural speech state, thereby improving the accuracy of assessments regarding cognitive decline.

[0052] 3.2. Details of Information Processing The details of the aforementioned information processing will be explained with reference to Figures 6 to 10.

[0053] (Screen 6 for checking visual impairment) Figure 6 shows an example of a visual impairment check screen 6 displayed on the display unit 34 of the medical terminal 3. In this embodiment, an example of determining the degree of color vision impairment as a visual impairment will be described. The visual impairment check screen 6 has an area 61 on which input instructions to the subject are displayed, an area 62 on which selection input is received from the subject, and a button 63.

[0054] As shown in Figure 6, area 61 displays the input instruction for the subject, "Select the box with a different color." Area 62 is the area where the subject selects and inputs their response to the input instruction. Area 62 has multiple checkboxes, including checkbox 621 and checkbox 622. As shown in Figure 6, of the multiple checkboxes, checkbox 621 is displayed in a different color from the other checkboxes. On the other hand, checkbox 622 is displayed in the same color as the other checkboxes, excluding checkbox 621.

[0055] In the example shown in Figure 6, the subject can answer the color vision deficiency check by selecting any checkbox in area 62 as a response to the instructions displayed in area 61. The acquisition unit 232 can obtain information indicating the degree of the subject's color vision deficiency by receiving the selection input from the subject. In other words, the acquisition unit 232 obtains user information by receiving input operations from the user, who is the subject.

[0056] For example, if the subject selects checkbox 621, the identification unit 233 may determine that the subject has a low probability of having color vision deficiency. On the other hand, if the subject selects any checkbox other than checkbox 621, the identification unit 233 may determine that the subject has a high probability of having color vision deficiency.

[0057] Furthermore, the identification unit 233 may determine the degree of color vision impairment of the subject based on the results of performing such checks multiple times. Button 63 is a button for displaying check content different from the check content displayed on the display unit 34. For example, the presentation unit 235 receives input from the subject to button 63, thereby transitioning the content displayed on the display unit 34 from the check screen 6 in Figure 6 to another check screen.

[0058] In this embodiment, an example of determining the degree of color vision impairment as a visual impairment has been described, but this is not limited to this. For example, the degree of visual acuity impairment may be determined as a visual impairment. Alternatively, for example, the degree of accommodation impairment may be determined as a visual impairment. Furthermore, the degree of hearing impairment may also be determined.

[0059] This embodiment describes a case where the subject is identified as having color blindness based on the selection input received from the subject. The identification unit 233 determines that the color scheme displayed on the display unit 34 of the medical terminal 3 should be changed to a color scheme that the subject can recognize. Specifically, the identification unit 233 changes the color scheme displayed on the instruction screen 7 in Figure 7, which will be described later. As a result, the subject can recognize instructions to prompt them to speak without the need for assistance from a third party.

[0060] (Instruction screen 7) Figure 7 shows an example of a voice input instruction screen 7 displayed on the display unit 34 of the medical device terminal 3. The voice input instruction screen 7 is an example of the screen displayed in activity A107 in Figure 5. The voice input instruction screen 7 has a character 71, an area 72 where instructions for the subject are displayed, and a button 73.

[0061] As shown in Figure 7, character 71 has an appearance that mimics, for example, the appearance of a female medical professional. Character 71 is used as a character to instruct the subject on the specific content of the voice input. The appearance of character 71 may be changed according to the presentation method identified in Activity A102. For example, if the subject is female, character 71 with a female appearance may be selected. Furthermore, the appearance of character 71 is not limited to a human being, but may also be an animal or a fictional character, etc.

[0062] Furthermore, as shown in Figure 7, region 72 displays the speech instruction to the subject, "Please utter 'pataka' five times in a row." Here, the instruction displayed in region 72 is simultaneously presented to the subject by auditory information IF2. In this configuration, the subject can record the voice after recognizing the intonation of the utterance. Additionally, character 71 may be displayed in a way that changes in conjunction with the auditory information IF2 presented to the subject. In other words, the presentation unit 235 further presents character 71 to the user, who is the subject, in a visible manner. Character 71 is configured to change according to the presentation state of auditory information IF2, which changes over time.

[0063] Furthermore, the visual information IF1, which changes in conjunction with the auditory information IF2, is not limited to the character 71. For example, the character information IF11 displayed in area 72 may be displayed in a manner that changes in conjunction with the auditory information IF2. That is, the visual information IF1 is configured to change according to the presentation state of the auditory information IF2, which changes over time. With this configuration, the subject can better understand the content of the test. Therefore, when testing the cognitive decline of the subject, the subject's psychological anxiety can be reduced.

[0064] Furthermore, as shown in Figure 7, button 73 is a button for initiating voice input from the subject. For example, the presentation unit 235 receives input from the subject or medical professional via button 73, and transitions the content displayed on the display unit 34 from the instruction screen 7 in Figure 7 to the recording screen 8 in Figure 8.

[0065] (Recording screen 8) Figure 8 shows an example of the recording screen 8 displayed on the display unit 34 of the medical device terminal 3. The recording screen 8 is an example of the screen displayed in activity A108 in Figure 5. The recording screen 8 includes area 81, area 82, a recording button 83, a stop button 84, area 85, area 86, and area 87.

[0066] Area 81 is the area where instructions regarding speech are displayed to the subject. As shown in Figure 8, Area 81 displays the instruction "Tap the record button and then read the following aloud." Area 82 is the area where the specific content of the subject's speech is displayed. As shown in Figure 8, Area 82 displays the speech content "pataka (5 times)."

[0067] The recording button 83 is used to start recording. The acquisition unit 232 acquires audio data of the subject's speech by receiving input from the subject or medical professional to the recording button 83. The stop button 84 is used to stop recording. The acquisition unit 232 stops acquiring audio data by receiving input from the subject or medical professional to the stop button 84.

[0068] Area 85 is an area where it is visually displayed that the subject's voice is being recorded. The display unit 235 displays "REC" in area 85 when it receives input from the subject or medical professional to the record button 83. The display unit 235 then terminates the display of "REC" in area 85 when it receives input from the subject or medical professional to the stop button 84.

[0069] Area 86 is an area where the elapsed time since the start of recording is displayed in a visible manner. As shown in Figure 8, the elapsed time may be displayed in area 86 by object IF14. Furthermore, the elapsed time may be displayed in area 86 by text information IF15 "00:02.00". In addition, the remaining time may be displayed in area 86 in a visible manner.

[0070] Area 87 is an area where information indicating the timing of the subject's speech is displayed. In other words, the presentation unit 235 presents information including instructions indicating the timing of speech to the user, who is the subject, in a manner that can be understood. In other words, the presentation unit 235 further presents guide information including information that guides the timing of speech to the specific user, who is the subject, in a manner that can be understood. As shown in Figure 8, area 87 displays information indicating the content of what the subject will say, "patakapataka". Furthermore, the character information displayed in area 87 includes black-filled character information IF12 and white-outlined character information IF13. As an example, in response to the timing of speech, the character representation indicating the content of speech changes from white-outlined character information IF13 to black-filled character information IF12. In the example in Figure 8, the next word the subject will say is "ka", and in response to the timing of speech, the white-outlined character information IF13 "ka" changes to a black-filled representation.

[0071] Here, the methods for determining the timing of speech are not limited to those described above. For example, the appearance of the characters indicating the content of the speech may change to emphasize it in response to the arrival of the timing of speech. Furthermore, the appearance of the characters may change to return to their original appearance in response to the end of the timing of speech.

[0072] Furthermore, instructions indicating the timing of speech are not limited to changes in the appearance of characters. For example, an object resembling a metronome may be displayed in area 87. The object displayed in area 87 may be presented to the user in a manner in which the pendulum needle moves from side to side in accordance with the timing of the subject's speech. Alternatively, an object resembling clapping hands may be displayed in area 87. The object displayed in area 87 may be presented to the user in a manner in which the hands clap in accordance with the timing of the subject's speech. With such methods, the subject can speak while grasping the timing and rhythm of their speech. Therefore, it is possible to obtain audio data that reflects the subject's original speech state.

[0073] Here, we will explain a method for evaluating the degree of cognitive decline in a user from audio data. Figure 9 is a schematic diagram of a neural network (NN), which is an example of a trained model. An input signal defined by various parameters is input to the first layer L1. The input signal here is the input audio data. This input signal is output from the computation nodes N_11 to N_13 of the first layer L1 to the computation nodes N_21 to N_25 of the second layer L2. At this time, the values ​​output from computation nodes N_11 to N_13 are multiplied by the weight w set between each computation node N and input to computation nodes N_21 to N_25.

[0074] Computation nodes N_21 to N_25 add up the input values ​​from computation nodes N_11 to N_13 and input this value (or a value obtained by adding a predetermined bias value to this sum) into a predetermined activation function.

[0075] The output value of the activation function is then propagated to the next node, computation node N_31. At this time, the output value is multiplied by the weight w set between computation nodes N_21~N_25 and computation node N_31, and this value is input to computation node N_31. Computation node N_31 adds up the input values ​​and outputs the sum as the output signal. Alternatively, computation node N_31 may add a bias value to the sum of the input values, input the result of adding it to the activation function, and output that value as the output signal. This outputs whether or not there is cognitive decline, or the degree thereof.

[0076] Please note that Figure 9 is for illustrative purposes only and is not an exhaustive list of possibilities.

[0077] Here, the information indicating the presence or degree of cognitive decline is not limited in format, but it is preferable to include information that indicates stages numerically as evaluation values. More preferably, the evaluation values ​​may be classified into "normal," "cognitive decline," and "risk of cognitive decline" or "risk of dementia" according to the stage. In this embodiment, the information indicating the presence or degree of cognitive decline will be described as an example in which information indicates stages numerically.

[0078] (Evaluation results display screen 9) Figure 10 shows an example of the evaluation result display screen 9 displayed on the display unit 34 of the medical professional terminal 3. The evaluation result display screen 9 includes area 91, area 92, area 93, and a re-measurement button 94.

[0079] Preferably, area 91 is the area where information on cognitive decline, which is the subject of evaluation, is displayed. As shown in Figure 9, area 91 displays "Voice evaluation results of cognitive function" as the subject of evaluation. Also preferably, area 92 is the area where the measurement result 921 is displayed. In the example in Figure 9, area 92 displays the information "Cognitive function is at a normal level" as the measurement result 921.

[0080] More preferably, area 93 is an area where a comparison result 931 is displayed, showing a comparison between the current evaluation result and the previous evaluation result, if the subject has previously received the same evaluation. For example, the comparison result 931 is evaluated based on the change in the current evaluation value 934 compared to the previous evaluation value 935, which will be described later. In other words, the acquisition unit 232 further acquires evaluation information, including the past evaluation results of a specific user who is a subject. The evaluation unit 234 further evaluates the change in the degree of cognitive decline of the specific user based on the evaluation information. In the example in Figure 9, area 93 displays the information "no change" as the comparison result 931 showing a comparison with the previous time.

[0081] Preferably, in area 93, item 932 "This Time", item 933 "Previous Time", the evaluation value for this time 934, and the evaluation value for the previous time 935 are further displayed in a way that is visible to the subject. In the example in Figure 9, for example, the evaluation unit 234 has evaluated the subject's speech and determined that it is "normal" for both the previous and current measurements.

[0082] This allows subjects to understand the extent of their own cognitive function. Furthermore, it enables quantitative evaluation of the progression of the user's cognitive decline.

[0083] [others] The following configurations may be adopted for the information processing system 1.

[0084] In the embodiment described above, the screen displaying the speech instruction was updated simultaneously with the receipt of voice data in activity A108, but this is not limited to this. The reception unit 231 may start receiving voice data after the presentation of the speech instruction is complete. In other words, the reception unit 231 may start receiving voice data after the presentation of the instruction in the presentation unit 235 is complete. With this configuration, if the subject is aware of the timing of their speech, they can concentrate on speaking. Therefore, it is possible to obtain voice data that reflects the subject's original speaking state.

[0085] In the embodiment described above, the presentation unit 235 presented instructions to the subject to prompt speech using visual information IF1 and auditory information IF2, but this is not limited to this. For example, if the subject is visually impaired, the presentation unit 235 may present instructions to the subject to prompt speech using Braille information. In other words, the instruction information, which is an instruction to prompt speech, includes tactile information to present a tactile sensation of bumps and grooves to the fingers of the specific user who is the subject. With this configuration, even subjects with visual impairments such as visual impairment can have their psychological anxiety about the test reduced. Therefore, it is possible to obtain audio data that reflects the subject's original speech state. When presenting Braille information to the subject, it is preferable that a display that supports Braille is used as the display unit 34 of the medical staff terminal 3. The presentation unit 235 converts the text information, which is the instruction information, into Braille information and presents it to the subject.

[0086] In the embodiments described above, the evaluation unit 234 evaluated the degree of cognitive decline based on audio data in which the subject uttered fixed words or phrases, but this is not limited to this. For example, the evaluation unit 234 may evaluate the degree of cognitive decline based on audio data in which the subject uttered in a conversational format. Specifically, in activity A108 in Figure 5, the reception unit 231 receives audio data from the subject in which they uttered answers to questions. For example, the presentation unit 235 may present the subject with questions such as, "What did you do yesterday?" or "What was the most memorable event this past week?" Furthermore, the presentation unit 235 may present the subject with further questions related to the content of the answers received from the subject. For example, the evaluation unit 234 may evaluate the degree of cognitive decline of the subject based on the amount of speech obtained from the received audio data. In such cases, it is preferable to use a pre-trained model that has been trained in advance on conversational exchanges.

[0087] When evaluating the degree of cognitive decline in a conversational format, the presentation unit 235 may present the subject with the timing for uttering answers to questions. In other words, the presentation unit 235 may further present guide information, including information that guides the timing of utterances, to the specific user who is the subject in a manner that is easy to understand. With this configuration, the subject can understand the timing for answering questions, thereby reducing tension and psychological anxiety regarding the test.

[0088] In the embodiment described above, the identification unit 233 determined the degree of the subject's disability through a simple check, but this is not limited to that. For example, the reception unit 231 may receive information on the degree of disability from the subject through self-declaration. The identification unit 233 may then determine the degree of disability based on the information received from the subject.

[0089] The above-described embodiment explains the case in which the subject undergoes examination at a medical institution, but this is not limited to that. For example, the subject may undergo examination using their own subject terminal. Specifically, the information processing system 1 may include a subject terminal in addition to the medical staff terminal 3 as a user terminal. The subject terminal has, for example, a hardware configuration substantially the same as that of the information processing device 2 and the medical staff terminal 3. The communication unit of the subject terminal may transmit the subject's voice data generated by the sound collection device 4 to the information processing device 2 or the medical staff terminal 3. This allows the subject to easily undergo an examination of the degree of cognitive decline at home or elsewhere, and to receive a diagnosis from a medical professional remotely.

[0090] The information processing device 2 may be in an on-premise configuration or a cloud configuration. In the case of a cloud-based information processing device 2, for example, the above functions and processing may be provided in the form of SaaS (Software as a Service) or cloud computing.

[0091] In the above embodiment, the information processing device 2 performed various storage and control functions, but instead of the information processing device 2, multiple external devices may be used. That is, various information and programs may be stored in a distributed manner across multiple external devices using blockchain technology or the like.

[0092] Another embodiment may be a program, which causes a computer to perform each step.

[0093] The product may be provided in any of the following embodiments.

[0094] (1) An information processing system comprising a processor capable of executing a program so as to perform the following steps, wherein in the acquisition step, specific user information is acquired, which is information relating to a specific user, where the specific user is a person equivalent to a test subject among the users involved in the information processing system, and the specific user information includes information that the specific user has a visual impairment or a hearing impairment; in the presentation step, instruction information including instructions to prompt the specific user to make a predetermined utterance is presented to the specific user in a variable manner according to the specific user information; in the reception step, audio data relating to the utterance from the specific user is received; and in the evaluation step, the presence or degree of cognitive decline of the specific user is evaluated based on the audio data and a pre-set trained model.

[0095] In this configuration, support for speech can be provided according to the degree of the user's disability.

[0096] (2) The information processing system described in (1) above, wherein the trained model is a model that has been pre-trained on the relationship between the content of a conversation or a predetermined speech voice feature quantity and the presence or degree of cognitive decline.

[0097] (3) An information processing system according to either (1) or (2) above, wherein the acquisition step further acquires evaluation information including the past evaluation results of the specific user, and the evaluation step further evaluates the amount of change in the degree of cognitive decline of the specific user based on the evaluation information.

[0098] This configuration allows for accurate assessment of the progression of the user's cognitive decline.

[0099] (4) In an information processing system described in any one of (1) to (3) above, the acquisition step involves acquiring the specific user information by receiving an input operation from the user.

[0100] (5) An information processing system according to any one of (1) to (4) above, wherein the presentation step further presents guide information, including information that guides the timing of speech, to the specific user in a manner that can be grasped.

[0101] This method can reduce the psychological anxiety of the subjects, thereby enabling the acquisition of high-quality audio data.

[0102] (6) An information processing system according to any one of (1) to (5) above, wherein the instruction information includes bump information for presenting a bumpy texture to the finger of the specific user.

[0103] In this configuration, even users with visual impairments can obtain high-quality audio data.

[0104] (7) In an information processing system described in any one of (1) to (6) above, the instruction information includes auditory information.

[0105] In this configuration, even users with visual impairments can obtain high-quality audio data.

[0106] (8) An information processing system according to any one of (1) to (7) above, wherein the instruction information includes visual information, and the visual information consists of at least one of characters, figures, and photographs.

[0107] In this configuration, even users with hearing impairments can obtain high-quality audio data.

[0108] (9) An information processing method comprising each step of an information processing system described in any one of (1) to (8) above.

[0109] (10) A program that causes a computer to perform any one of the steps described in (1) to (8) above. Of course, this is not always the case.

[0110] Finally, while various embodiments relating to this disclosure have been described, these are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0111] 1: Information Processing System 2: Information Processing Device 3: Medical staff terminal 4: Sound collection device 6: Check screen 7: Instruction screen 8: Recording screen 9: Evaluation results display screen 20: Communications bus 21: Communications Department 22: Storage section 23: Processor 30: Communications bus 31: Communications Department 32: Storage section 33: Processor 34:Display section 35: Input section 61 :Area 62 :Area 63: Button 71: character 72 :Area 73: Button 81 :Area 82 :Area 83: Record button 84: Stop button 85 :Area 86 :Area 87 :Area 91: area 92: area 93 :Area 94: Remeasure button 110: Acquisition section 120: Calculation Unit 130: Evaluation Department 140: Output section 160: Instruction section 231: Reception Department 232: Acquisition Department 233: Specific part 234: Evaluation Department 235: Presentation section 621: Checkbox 622: Checkbox 921: Measurement result 931 :Comparison result 932 :Item 933 :Item 934: Evaluation value 935: Evaluation value IF1: Visual information IF11: Text Information IF12: Text Information IF13: Text Information IF14: Object IF15: Text Information IF2: Auditory information L1: 1st layer L2: 2nd layer N: Computation nodes NN: Neural Network N_11: Computation node N_21: Computation node N_31: Computation node w: weight

Claims

1. An information processing system, A processor capable of executing a program is provided so that each of the following steps is performed: In the acquisition step, we acquire specific user information, which is information about a specific user, and here The aforementioned specific user is a person who corresponds to the subject among the users involved in the information processing system, The aforementioned specific user information includes information indicating that the specific user has a visual impairment or hearing impairment. In the presentation step, instruction information, including instructions to prompt the specific user to make a predetermined utterance, is presented to the specific user in a variable manner according to the specific user information. In the reception step, the audio data relating to the utterance from the specific user is received. In the evaluation step, the system evaluates whether or not there is a decline in the cognitive function of the specific user, or the degree thereof, based on the voice data and a pre-set, trained model.

2. In the information processing system described in claim 1, The aforementioned trained model is a system that has been pre-trained to recognize the relationship between the content of a conversation or specific speech features and the presence or degree of cognitive decline.

3. In the information processing system described in claim 1, In the acquisition step, further, evaluation information including the past evaluation results of the specific user is acquired. The system further evaluates, based on the evaluation information, the degree of change in the degree of cognitive decline of the specific user in the evaluation step.

4. In the information processing system described in claim 1, In the acquisition step, the system acquires the specific user information by receiving input operations from the user.

5. In the information processing system described in claim 1, The system further presents the specific user with guide information, including information that guides the timing of speech, in a manner that makes it easy to understand, in the aforementioned presentation step.

6. In the information processing system described in claim 1, The system includes, in its instruction information, information on bumps and ridges for presenting a tactile sensation of bumps and ridges to the fingers of the specific user.

7. In the information processing system described in claim 1, The aforementioned instruction information includes auditory information.

8. In the information processing system described in claim 1, The system includes visual information, wherein the visual information consists of at least one of characters, figures, and photographs.

9. Information processing method, A method comprising each step in an information processing system according to any one of claims 1 to 8.

10. It is a program, A device that causes a computer to perform each of the steps described in any one of claims 1 to 8.

Images