Information processing system, information processing method and program

Abstract

To provide an information processing system which evaluates presence / absence or a degree of cognitive function deterioration of a user.SOLUTION: In an information processing system, an information processing device 2, a medical person terminal 3, a sound collecting device 4, and a business operator terminal 5 are communicable through an electric telecommunication line. The information processing device includes a processor. In a presentation step, an instruction is presented for urging a predetermined utterance voice to a user. In a reception step, voice data is received regarding the utterance voice from the user collected by the sound collecting device. In a determination step, it is determined whether or not the sound collecting device is functioning normally based on the voice data. In an evaluation step, in the case where the sound collecting device is functioning normally, based on the voice data and a learned model which has learned beforehand a relation between the voice data, and feature quantity of predetermined utterance voice and presence / absence or a degree of cognitive function deterioration, presence / absence or a degree of cognitive function deterioration of the user is evaluated.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses a device for evaluating the degree of cognitive decline from a user's speech. The device includes an instruction unit 160 that instructs a subject to quickly pronounce a phrase in which a predetermined syllable is repeated, an acquisition unit 110 that acquires auditory information indicating a speech uttered by the subject receiving the instruction, a calculation unit 120 that calculates a feature value from the auditory information acquired by the acquisition unit 110, an evaluation unit 130 that evaluates the cognitive function of the subject from the feature value calculated by the calculation unit 120, and an output unit 140 that outputs the evaluation result evaluated by the evaluation unit 130. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-083903 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, there is still room for improvement in the functions of the above-mentioned known techniques. In particular, there is a need for techniques that can further assist in evaluating the degree of cognitive decline with high accuracy.

[0005] In consideration of the above circumstances, the present disclosure provides an information processing system and the like that is capable of evaluating the degree of cognitive decline with high accuracy. [Means for solving the problem]

[0006] According to one aspect of the present invention, an information processing system is provided. The information processing system includes a processor capable of executing a program to perform the following steps: In the presenting step, instructions are presented to the user to prompt the user to produce a predetermined speech sound; In the receiving step, voice data relating to the speech sound from the user collected by the sound collection device is received; In the determining step, it is determined whether or not the sound collection device is functioning normally based on the voice data; In the evaluating step, when the sound collection device is functioning normally, the presence or absence or degree of cognitive function decline of the user is evaluated based on the voice data and a previously set trained model. The trained model is a model that has previously trained the relationship between the feature amount of a predetermined speech sound and the presence or absence or degree of cognitive function decline.

[0007] According to the present disclosure, it is possible to provide an information processing system etc. capable of evaluating the degree of cognitive decline with high accuracy. [Brief description of the drawings]

[0008] [Figure 1] 1 is a configuration diagram illustrating an information processing system 1. [Diagram 2] 2 is a block diagram showing a hardware configuration of an information processing device 2. FIG. [Diagram 3] FIG. 2 is a block diagram showing the hardware configuration of a user terminal. [Figure 4] 2 is a functional block diagram showing functions of an information processing device 2. FIG. [Diagram 5] 2 is an activity diagram showing an example of the flow of information processing executed in the information processing system 1. FIG. [Figure 6] 1 shows an example of a visual impairment check screen 6 displayed on the display unit 34 of the medical professional terminal 3. [Figure 7] 1 shows an example of a voice input instruction screen 7 displayed on the display unit 34 of the medical professional terminal 3. [Figure 8] 1 shows an example of a recording screen 8 displayed on the display unit 34 of the medical professional terminal 3. [Figure 9]FIG. 1 is a schematic diagram of a neural network NN, which is an example of a trained model. [Figure 10] 13 is an example of an evaluation result display screen 9 displayed on the display unit 34 of the medical professional terminal 3. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings. Various characteristic features shown in the following embodiments can be combined with each other.

[0010] Incidentally, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable recording medium, or may be provided so as to be downloadable from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0011] In this embodiment, the term "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, this embodiment handles various information, which may be represented by, for example, physical values ​​of signal values ​​representing voltage and current, high and low signal values ​​as a binary bit set consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculation may be performed on a circuit in the broad sense.

[0012] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. In other words, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0013] 1. Hardware Configuration This section describes the hardware configuration.

[0014] <Information Processing System 1> FIG. 1 is a configuration diagram showing an information processing system 1. The information processing system 1 includes an information processing device 2, a medical practitioner terminal 3, a sound collection device 4, and a business operator terminal 5. The information processing device 2, the medical practitioner terminal 3, the sound collection device 4, and the business operator terminal 5 are configured to be able to communicate with each other via a telecommunications line. In one embodiment, the information processing system 1 is made up of one or more devices or components. For example, if the information processing system 1 is made up of only the information processing device 2, the information processing system 1 can be the information processing device 2. These components will be described below.

[0015] <Information processing device 2> 2 is a block diagram showing a hardware configuration of the information processing device 2. The information processing device 2 includes a communication bus 20, a communication unit 21, a storage unit 22, and a processor 23. The communication unit 21, the storage unit 22, and the processor 23 are electrically connected via the communication bus 20 inside the information processing device 2.

[0016] <Communications Division 21> The communication unit 21 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), wired LAN network communication, etc., but may also include wireless LAN network communication, mobile communication such as 3G / LTE / 5G, BLUETOOTH (registered trademark) communication, etc. as necessary. In other words, it is more preferable to implement it as a collection of multiple communication means. In other words, the information processing device 2 may communicate various information from the outside via the communication unit 21 and the network.

[0017] <Storage section 22> The storage unit 22 stores various information defined by the above description. This can be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs and the like related to the information processing device 2 executed by the processor 23, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to the program calculations. The storage unit 22 stores various programs, variables, etc. related to the information processing device 2 executed by the processor 23.

[0018] <Processor 23> The processor 23 processes and controls the overall operations related to the information processing device 2. The processor 23 is, for example, a central processing unit (CPU) not shown. The processor 23 realizes various functions related to the information processing device 2 by reading out a predetermined program stored in the storage unit 22. That is, information processing by software stored in the storage unit 22 can be specifically realized by the processor 23, which is an example of hardware, and executed as each functional unit included in the processor 23. These will be described in more detail in the next section. The processor 23 is not limited to being single, and may be implemented with multiple processors 23 for each function. Also, a combination of these may be used.

[0019] <User device> The user terminal is a terminal possessed by a user. In this embodiment, a case will be described in which the user terminal is a terminal 3 (hereinafter referred to as medical professional terminal 3) operated by a user who is a medical professional such as a doctor (hereinafter referred to as a medical professional). The user terminal may further include a terminal (hereinafter referred to as a subject terminal) operated by a user who is a patient whose cognitive function is to be evaluated (hereinafter referred to as a subject terminal). Details will be described later. The user terminal may be in any form, such as a smartphone, tablet terminal, computer, or any other device that can access the information processing device 2 via an electric communication line.

[0020] 3 is a block diagram showing the hardware configuration of a user terminal. The following description will be given taking the medical professional terminal 3 as an example. The medical professional terminal 3 includes a communication bus 30, a communication unit 31, a storage unit 32, a processor 33, a display unit 34, and an input unit 35. The communication unit 31, the storage unit 32, the processor 33, the display unit 34, and the input unit 35 are electrically connected via the communication bus 30 inside the medical professional terminal 3. The description of the communication unit 31, the storage unit 32, and the processor 33 will be omitted as they are the same as the descriptions of the respective units in the information processing device 2.

[0021] <Display section 34> The display unit 34 displays a screen of a graphical user interface (GUI) that can be operated by the user. The display unit 34 may be included in the housing of the medical professional terminal 3, or may be attached externally. Specifically, the display unit 34 may be implemented as a display device such as a CRT display, a liquid crystal display, an organic EL display, or a plasma display. These display devices are preferably implemented by selectively using them according to the type of the medical professional terminal 3.

[0022] <Input section 35> The input unit 35 accepts an operation input made by a medical professional or a subject. The operation input is transferred as a command signal to the processor 33 via the communication bus 30. The processor 33 can execute a predetermined control or calculation based on the transferred command signal as necessary. The input unit 35 may be included in the housing of the medical professional terminal 3, or may be externally attached. For example, the input unit 35 may be implemented as a touch panel integrated with the display unit 34. When the input unit 35 is implemented as a touch panel, the medical professional or a user can input a tap operation, a swipe operation, or the like to the input unit 35. As the input unit 35, a switch button, a mouse, a QWERTY keyboard, or the like can be adopted instead of a touch panel.

[0023] <Sound collection device 4> The sound collector 4 is a so-called microphone that is configured to convert external sounds into signals. The sound collector 4 may be provided by directly connecting the sound collector 4 to the information processing device 2, but is provided in or connected to the medical professional terminal 3.

[0024] The sound collection device 4 is configured to generate voice data by collecting the user's speech. The voice data is temporarily stored in a memory in the user terminal, and does not have to be stored in a non-volatile manner in the storage unit 32. The voice data generated by the sound collection device 4 is configured to be transferable to the information processing device 2 via a network.

[0025] The sound collection device 4 collects, but is not limited to, at least sounds in the human audible range, sounds with frequencies between 20 Hz and 20,000 Hz, and converts them into electrical signals. The sounds may be recorded in monaural or stereo. The sampling rate for digitally processing the sound data may be, for example, 48000 Hz, 44100 Hz, 32000 Hz, 22050 Hz, 16000 Hz, 11025 Hz, 11000 Hz, 8000 Hz, etc. The sampling rate may be within any of the ranges of values ​​exemplified here. By increasing the sampling rate, the temporal timing of the sound can be discretized more precisely, and the accuracy of voice recognition can be improved.

[0026] The data collected by the sound collection device 4 may be appropriately compressed by the processor 33 of the medical professional terminal 3, and the compression format may be any of MP3, AAC, WMA, Vorbis, AC3, MP2, FLAC, TAK, etc. By compression, it is possible to reduce communication traffic caused by data transfer from the user terminal to the information processing device 2.

[0027] <Operator terminal 5> The business operator terminal 5 is a terminal owned by a business operator who manufactures or maintains the sound collection device 4. Like the medical professional terminal 3, this business operator terminal 5 may have a communication bus, a communication unit, a memory unit, a processor, a display unit, and an input unit, but for the sake of simplicity, a description of each will be omitted here.

[0028] 2. Functional configuration In this section, the functional configuration will be described. Fig. 4 is a functional block diagram showing the functions of the information processing device 2. As described above, information processing by software stored in the storage unit 22 can be specifically realized by hardware (specifically, the processor 23) and executed as each functional unit included in the processor 23. In other words, the information processing system 1 includes each functional unit in a program.

[0029] Specifically, the processor 23 includes, as its functional units, a reception unit 231, an acquisition unit 232, an identification unit 233, a determination unit 234, a warning unit 235, an evaluation unit 236, and a presentation unit 237.

[0030] The receiving unit 231 is configured to receive various information as a receiving step. Specifically, the receiving unit 231 is configured to receive information via the communication unit 21 or the storage unit 22 and to be able to read the information into the working memory. Preferably, the receiving unit 231 receives input of voice data related to a speech voice from the subject via the sound collection device 4.

[0031] The acquiring unit 232 is configured to acquire various pieces of information as an acquiring step. Specifically, the acquiring unit 232 acquires user information, which is information related to the subject.

[0032] The specifying unit 233 is configured to specify various pieces of information as a specifying step. Specifically, the specifying unit 233 specifies a presentation mode related to an instruction for encouraging the subject to speak, based on the user information acquired by the acquiring unit 232. Details will be described later.

[0033] As a determination step, the determination unit 234 determines whether various devices related to the information processing system 1 are functioning normally. More specifically, the determination unit 234 determines whether the sound collection device 4 is functioning normally based on the voice data accepted by the acceptance unit 231. Details will be described later.

[0034] The warning unit 235 issues various warnings as a warning step. More specifically, when the sound collection device 4 is not functioning normally, the warning unit 235 issues a warning that the sound collection device 4 is not functioning normally. The destination of this warning can be set in various ways, but typically, it is someone related to the information processing system 1. The details of this warning will be described later.

[0035] The evaluation unit 236 is configured to evaluate various characteristics as an evaluation step. Preferably, the evaluation unit 236 evaluates the degree of cognitive decline based on the voice data of the subject acquired by the reception unit 231. Details will be described later.

[0036] As a presentation step, the presentation unit 237 presents various information stored in the storage unit 32 in a form that can be understood by the subject or medical practitioner. For example, the presentation unit 237 may display a screen or the like including various information in a form that can be viewed on a terminal such as the medical practitioner terminal 3. This allows various information to be presented to various users who operate the terminal such as the medical practitioner terminal 3. The presentation unit 237 also controls visual information such as a screen, an image, an icon, a message, etc. to be displayed on the display of the terminal. The presentation unit 237 may generate only rendering information for displaying visual information on the terminal. The presentation unit 237 may also present various information to the subject or medical practitioner as auditory information. Furthermore, the presentation unit 237 may present various information to the subject as unevenness information that presents an uneven tactile sensation to a finger. Details will be described later.

[0037] 3. Operation of Information Processing System 1 In this section, the flow of the information processing method executed by the information processing system 1 will be described with reference to the drawings. As shown below, the information processing method includes the following steps. The order of the processes can be changed as appropriate, multiple processes may be executed simultaneously, or some processes may be omitted. Note that, as a presentation step, a process of presenting an instruction to the subject (user) by auditory information to prompt the subject (user) to produce a predetermined speech sound is shown below, but this instruction may be presented by information other than auditory information.

[0038] 3.1. Overview of information processing FIG. 5 is an activity diagram showing an example of the flow of information processing executed in the information processing system 1. As an example, a case will be described in which the degree of cognitive function of a subject is examined in a medical institution by analyzing the subject's speech voice data. In this embodiment, a case will be described in which both instructions in the form of auditory information IF2 and instructions in the form of visual information IF1 are presented to the subject as instructions to encourage the subject to speak. Moreover, speaking in this embodiment refers to speaking a fixed word or phrase.

[0039] First, as an acquisition step, the acquisition unit 232 accepts and acquires the input of the subject's personal ID from a medical professional (A101). Specifically, for example, the acquisition unit 232 accepts and acquires information indicating the subject's personal ID from the medical professional terminal 3 via the communication unit 21. Furthermore, the identification unit 233 may identify the personal information of the subject from the information stored in the storage unit 22 by authenticating the user based on the personal ID accepted in the activity A101.

[0040] Authentication of the user does not necessarily have to be performed. The acquisition unit 232 may accept and acquire user information of the subject instead of the subject's personal ID. The acquisition unit 232 may accept and acquire personal information of the subject, such as the subject's name, age, or sex, as the user information. The user information may be input from the subject himself / herself.

[0041] Next, as a specification step, the specification unit 233 specifies a presentation mode of instructions for encouraging the subject to speak (A102). Preferably, the specification unit 233 specifies the presentation mode based on the subject's personal information (name, age, sex) associated with the subject's personal ID acquired in activity A101. Specifically, for example, when age information is acquired as the subject's personal information, the specification unit 233 specifies the presentation mode so that the auditory information IF2 is presented at a slower speed than normal for older subjects. Also, for example, when gender information "female" is acquired as the subject's personal information, the specification unit 233 specifies the presentation mode so that audio data giving instructions in a female voice is presented. In particular, it is considered that instructions given by a voice of the same sex can give the subject a sense of security.

[0042] Meanwhile, a process of identifying the degree of the disability of the subject is executed. As an example of a presentation step, the presentation unit 237 displays a screen for determining the degree of disability on the display unit 34 of the medical practitioner terminal 3 (A103). Specifically, the presentation unit 237 displays a question for identifying the degree of visual impairment or hearing impairment on the display unit 34 of the medical practitioner terminal 3 so that the question can be presented to the subject. Details will be described later.

[0043] Next, the receiving unit 231 receives an input of an answer from the subject to the question displayed in activity A103 (A104). Furthermore, the acquiring unit 232 acquires information indicating the degree of the subject's disability based on the received answer. In other words, the acquiring unit 232 acquires specific user information that is information about a specific user. Here, the specific user is a person (subject) who corresponds to a subject among users involved in the information processing system 1. Furthermore, the specific user information includes information that the specific user has a visual impairment or a hearing impairment. Details will be described later.

[0044] Next, the identification unit 233 identifies the degree of the disability of the subject (A105) as an identification step, as will be described in detail later.

[0045] Here, a case where the subject's disability is identified in activity A105 will be described. As an identification step, the identification unit 233 identifies a change in the mode of the screen to be displayed on the display unit 34 of the medical professional terminal 3 based on the degree of the subject's disability identified in activity A105 (A106). Here, the screen to be displayed on the display unit 34 is a screen that displays instructions to encourage speaking. For example, when the subject is identified as having color vision deficiency, the identification unit 233 may change the color of the display mode. Also, for example, when the subject is identified as having age-related deterioration of eyesight, the identification unit 233 may change the brightness and size of characters to be displayed on the screen. Details will be described later.

[0046] Next, the presentation unit 237 presents an instruction to the user, who is the subject, to prompt the user to make a predetermined speech sound by the auditory information IF2 (A107). That is, the instruction information to prompt the subject to speak may include the auditory information IF2. The auditory information IF2 is audio information presented to the subject. In this embodiment, the presentation unit 237 may present the instruction to the user by the visual information IF1. In this case, the visual information IF1 is composed of at least one of a character, a figure, and a photo. Hereinafter, the explanation will be continued by giving an example in which the instruction to prompt the subject to perform an action includes both the auditory information IF2 and the visual information IF1. Specifically, the presentation unit 237 presents an instruction to prompt the user to make a speech sound based on the presentation mode specified in the activity A102 and the change screen specified in the activity A106. In other words, the presentation unit 237 presents the instruction information including an instruction to prompt the specific user to make a predetermined speech to the specific user, which is variable according to the specific user information including information regarding the subject's disability. Details will be provided below.

[0047] On the other hand, if the subject's disability is not identified in activity A105, the process proceeds to activity A107. In this case, the presenting unit 237 presents to the user an instruction to encourage speech based on the presentation mode identified in activity A102 (A107). In other words, the acquiring unit 232 acquires user information, which is information about the user, in activity A101. Furthermore, the presenting unit 237 controls the content of the auditory information IF2 based on the user information and presents it.

[0048] Next, the receiving unit 231 receives voice data related to a speech voice from a user who is a subject (A108). The voice data is data generated by the sound collection device 4 based on the speech voice of the subject.

[0049] At the same time as receiving the voice data of activity A108, the presenter 237 updates the screen on which the speaking instruction is displayed (A109). Here, the screen is a screen on which an instruction is displayed to enable the subject to understand the timing at which he or she should speak. In other words, the reception of the voice data by the reception unit 231 starts before the presentation of the instruction by the presenter 237 is completed.

[0050] When the voice data is received as described above, the determination unit 234 determines whether or not the sound collection device 4 is functioning normally based on the voice data (A110). The method of this determination will be described in detail later.

[0051] When it is determined in the activity A110 that the sound collection device 4 is functioning normally, the evaluation unit 236 evaluates the presence or absence or the degree of cognitive decline of the user based on the voice data and a previously set trained model (A111). Here, the trained model is a model that has previously trained the relationship between the feature amount of a predetermined speech voice and the presence or absence or the degree of cognitive decline. Details will be described later.

[0052] Then, the presentation unit 237 presents the evaluation results to the subject and the medical staff (A112), as will be described in detail later.

[0053] If it is determined in activity A110 that the sound collection device 4 is not functioning normally, a predetermined warning is issued by the warning unit 235 (A113). The contents of this warning will be described in detail later.

[0054] In summary, the information processing system 1 according to this embodiment includes a processor 23 capable of executing a program to perform the following functions. The presentation unit 237 presents to the user an instruction to prompt the user to produce a predetermined speech sound. The reception unit 231 receives voice data related to the speech sound from the user collected by the sound collection device 4. The determination unit 234 determines whether the sound collection device 4 is functioning normally based on the voice data. When the sound collection device 4 is functioning normally, the evaluation unit 236 evaluates the presence or absence or degree of cognitive function decline of the user based on the voice data and a previously set trained model. The trained model is a model that has previously trained the relationship between the feature amount of a predetermined speech sound and the presence or absence or degree of cognitive function decline.

[0055] According to this embodiment, the evaluation can be performed on the condition that the sound collection device 4 is functioning reliably. Therefore, the accuracy of the evaluation regarding the cognitive decline can be improved. In addition, when evaluating a user, time loss due to a breakdown of the sound collection device 4 can be eliminated, and as a result, the evaluation of the user can be performed smoothly.

[0056] 3.2 Details of data processing The above-mentioned information processing will be described in detail with reference to FIGS.

[0057] (Visual impairment check screen 6) 6 shows an example of a visual impairment check screen 6 displayed on the display unit 34 of the medical professional terminal 3. In this embodiment, an example of determining the degree of color vision deficiency as a visual impairment will be described. The visual impairment check screen 6 has an area 61 in which input instructions for the subject are displayed, an area 62 in which selection input is received from the subject, and a button 63.

[0058] As shown in FIG. 6, area 61 displays an input instruction to the subject, "Please select a box that is a different color." Area 62 is an area that accepts a selective input of an answer to the input instruction from the subject. Area 62 has multiple checkboxes including checkbox 621 and checkbox 622. As shown in FIG. 6, among the multiple checkboxes, checkbox 621 is displayed in a color different from the other checkboxes. Meanwhile, checkbox 622 is displayed in the same color as the other checkboxes except for checkbox 621.

[0059] 6, the subject can answer the color vision deficiency check by selecting any check box in area 62 as a response to the instruction displayed in area 61. The acquisition unit 232 can acquire information indicating the degree of the subject's color vision deficiency by receiving a check box selection input from the subject. In other words, the acquisition unit 232 acquires user information by receiving an input operation from the user who is the subject.

[0060] As an example, when a selection input of the check box 621 is received from the subject, the identifying unit 233 may determine that the subject is unlikely to have a color vision deficiency. On the other hand, when a selection input of a check box other than the check box 621 is received from the subject, the identifying unit 233 may determine that the subject is likely to have a color vision deficiency.

[0061] Furthermore, the identification unit 233 may determine the degree of color vision deficiency of the subject based on the results of performing such a check multiple times. The button 63 is a button for displaying check content different from the check content displayed on the display unit 34. For example, the presentation unit 237 receives an input to the button 63 from the subject, thereby transitioning the content displayed on the display unit 34 from the check screen 6 in FIG. 6 to another check screen.

[0062] In the present embodiment, an example has been described in which the degree of color vision impairment is determined as a visual impairment, but this is not limiting. For example, the degree of visual impairment may be determined as a visual impairment. Also, for example, the degree of accommodation disorder may be determined as a visual impairment. Furthermore, the degree of hearing impairment may be determined.

[0063] In this embodiment, a case will be described in which the subject is identified as having color vision deficiency based on the result of a selection input received from the subject. The identification unit 233 determines that the color scheme displayed on the display unit 34 of the medical professional terminal 3 should be changed to a color scheme that can be recognized by the subject. Specifically, the identification unit 233 changes the color scheme displayed on the instruction screen 7 in Fig. 7, which will be described later. This allows the subject to recognize the instruction to encourage speech without the need for assistance from a third party.

[0064] (Instruction screen 7) Fig. 7 shows an example of a voice input instruction screen 7 displayed on the display unit 34 of the medical professional terminal 3. The voice input instruction screen 7 is an example of a screen displayed in activity A107 of Fig. 5. The voice input instruction screen 7 has a character 71, an area 72 in which instructions for the subject are displayed, and a button 73.

[0065] As shown in FIG. 7, the character 71 has an appearance display that imitates, for example, the appearance of a female medical professional. The character 71 is used as a character that instructs the subject on the specific content of the voice input. The appearance of the character 71 may be changed according to the presentation mode specified in the activity A102. For example, if the subject is a woman, the character 71 having a female appearance display may be selected. Furthermore, the appearance of the character 71 is not limited to a person, and may be an animal, a fictional character, or the like.

[0066] Also, as shown in FIG. 7, the area 72 displays a speech instruction to the subject, "Please say 'pataka' five times in a row." Here, the instruction displayed in the area 72 is also simultaneously presented to the subject by the auditory information IF2. In this manner, the subject can record the voice after recognizing the intonation of the speech. Also, the character 71 may be displayed so as to change in conjunction with the auditory information IF2 presented to the subject. In other words, the presentation unit 237 further presents the character 71 to the user, who is the subject, in a visually recognizable manner. The character 71 is configured to change according to the presentation state of the auditory information IF2, which changes along a time series.

[0067] Furthermore, the visual information IF1 that changes in conjunction with the auditory information IF2 is not limited to the character 71. As an example, the text information IF11 displayed in the area 72 may be displayed in a manner that changes in conjunction with the auditory information IF2. That is, the visual information IF1 is configured to change according to the presentation state of the auditory information IF2 that changes along a time series. According to such an embodiment, the subject can further grasp the contents of the test. Therefore, the psychological anxiety of the subject can be reduced when testing the cognitive decline of the subject.

[0068] 7, the button 73 is a button for starting the subject's voice input. For example, the presentation unit 237 receives an input to the button 73 from the subject or the medical professional, and transitions the content displayed on the display unit 34 from the instruction screen 7 in FIG. 7 to the recording screen 8 in FIG. 8.

[0069] (Recording screen 8) Fig. 8 shows an example of a recording screen 8 displayed on the display unit 34 of the medical professional terminal 3. The recording screen 8 is an example of a screen displayed in activity A108 of Fig. 5. The recording screen 8 includes an area 81, an area 82, a record button 83, a stop button 84, an area 85, an area 86, and an area 87.

[0070] Area 81 is an area where instructions regarding speech are displayed to the subject. As shown in FIG. 8, area 81 displays the instruction "After tapping the record button, please read the following out loud." Area 82 is an area where the specific speech content to be spoken by the subject is displayed. As shown in FIG. 8, area 82 displays the speech content "Pataka (5 times)."

[0071] The record button 83 is a button for starting recording. The acquisition unit 232 acquires voice data of the subject's speech by receiving an input to the record button 83 from the subject or medical professional. The stop button 84 is a button for stopping recording. The acquisition unit 232 stops acquiring the voice data by receiving an input to the stop button 84 from the subject or medical professional.

[0072] The area 85 is an area where it is visually displayed that the subject's voice is being recorded. The presentation unit 237 displays "REC" in the area 85 by receiving an input to the record button 83 from the subject or the medical professional. The presentation unit 237 also stops displaying "REC" in the area 85 by receiving an input to the stop button 84 from the subject or the medical professional.

[0073] Area 86 is an area where the elapsed time from the start of recording is visibly displayed. As shown in Fig. 8, area 86 may display the elapsed time by object IF14. Area 86 may also display the elapsed time by text information IF15 "00:02.00". Area 86 may also visibly display the remaining time.

[0074] The area 87 is an area where information indicating the subject's speech timing is displayed. In other words, the presentation unit 237 further presents information including an instruction indicating the speech timing to the user who is the subject in a graspable manner. In further other words, the presentation unit 237 further presents guide information including information guiding the speech timing to the specific user who is the subject in a graspable manner. As shown in FIG. 8, the area 87 displays information "patakapataka" indicating the content to be spoken by the subject. Furthermore, the character information displayed in the area 87 includes blackened character information IF12 and whitened character information IF13. As an example, the character form indicating the speech content is changed from the whitened character information IF13 to the blackened character information IF12 in response to the arrival of the speech timing. In the example of FIG. 8, the word to be spoken by the subject next is "ka", and the whitened character information IF13 "ka" changes to a blackened character form in response to the arrival of the speech timing.

[0075] Here, the manner in which the speech timing can be grasped is not limited to this. For example, the manner of characters indicating the speech content may change so as to be emphasized when the speech timing arrives. Furthermore, the manner of characters may change so as to return to the original manner when the speech timing ends.

[0076] Furthermore, the instruction indicating the speaking timing is not limited to a change in the appearance of the characters. For example, an object having the appearance of a metronome may be displayed in the area 87. The object displayed in the area 87 may be presented to the user in a manner in which a pendulum needle moves left and right in accordance with the speaking timing of the subject. Also, for example, an object having the appearance of clapping hands may be displayed in the area 87. The object displayed in the area 87 may be presented to the user in a manner in which hands are clapping in accordance with the speaking timing of the subject. According to such an embodiment, the subject can speak while grasping the speaking timing and rhythm. Therefore, it is possible to obtain voice data that reflects the subject's original speaking state.

[0077] After receiving the voice data as described above, the determination unit 234 determines whether the sound collection device 4 is functioning normally based on the voice data. That is, the determination unit 234 may determine that the sound collection device 4 is not functioning normally when the received voice data does not include the subject's speech or there is an unnatural interruption in the subject's speech. This determination may be made based on not only the voice data of the subject's speech but also on voice data related to the environment in which the sound collection device 4 collects sound (typically, voice data related to environmental sounds around the place where the subject spoke) to determine whether the sound collection device 4 is functioning normally. For example, if the voice data does not include the subject's speech but the environmental sounds themselves are collected normally, it is assumed that the subject's speech is not sufficient, so in this case, it is not necessarily necessary to determine that there is an abnormality in the sound collection device 4.

[0078] Note that such a determination regarding the sound collection device 4 may be made based on, for example, preset criterion information. That is, in this embodiment, the behavior of the above-mentioned speech sound data and environmental sound sound data when the sound collection device 4 is functioning normally (or when there is an abnormality in the sound collection device 4) can be stored in advance as criterion information. Also, in this embodiment, by matching the received sound data with the criterion information, it can be determined whether the sound collection device 4 is functioning normally or not.

[0079] Furthermore, the determination unit 234 may determine, from the voice data, the cause of the failure of the sound collection device 4. For example, when there is an unnatural pause in the subject's speech, the determination unit 234 may determine that there is a poor connection between the sound collection device 4 and the communication unit 31.

[0080] When the judgment unit 234 determines that the sound collection device 4 is functioning normally as described above, the evaluation unit 236 evaluates the presence or absence or degree of cognitive function decline of the user based on the voice data and a pre-set trained model.

[0081] Here, a method for evaluating the degree of cognitive decline of a user from voice data will be described. FIG. 9 is a schematic diagram of a neural network NN, which is an example of a trained model. An input signal defined by various parameters is input to the first layer L1. The input signal here is input voice data. The input signal is output from the computation nodes N_11 to N_13 of the first layer L1 to the computation nodes N_21 to N_25 of the second layer L2, respectively. At this time, the value output from the computation nodes N_11 to N_13 is multiplied by the weight w set between each computation node N, and the resulting value is input to the computation nodes N_21 to N_25.

[0082] The calculation nodes N_21 to N_25 add up the input values ​​from the calculation nodes N_11 to N_13, and input this value (or a value obtained by adding a predetermined bias value to this value) to a predetermined activation function.

[0083] Then, the output value of the activation function is propagated to the next node, the computation node N_31. At this time, a value obtained by multiplying the weight w set between the computation nodes N_21 to N_25 and the computation node N_31 by the output value is input to the computation node N_31. The computation node N_31 adds up the input values ​​and outputs the sum as an output signal. At this time, the computation node N_31 may add up the input values, input the sum to a bias value, and input the resultant value to the activation function and output the output value as an output signal. As a result, the presence or absence or degree of cognitive decline is output.

[0084] It should be noted that what is shown in FIG. 9 is for illustrative purposes only and is not limiting.

[0085] Here, the information indicating the presence or absence or the degree of cognitive decline is not limited in format, but preferably includes information indicating a stage in numbers as an evaluation value. More preferably, the evaluation value may be classified into "normal", "cognitive decline", "risk of cognitive decline", or "risk of dementia" according to the stage. In this embodiment, an example will be described in which the information indicating the presence or absence or the degree of cognitive decline is information indicating a stage in numbers.

[0086] (Evaluation result display screen 9) 10 is an example of the evaluation result display screen 9 displayed on the display unit 34 of the medical professional terminal 3. The evaluation result display screen 9 includes an area 91, an area 92, an area 93, and a remeasurement button 94.

[0087] Preferably, area 91 is an area where information on cognitive decline, which is the subject of evaluation, is displayed. As shown in Fig. 9, area 91 displays "audio evaluation result of cognitive function" as the subject of evaluation. Also preferably, area 92 is an area where measurement result 921 is displayed. In the example of Fig. 9, area 92 displays information that "cognitive function is at a normal level" as measurement result 921.

[0088] More preferably, the area 93 is an area in which a comparison result 931 showing a comparison between the current evaluation result and the previous evaluation result is displayed when the subject has previously received the same evaluation. For example, the comparison result 931 is evaluated based on the amount of change in the current evaluation value 934 relative to the previous evaluation value 935 described later. In other words, the acquisition unit 232 further acquires evaluation information including the past evaluation results of the specific user who is the subject. The evaluation unit 236 further evaluates the amount of change in the degree of cognitive function decline of the specific user based on the evaluation information. In the example of FIG. 9, the area 93 displays information "no change" as the comparison result 931 showing the comparison with the previous time.

[0089] Preferably, in the area 93, an item 932 “this time”, an item 933 “last time”, a current evaluation value 934, and a previous evaluation value 935 are further displayed so as to be visible to the subject. In the example of Fig. 9, for example, the evaluation unit 236 evaluates the speech of the subject in both the previous measurement and the current measurement and determines that the speech is “normal”.

[0090] This allows the subject to understand the level of their own cognitive function, and further allows the progress of the user's cognitive decline to be quantitatively evaluated.

[0091] The evaluation unit 236 may perform calculation (conversion) so that the degree of cognitive function decline of the user matches an evaluation value (score) of a predetermined index from the received speech voice data, etc. Here, the evaluation value (score) in this embodiment does not necessarily refer to a single numerical value, but may specify a numerical range (numerical range) such as "10 to 12". Furthermore, the evaluation value (score) in this embodiment may specify an upper limit value or a lower limit value such as "20 or more" or "less than 15". On the other hand, the "predetermined index" may be exemplified by various indexes (scales) capable of evaluating the degree of cognitive function decline. Examples include the Montreal Cognitive Assessment (MoCA; the Japanese version may be referred to as MoCA-J) scale, the Behavior Rating Inventory of Executive Function (BRIEF) scale, the Mini-Menta State Examination (MMSE) scale, the Hasegawa Brief Intelligence Scale, and the Wechsler Memory Scale, but are not limited thereto. In one embodiment, based on the above-mentioned evaluation value (score), a display may be further displayed that classifies the patient into "healthy (normal)", "cognitive decline", "risk of cognitive decline", or "risk of dementia" according to the stage.

[0092] In addition, if the determination unit 234 determines that there is an abnormality in the sound collection device 4, a predetermined warning may be issued. This warning may be, for example, a warning message displayed on a display, a warning sound generated from a predetermined device, or a combination of these, but is not limited to these, and various warning modes may be adopted. That is, the warning in this embodiment may be for other five senses information such as auditory information, tactile information, etc., and may be a combination of two or more of these.

[0093] As an example, the warning step executed by the warning unit 235 is executed by displaying on a predetermined screen that the sound collection device 4 is not functioning normally. That is, the warning unit 235 may be configured to display on the medical professional terminal 3 or the like that the sound collection device 4 is not functioning normally (for example, that a poor connection between the sound collection device 4 and the communication unit 31 is suspected). The display unit 34 is exemplified as a display destination, but is not limited thereto.

[0094] The warning step executed by the warning unit 235 may be executed on a terminal owned by a business that manufactures or maintains the sound collection device 4. That is, the warning issued by the warning unit 235 may be issued to the business terminal 5 owned by the business that manufactures or maintains the sound collection device 4. By employing such an embodiment, the sound collection device 4 can be promptly maintained.

[0095] [others] The information processing system 1 may have the following configuration.

[0096] In the above embodiment, the screen on which the speech instruction is displayed is updated at the same time as the speech data is received by the activity A108, but this is not limited thereto. The reception unit 231 may start receiving the speech data after the presentation of the speech instruction is completed. In other words, the reception unit 231 may start receiving the speech data after the presentation of the instruction is completed by the presentation unit 237. According to this aspect, if the subject knows the timing of speaking, the subject can concentrate on speaking. Therefore, it is possible to obtain speech data that reflects the subject's original speaking state.

[0097] In the above embodiment, the presenting unit 237 presents the instruction to encourage the subject to speak using the visual information IF1 and the audio information IF2, but this is not limited thereto. For example, if the subject has weak eyesight, the presenting unit 237 may present the instruction to encourage the subject to speak using Braille information. In other words, the instruction information, which is an instruction to encourage the subject to speak, includes uneven information for presenting an uneven tactile sensation to the finger of the specific user who is the subject. According to this aspect, even if the subject has a visual impairment such as weak eyesight, psychological anxiety about the examination can be reduced. Therefore, it is possible to obtain voice data that reflects the subject's original speech state. In addition, when presenting Braille information to the subject, it is preferable to use a display compatible with Braille as the display unit 34 of the medical professional terminal 3. The presenting unit 237 converts the character information, which is the instruction information, into Braille information and presents it to the subject.

[0098] In the above embodiment, the evaluation unit 236 evaluates the degree of cognitive decline based on the voice data in which the subject speaks about fixed words or phrases, but this is not limited to the above. For example, the evaluation unit 236 may evaluate the degree of cognitive decline based on the voice data in which the subject speaks in a conversational format. Specifically, in activity A108 in FIG. 5, the reception unit 231 receives voice data of the subject who speaks about an answer to a question. For example, the presentation unit 237 may present to the subject, as a question for the subject, the content such as "What did you do yesterday?" or "What was the most memorable event in the past week?". Furthermore, the presentation unit 237 may further present to the subject a question related to the content of the answer received from the subject. For example, the evaluation unit 236 may evaluate the degree of cognitive decline of the subject based on the amount of speech acquired from the received voice data. In such a case, a trained model that has been trained in advance on conversational exchanges may be used.

[0099] When evaluating the degree of cognitive decline in a conversational manner, the presentation unit 237 may present the subject with the timing to speak a response to a question. In other words, the presentation unit 237 may further present guide information including information guiding the timing of speaking to a specific user who is the subject in a manner that the user can grasp. According to such a manner, the subject can grasp the timing to respond to a question, and the tension and psychological anxiety about the test can be reduced.

[0100] In the above embodiment, the identification unit 233 identifies the degree of the disability of the subject by a simple check, but this is not limited to the above. For example, the reception unit 231 may receive the degree of disability from the subject through self-reporting by the subject. The identification unit 233 may identify the degree of disability according to the information received from the subject.

[0101] In the above embodiment, the subject is examined at a medical institution, but this is not the only possible case. For example, the subject may be examined using his / her own subject terminal. Specifically, the information processing system 1 may further include a subject terminal as a user terminal in addition to the medical professional terminal 3. The subject terminal may have, for example, substantially the same hardware configuration as the information processing device 2 and the medical professional terminal 3. The communication unit of the subject terminal may transmit the subject's voice data generated by the sound collection device 4 to the information processing device 2 or the medical professional terminal 3. This allows the subject to easily be examined for the degree of cognitive decline at home or the like, and to be diagnosed by a medical professional remotely.

[0102] The information processing device 2 may be an on-premise type or a cloud type. As the information processing device 2 in the cloud type, the above functions and processes may be provided in the form of, for example, SaaS (Software as a Service) or cloud computing.

[0103] In the above embodiment, the information processing device 2 performs various storage and control, but multiple external devices may be used instead of the information processing device 2. That is, various information and programs may be distributed and stored in multiple external devices using block chain technology or the like.

[0104] Another aspect of the embodiment may be a program that causes a computer to execute each step.

[0105] It may be provided in any of the following ways:

[0106] (1) An information processing system comprising a processor capable of executing a program to perform each of the following steps: a presentation step presents instructions to a user to encourage the user to produce a predetermined speech sound; a reception step receives voice data related to the speech sound from the user collected by a sound collection device; a determination step determines whether the sound collection device is functioning normally based on the voice data; and an evaluation step, if the sound collection device is functioning normally, evaluates the presence or degree of cognitive decline of the user based on the voice data and a predetermined trained model, wherein the trained model is a model that has been trained in advance regarding the relationship between features of a predetermined speech sound and the presence or degree of cognitive decline.

[0107] (2) In the information processing system described in (1) above, in the presenting step, the instruction is presented to the user by auditory information.

[0108] (3) In the information processing system described in (1) or (2) above, in the presentation step, the instruction is presented to the user using visual information, and the visual information is composed of at least one of text, figures, and photographs.

[0109] (4) In the information processing system according to any one of (1) to (3) above, the reception of the voice data in the reception step is started after the presentation of the instruction in the presentation step is completed.

[0110] (5) In the information processing system according to any one of (1) to (3) above, the reception of the voice data in the reception step is started before the presentation of the instruction in the presentation step is completed.

[0111] (6) In the information processing system according to any one of (1) to (5) above, in the warning step, when the sound collection device is not functioning normally, a warning is given that the sound collection device is not functioning normally.

[0112] (7) In the information processing system described in (6) above, the warning step is executed by displaying on a specified screen a message indicating that the sound collection device is not functioning properly.

[0113] (8) In the information processing system described in (6) or (7) above, the warning step is executed on a terminal owned by a business that manufactures or maintains the sound collection device.

[0114] (9) An information processing method, comprising the steps of the information processing system according to any one of (1) to (8) above.

[0115] (10) A program that causes a computer to execute each of the steps described in any one of (1) to (8) above. Of course, this is not the case.

[0116] Finally, although various embodiments according to the present disclosure have been described, these are presented as examples and are not intended to limit the scope of the invention. The novel embodiment can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. The embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0117] 1: Information processing system 2: Information processing equipment 3: Medical device 4: Sound collection device 5: Operator terminal 6: Check screen 7: Instruction screen 8: Recording screen 9: Evaluation result display screen 20: Communication bus 21: Communications Department 22: Storage section 23: Processor 30: Communication bus 31: Communications Department 32: Storage section 33: Processor 34:Display section 35: Input section 61 :Area 62 :Area 63: Button 71: Character 72 :Area 73: Button 81 :Area 82 :Area 83: Record button 84: Stop button 85 :Area 86 :Area 87 :Area 91: area 92: area 93 :Area 94: Remeasurement button 110: Acquisition section 120: Calculation section 130: Evaluation section 140: Output section 160: Instruction section 231: Reception 232: Acquisition Department 233: Specific part 234: Judgment section 235: Warning section 236: Evaluation section 237:Presentation part 621: Checkbox 622: Checkbox 921: Measurement result 931 :Comparison result 932 :Item 933 :Item 934: Rating value 935: Rating

Claims

1. An information processing system, A processor capable of executing a program to perform the following steps: In the presenting step, an instruction for prompting the user to produce a predetermined speech sound is presented to the user; In the receiving step, voice data relating to the speech voice from the user collected by a sound collection device is received, In the determination step, it is determined whether or not the sound collection device is functioning normally based on the voice data; In the evaluation step, when the sound collection device is functioning normally, the presence or degree of cognitive decline of the user is evaluated based on the voice data and a predetermined trained model, wherein the trained model is a model that has been trained in advance to determine the relationship between features of a predetermined speech voice and the presence or degree of cognitive decline.

2. 2. The information processing system according to claim 1, In the presenting step, the system presents the instructions to the user by auditory information.

3. 2. The information processing system according to claim 1, In the presenting step, the instruction is presented to the user by visual information; A system in which the visual information is composed of at least one of text, graphics, and photographs.

4. 2. The information processing system according to claim 1, A system in which the reception of the voice data in the reception step is started after the presentation of the instruction in the presentation step is completed.

5. 2. The information processing system according to claim 1, A system in which the reception of the voice data in the reception step is started before the presentation of the instruction in the presentation step is completed.

6. 2. The information processing system according to claim 1, In the warning step, when the sound collecting device is not functioning normally, a warning is given that the sound collecting device is not functioning normally.

7. 7. The information processing system according to claim 6, The system, wherein the warning step is executed by displaying on a specified screen a message indicating that the sound collecting device is not functioning properly.

8. 7. The information processing system according to claim 6, A system in which the warning step is executed on a terminal owned by a business that manufactures or maintains the sound collecting device.

9. 1. An information processing method, comprising: A method comprising the steps in an information processing system according to any one of claims 1 to 8.

10. A program, A program causing a computer to execute each step according to any one of claims 1 to 8.

Images