Social phobia rehabilitation auxiliary social scene simulation robot and interactive training system

Abstract

The invention discloses a social scene simulation robot for assisting social phobia rehabilitation and an interactive training system, belongs to the technical field of social phobia rehabilitation assisting equipment, and can adjust the number of interactive people and topic difficulty according to the tolerance of a patient. Comprising a robot body, a camera and a microphone, the robot body is provided with a processing device, and the processing device is used for receiving a specified request, controlling social scene simulation equipment to display a corresponding social scene and an interaction number according to the specified request and calling a topic of a corresponding topic difficulty level from a preset database; the camera and the microphone are mounted on the robot body and are in communication connection with the processing device; the camera is used for capturing facial expressions and body actions of a user, and the microphone is used for collecting voice information of the user; the processing device is further used for calculating the tolerance degree of the user according to the facial expression and the body action of the user captured by the camera and the voice information of the user collected by the microphone, and controlling the social scene simulation equipment to adjust the number of interacting people and topics according to the tolerance degree.

Description

Technical Field

[0001] The present invention relates to the technical field of social phobia rehabilitation auxiliary equipment, and in particular to a social phobia rehabilitation auxiliary social scene simulation robot and an interactive training system. Background Art

[0002] Social phobia (SAD) is a common psychological disorder, characterized by strong anxiety or fear in social situations. Users are usually afraid to express themselves in front of others, worrying about being negatively evaluated or embarrassed by others, and even have obvious physical and psychological discomfort when facing public occasions, such as accelerated heartbeat, sweating, language barriers, etc. This psychological disorder has a serious negative impact on users' daily life, study, work and interpersonal communication.

[0003] At present, the intervention and treatment methods for social phobia mainly include drug therapy, psychological intervention and exposure therapy. Among them, exposure therapy is widely regarded as one of the effective treatment methods. Its core is to reduce the anxiety level of users by gradually exposing them to the social scenes they fear. However, in practice, traditional exposure therapy has certain difficulties. For example, users may resist facing the real social environment directly, or it may be difficult to construct a suitable social scene due to the limitations of actual conditions.

[0004] Some related treatment devices, such as virtual reality (VR) technology or simple role-playing methods, can simulate social scenes to a certain extent, but lack the ability to dynamically adjust the number of people interacting, the difficulty of the topic, etc. The complexity of the scene cannot be gradually adjusted according to the patient's tolerance, which may cause the patient to interrupt the treatment due to overstimulation, or fail to make significant progress due to insufficient stimulation.

[0005] The disclosure of the above background technology content is only used to assist in understanding the concept and technical solution of the present invention. It does not necessarily belong to the prior art of this patent application. In the absence of clear evidence that the above content has been disclosed on the filing date of this patent application, the above background technology should not be used to evaluate the novelty and creativity of the present application. Summary of the invention

[0006] The present application provides a social phobia rehabilitation auxiliary social scene simulation robot and interactive training system, which can adjust the number of interactions and the difficulty of topics according to the user's tolerance to improve the treatment effect.

[0007] To achieve the above objectives, the present application discloses the following technical solutions:

[0008] In the first aspect, an embodiment of the present application provides a social scene simulation robot for assisting in the rehabilitation of social phobia, including a robot body, a camera and a microphone. The robot body is provided with a processing device, and the processing device is used to receive a specified request, the specified request including a specified social scene, a specified number of interactions and a specified topic difficulty. The processing device is also used to control the social scene simulation device to display the corresponding social scene and the number of interactions and to call topics of corresponding topic difficulty levels from a preset database according to the specified request; the camera and the microphone are installed on the robot body and are communicatively connected with the processing device; the camera is used to capture the user's facial expressions and body movements, and the microphone is used to collect the user's voice information; the processing device is also used to calculate the user's tolerance based on the user's facial expressions and body movements captured by the camera and the user's voice information collected by the microphone, and to control the social scene simulation device to adjust the number of interactions and topics according to the tolerance.

[0009] In an embodiment of the present application, a request for specifying a social scene, number of interactions, and topic difficulty can be received through the processing device of the robot body, and the social scene simulation device can be controlled to display the corresponding social scene and number of interactions and call topics of corresponding topic difficulty levels, thereby preliminarily meeting the user's demand for scene complexity.

[0010] The user's facial expressions, body movements and voice information are captured in real time through cameras and microphones, and the processing device is used to calculate the user's tolerance based on this information. The number of interactions and topics are then adjusted based on the tolerance. This allows the complexity of the social scene to be gradually adjusted based on the user's real-time tolerance level, preventing the user from interrupting treatment due to excessive stimulation or having difficulty making significant progress due to insufficient stimulation.

[0011] In some possible implementations of the first aspect, the step of calculating the user's tolerance based on the user's facial expressions and body movements captured by the camera and the user's voice information collected by the microphone includes:

[0012] The facial expression data captured by the camera is subjected to feature extraction according to a preset expression feature classification model to obtain a facial expression feature vector;

[0013] Extracting acoustic features of speech signals from the speech information collected by the microphone to obtain speech acoustic feature vectors, wherein the acoustic features include short-time energy, zero-crossing rate, and resonance peak;

[0014] The body motion data captured by the camera are subjected to feature extraction according to a preset body motion feature classification model to obtain a body motion feature vector;

[0015] Compare the facial expression feature vector with a preset expression feature vector template library under different tolerance levels to obtain an expression matching value;

[0016] Compare the speech acoustic feature vector with a preset speech acoustic feature vector template library under different tolerance levels to obtain a speech matching value;

[0017] Compare the body motion feature vector with a preset body motion feature vector template library under different tolerance levels to obtain a body motion matching value;

[0018] The expression matching value, voice matching value and body movement matching value are weightedly calculated to obtain the tolerance.

[0019] In this way, through multi-dimensional (facial expressions, voice, body movements) data collection and analysis, the user's tolerance can be comprehensively and accurately calculated, providing more reliable data support for the subsequent adjustment of social scenarios based on tolerance.

[0020] In some possible implementations of the first aspect, the step of the processing device controlling the social scene simulation device to adjust the number of interactors and topics according to the tolerance includes:

[0021] Read the range of the number of people interacting and the difficulty level of the topic corresponding to different tolerance intervals from the preset database;

[0022] Compare the current user's tolerance with the preset tolerance range to determine the specific tolerance range in which the user is located;

[0023] Determine the number of interactions to be adjusted according to the range of the number of interactions corresponding to the determined tolerance interval;

[0024] According to the number of interactions to be adjusted and the difficulty level of the topic, the social scene simulation device is controlled to adjust the displayed number of interactions, and topics of corresponding difficulty levels are recalled from a preset database.

[0025] In this way, the number of interacting people and topics in social scenes can be accurately and dynamically adjusted according to the user's real-time tolerance, so that the complexity of the social scene matches the user's actual tolerance, thereby avoiding excessive stimulation of the user due to excessive difficulty of the scene, or insufficient stimulation due to overly simple scenes, and effectively improving the effect of rehabilitation training.

[0026] In some possible implementations of the first aspect, the step of determining the number of interactive persons to be adjusted according to the determined range of the number of interactive persons corresponding to the tolerance interval includes:

[0027] The current user's tolerance is normalized with the minimum and maximum values ​​of the preset tolerance range to obtain a normalization coefficient α, which is calculated as follows:

[0028]

[0029] Get the minimum interactive number value N from the interactive number range corresponding to the tolerance interval min and the maximum number of interactions N max ;

[0030] According to the minimum interactive number value N min , Maximum number of interactive people N max And the normalization coefficient α is used to calculate the initial number of interactive people N. The specific formula is as follows:

[0031] N=N min +α×(N max -N min )

[0032] The initial number of interactions N is rounded up to obtain the number of interactions to be adjusted.

[0033] In this way, the user tolerance is mapped to the proportional relationship of the interactive number range, and a reasonable interactive number adjustment value is calculated in combination with the boundary value of the interactive number range, so that the adjustment of the interactive number is more accurate and meets actual needs.

[0034] In some possible implementations of the first aspect, the processing device is further used for:

[0035] Record the user's facial expression feature vector, voice acoustic feature vector, body movement feature vector and real-time tolerance in each social scene simulation to generate interactive behavior trajectory data;

[0036] Data mining is performed on the interactive behavior trajectory data to analyze the user's performance patterns in different topic difficulty levels and different interactive number scenarios, generate a comprehensive performance score for the user in each simulation process, and associate it with historical simulation records to form a user tolerance improvement trend chart. In this way, it helps medical staff or users understand the user's performance in different scenarios, intuitively see the tolerance improvement trend, and facilitate the formulation of a more reasonable treatment plan.

[0037] In some possible implementations of the first aspect, the robot body further has a touch screen, and the processing device receives the specified request through the touch screen. In this way, the touch screen can be used as a human-computer interaction interface to receive user input and transmit it to the processing device, which is convenient for user operation.

[0038] In some possible implementations of the first aspect, the social scene includes a party and a speech.

[0039] In a second aspect, an embodiment of the present application provides an interactive training system, including a social phobia rehabilitation-assisted social scene simulation robot and a social scene simulation device as described in any technical solution of the first aspect.

[0040] Since the interactive training system provided in the embodiment of the present application includes a social phobia rehabilitation auxiliary social scene simulation robot as described in any of the above technical solutions, the two can solve the same technical problems and achieve the same technical effects, which will not be repeated here.

[0041] In some possible implementations of the second aspect, the social scene simulation device is a VR device. In this way, an immersive social scene simulation can be provided, thereby enhancing the user's experience and improving the treatment effect.

[0042] In some possible implementations of the second aspect, when a virtual interactive object in a VR device performs language feedback, the feedback state of the virtual interactive object is adjusted according to the user's expression tone and emotional characteristics; the feedback state includes feedback tone and feedback expression. In this way, the feedback state of the virtual interactive object can be adjusted to make the virtual interaction more real and natural, and enhance the user's sense of participation and therapeutic effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the structure of a social phobia rehabilitation-assisted social scene simulation robot and an interactive training system provided in some embodiments of the present application;

[0044] Figure 2 for Figure 1 The interactive relationship diagram of the social phobia rehabilitation-assisted social scene simulation robot and the social scene simulation device shown;

[0045] Figure 3 Schematic diagram of calculation process of tolerance;

[0046] Figure 4 A flowchart for adjusting the number of people interacting and topics according to tolerance;

[0047] Figure 5 A schematic diagram of the structure of an interactive training system provided in some embodiments of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] It should be noted that, in the description of the present invention, terms such as "center", "upper", "lower", "horizontal", "inner", etc. indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0050] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0051] See also Figure 1 Combined with Figure 2 The embodiment of the present application provides a social phobia rehabilitation assisting social scene simulation robot 1, comprising a robot body 11, a camera 12 and a microphone 13. The robot body 11 is provided with a processing device 111, and the processing device 111 is used to receive a specified request, the specified request includes a specified social scene, a specified number of interacting people and a specified topic difficulty. The processing device 111 is also used to control the social scene simulation device 2 to display the corresponding social scene and the number of interacting people and call topics of corresponding topic difficulty levels from a preset database according to the specified request; exemplarily, the social scenes include parties and speeches.

[0052] The camera 12 and the microphone 13 are installed on the robot body 11 and are in communication connection with the processing device 111; the camera 12 is used to capture the user's facial expressions and body movements, and the microphone 13 is used to collect the user's voice information; the processing device 111 is also used to calculate the user's tolerance based on the user's facial expressions and body movements captured by the camera 12 and the user's voice information collected by the microphone 13, and control the social scene simulation device 2 to adjust the number of interactions and topics based on the tolerance.

[0053] In this way, the processing device 111 of the robot body 11 can receive requests for specifying social scenes, number of interactions and topic difficulty, control the social scene simulation device 2 to display the corresponding social scenes and number of interactions and call topics of corresponding topic difficulty levels, thereby preliminarily meeting the user's demand for scene complexity.

[0054] The camera 12 and microphone 13 are used to capture the user's facial expressions, body movements and voice information in real time, and the processing device 111 is used to calculate the user's tolerance based on this information, and then the number of interactions and topics are adjusted according to the tolerance. This can gradually adjust the complexity of the social scene according to the user's real-time tolerance level, thereby preventing the user from interrupting treatment due to excessive stimulation or having difficulty making significant progress due to insufficient stimulation.

[0055] See also Figure 1 On the basis of the above embodiment, the robot body 11 is further provided with a touch screen 14, and the processing device 111 receives the designated request through the touch screen 14. In this way, the touch screen 14 can be used as a human-machine interaction interface to receive user input and transmit it to the processing device 111, so as to facilitate user operation.

[0056] It is understandable that the social phobia rehabilitation auxiliary social scene simulation robot 1 can have an anthropomorphic appearance and can perform social simulation with the user without the aid of the social scene simulation device 2, and can display rich facial expressions and movements through its own touch screen 14.

[0057] See also Figure 3 In some embodiments, the step of calculating the user's tolerance based on the user's facial expressions and body movements captured by the camera 12 and the user's voice information collected by the microphone 13 includes:

[0058] S101: extracting features from the facial expression data captured by the camera 12 according to a preset expression feature classification model to obtain a facial expression feature vector;

[0059] S102: extracting acoustic features of the speech signal from the speech information collected by the microphone 13 to obtain a speech acoustic feature vector, wherein the acoustic features include short-time energy, zero-crossing rate, and resonance peak;

[0060] S103: extracting features from the body motion data captured by the camera 12 according to a preset body motion feature classification model to obtain a body motion feature vector;

[0061] S104: comparing the facial expression feature vector with a preset expression feature vector template library under different tolerance levels to obtain an expression matching value;

[0062] S105: Compare the speech acoustic feature vector with a preset speech acoustic feature vector template library under different tolerance levels to obtain a speech matching value;

[0063] S106: comparing the body motion feature vector with a preset body motion feature vector template library under different tolerance levels to obtain a body motion matching value;

[0064] S107: Perform weighted calculation on the expression matching value, the voice matching value and the body movement matching value to obtain the tolerance.

[0065] In this way, through multi-dimensional (facial expressions, voice, body movements) data collection and analysis, the user's tolerance can be comprehensively and accurately calculated, providing more reliable data support for the subsequent adjustment of social scenarios based on tolerance.

[0066] See also Figure 4 In some embodiments, the processing device 111 controls the social scene simulation device 2 to adjust the number of interactors and topics according to the tolerance level, including:

[0067] S201: Reading the range of the number of people interacting and the difficulty level of the topic corresponding to different tolerance intervals from a preset database;

[0068] S202: Compare the tolerance of the current user with a preset tolerance range to determine the specific tolerance range in which the user is located;

[0069] S203: Determine the number of interactions to be adjusted according to the determined range of the number of interactions corresponding to the tolerance interval;

[0070] S204: According to the number of interactions to be adjusted and the topic difficulty level, the social scene simulation device 2 is controlled to adjust the displayed number of interactions, and topics of corresponding topic difficulty levels are recalled from a preset database.

[0071] In this way, the number of interacting people and topics in social scenes can be accurately and dynamically adjusted according to the user's real-time tolerance, so that the complexity of the social scene matches the user's actual tolerance, thereby avoiding excessive stimulation of the user due to excessive difficulty of the scene, or insufficient stimulation due to overly simple scenes, and effectively improving the effect of rehabilitation training.

[0072] In some embodiments, the step of determining the number of interactive persons to be adjusted according to the determined range of the number of interactive persons corresponding to the tolerance interval includes:

[0073] The current user's tolerance is normalized with the minimum and maximum values ​​of the preset tolerance range to obtain a normalization coefficient α, which is calculated as follows:

[0074]

[0075] Get the minimum interactive number value N from the interactive number range corresponding to the tolerance interval min and the maximum number of interactions N max ;

[0076] According to the minimum interactive number value N min , Maximum number of interactive people N maxAnd the normalization coefficient α is used to calculate the initial number of interactive people N. The specific formula is as follows:

[0077] N=N min +α×(N max -N min )

[0078] The initial number of interactions N is rounded up to obtain the number of interactions to be adjusted.

[0079] In this way, the user tolerance is mapped to the proportional relationship of the interactive number range, and a reasonable interactive number adjustment value is calculated in combination with the boundary value of the interactive number range, so that the adjustment of the interactive number is more accurate and meets actual needs.

[0080] In some embodiments, the processing device 111 is further used to: record the user's facial expression feature vector, voice acoustic feature vector, body movement feature vector and real-time tolerance in each social scene simulation to generate interactive behavior trajectory data;

[0081] Data mining is performed on the interactive behavior trajectory data to analyze the user's performance patterns in different topic difficulty levels and different interactive number scenarios, generate a comprehensive performance score for the user in each simulation process, and associate it with historical simulation records to form a user tolerance improvement trend chart. In this way, it helps medical staff or users understand the user's performance in different scenarios, intuitively see the tolerance improvement trend, and facilitate the formulation of a more reasonable treatment plan.

[0082] See also Figure 5 The embodiment of the present application also provides an interactive training system, including the social phobia rehabilitation auxiliary social scene simulation robot 1 and the social scene simulation device 2 described in any technical solution of the first aspect.

[0083] Exemplarily, the social scene simulation device 2 is a VR device. In this way, an immersive social scene simulation can be provided, the user's experience can be enhanced, and the treatment effect can be improved. Of course, the present application is not limited to this. In other embodiments, the social scene simulation device 2 can also be a holographic projector or a display screen.

[0084] Based on the above embodiment, when the virtual interactive object in the VR device performs language feedback, the feedback state of the virtual interactive object is adjusted according to the user's expression tone and emotional characteristics; the feedback state includes feedback tone and feedback expression. In this way, the feedback state of the virtual interactive object can be adjusted to make the virtual interaction more real and natural, and enhance the user's sense of participation and therapeutic effect.

[0085] The present invention and its embodiments are described above, and such description is not restrictive. What is shown in the full text is only one of the embodiments of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A social phobia rehabilitation assisting social scene simulation robot, characterized in that: include: A robot body, wherein the robot body is provided with a processing device, wherein the processing device is used to receive a specified request, wherein the specified request includes a specified social scene, a specified number of interactors, and a specified topic difficulty, and the processing device is further used to control a social scene simulation device to display a corresponding social scene and number of interactors and to call a topic of a corresponding topic difficulty level from a preset database according to the specified request; Camera; The camera and the microphone are installed on the robot body and are in communication with the processing device; the camera is used to capture the user's facial expressions and body movements, and the microphone is used to collect the user's voice information; the processing device is also used to calculate the user's tolerance based on the user's facial expressions and body movements captured by the camera and the user's voice information collected by the microphone, and control the social scene simulation device to adjust the number of interactions and topics based on the tolerance.

2. The social phobia rehabilitation-assisted social scene simulation robot according to claim 1, characterized in that: The step of calculating the user's tolerance based on the user's facial expressions and body movements captured by the camera and the user's voice information collected by the microphone includes: Extracting features from the facial expression data captured by the camera according to a preset expression feature classification model to obtain a facial expression feature vector; Extracting acoustic features of speech signals from the speech information collected by the microphone to obtain speech acoustic feature vectors, wherein the acoustic features include short-time energy, zero-crossing rate, and resonance peak; The body motion data captured by the camera are subjected to feature extraction according to a preset body motion feature classification model to obtain a body motion feature vector; Compare the facial expression feature vector with a preset expression feature vector template library under different tolerance levels to obtain an expression matching value; Compare the speech acoustic feature vector with a preset speech acoustic feature vector template library under different tolerance levels to obtain a speech matching value; Compare the body motion feature vector with a preset body motion feature vector template library under different tolerance levels to obtain a body motion matching value; The tolerance is obtained by performing weighted calculation on the expression matching value, the voice matching value and the body movement matching value.

3. The social phobia rehabilitation-assisted social scene simulation robot according to claim 2, characterized in that: The step of the processing device controlling the social scene simulation device to adjust the number of interacting people and topics according to the tolerance includes: Read the range of the number of people interacting and the difficulty level of the topic corresponding to different tolerance intervals from the preset database; Compare the current user's tolerance with the preset tolerance range to determine the specific tolerance range in which the user is located; Determine the number of interactions to be adjusted according to the range of the number of interactions corresponding to the determined tolerance interval; According to the number of interactors to be adjusted and the topic difficulty level, the social scene simulation device is controlled to adjust the displayed number of interactors, and topics of corresponding topic difficulty levels are recalled from a preset database.

4. The social phobia rehabilitation-assisted social scene simulation robot according to claim 3, characterized in that: The step of determining the number of interactive persons to be adjusted according to the range of the number of interactive persons corresponding to the determined tolerance interval includes: The current user's tolerance is normalized with the minimum and maximum values ​​of the preset tolerance range to obtain a normalization coefficient α, which is calculated as follows: Obtain the minimum interactive number value N from the interactive number range corresponding to the tolerance interval min and the maximum number of interactions N max ; According to the minimum interactive number value N min , the maximum interactive number value N max The initial number of interactive persons N is calculated by using the normalization coefficient α. The specific formula is as follows: N=N min +α×(N max -N min ) The initial number of interactive persons N is rounded up to obtain the number of interactive persons to be adjusted.

5. The social phobia rehabilitation-assisted social scene simulation robot according to claim 4, characterized in that: The processing device is also used for: Record the user's facial expression feature vector, voice acoustic feature vector, body movement feature vector and real-time tolerance in each social scene simulation to generate interactive behavior trajectory data; Data mining is performed on the interactive behavior trajectory data to analyze the user's performance patterns in different topic difficulty levels and different interactive number scenarios, generate a comprehensive performance score of the user in each simulation process, and associate it with historical simulation records to form a user tolerance improvement trend chart.

6. The social phobia rehabilitation-assisted social scene simulation robot according to claim 5, characterized in that: The robot body also has a touch screen, and the processing device receives the designated request through the touch screen.

7. The social phobia rehabilitation-assisted social scene simulation robot according to claim 1, characterized in that: The social scenes include parties and speeches.

8. An interactive training system, characterized in that: It comprises a social scene simulation device and a social phobia rehabilitation-assisting social scene simulation robot as described in any one of claims 1-7.

9. The interactive training system according to claim 8, characterized in that: The social scene simulation device is a VR device.

10. The interactive training system according to claim 9, characterized in that: When a virtual interactive object in a VR device performs language feedback, the feedback state of the virtual interactive object is adjusted according to the expression tone and emotional characteristics of the user; the feedback state includes feedback tone and feedback expression.

Images