A visual-tactile combined feedback system and method for reaction force testing

Abstract

The present invention discloses a combined visual and tactile feedback system and method for reaction force testing, relating to the field of new medical technologies. The feedback system comprises: a computer provided with an interactive experimental scene, a force acquisition platform and a VR helmet; the computer is connected to the force acquisition platform and the VR helmet respectively; the computer is used to receive fingertip pressure acquired by the force acquisition platform and record the duration of the fingertip pressure that meets a set threshold range, and to determine a reaction force curve based on the fingertip pressure and the duration, and is also used to send the interactive experimental scene to the VR helmet; the VR helmet is worn on the head of a subject; the VR helmet is used to display the interactive experimental scene to the subject; the force acquisition platform is used to collect the fingertip pressure of the subject according to the interactive experimental scene. The present invention can be used for user attention assessment and training.

Description

Technical Field

[0001] The present invention relates to the field of new medical technologies, and in particular to a visual-tactile combined feedback system and method for reaction force testing. Background Art

[0002] Attention Deficit Hyperactivity Disorder (ADHD), also known as ADHD, is a common behavioral disorder in childhood. Clinically, it manifests as inattention, hyperactivity, and impulsivity, accompanied by cognitive, emotional, and behavioral deviations. These symptoms severely impact children's learning and well-being, leading to significant learning difficulties, strained relationships with family and peers, low self-esteem, and motor coordination difficulties. The prevalence of ADHD among preschoolers in China ranges from 4.31% to 5.83%. Recent data indicate that childhood ADHD can persist into adolescence and even adulthood, with 30% to 70% of children with childhood ADHD still experiencing significant symptoms in adulthood, impacting their daily lives and work. Therefore, scientific and accurate early screening and assessment of attention deficit disorder in children with ADHD, along with timely intervention and training, are crucial. At present, traditional attention rehabilitation assessment is mainly based on scale assessment and computer-assisted assessment. Scale assessment is subject to the evaluator's personal subjectivity and the limitations of the clinician's diagnosis and treatment experience. It is highly subjective. The existing quantitative assessment methods for attention rehabilitation are not objective enough, and the test has great limitations, making it impossible to obtain more accurate quantitative assessment results of attention. Summary of the Invention

[0003] The purpose of the present invention is to provide a visual-tactile combined feedback system and method for reaction force testing, which can be used for user attention assessment and training.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A visual-tactile combined feedback system for reaction force testing, the feedback system comprising: a computer with an interactive experimental scene, a force acquisition platform, and a VR helmet;

[0006] The computer is connected to the force acquisition platform and the VR helmet respectively; the computer is used to receive the fingertip pressure acquired by the force acquisition platform and record the duration of the fingertip pressure that meets a set threshold range, and determine a reaction force curve based on the fingertip pressure and the duration, and is also used to send the interactive experiment scene to the VR helmet;

[0007] The VR helmet is worn on the subject's head; the VR helmet is used to display the interactive experiment scene to the subject;

[0008] The force acquisition platform is used to collect the fingertip pressure of the subject according to the interactive experiment scenario.

[0009] Optionally, the feedback system further includes:

[0010] an analog-to-digital conversion module, connected to the force acquisition platform, for converting the fingertip pressure into a digital signal;

[0011] A communication module is connected to the analog-to-digital conversion module and the computer respectively, and is used to send the digital signal to the computer.

[0012] Optionally, the computer is provided with Unity software; the Unity software is used to render the interactive experiment scene.

[0013] Optionally, the force acquisition platform is a pressure sensor.

[0014] Optionally, the sampling frequency of the force acquisition platform is 50 Hz.

[0015] Optionally, the computer is connected to the VR helmet via an HDMI interface.

[0016] Optionally, the force acquisition platform includes:

[0017] a middle finger pressure collection module, connected to the computer, for collecting the pressure of the subject's middle fingertip according to the interactive experiment scenario;

[0018] The index finger pressure collection module is connected to the computer and is used to collect the index fingertip pressure of the subject according to the interactive experiment scenario.

[0019] Optionally, the middle finger pressure acquisition module includes a left middle finger pressure acquisition module and a right middle finger pressure acquisition module; the index finger pressure acquisition module includes a left index finger pressure acquisition module and a right index finger pressure acquisition module.

[0020] A combined visual and tactile feedback method for reaction force testing is applied to the above-mentioned combined visual and tactile feedback system for reaction force testing, and the feedback method includes:

[0021] Determine the user's interactive experiment scenario according to the task rules; the task rules include a task number, setting requirements, a setting threshold range, and a time threshold range; the setting requirements include the pressure of the left index finger, the pressure of the left middle finger, the pressure of the right index finger, and the pressure of the right middle finger;

[0022] According to the interactive experimental scenario, obtaining fingertip pressure;

[0023] When the fingertip pressure meets the set requirement, determining whether the fingertip pressure meets the set threshold range;

[0024] When the fingertip pressure does not meet the set requirements, record the task failure and return to the step of "obtaining the fingertip pressure according to the interactive experiment scenario";

[0025] When the fingertip pressure meets a set threshold range, obtaining the duration of the fingertip pressure;

[0026] When the fingertip pressure does not meet the set threshold range, record the task failure and return to the step of "obtaining the fingertip pressure according to the interactive experiment scenario";

[0027] When the duration does not meet the time threshold range, record the task failure and return to the step of "obtaining fingertip pressure according to the interactive experiment scenario";

[0028] When the duration meets the time threshold range, the task is recorded as successful;

[0029] Count the number of task failures and the number of task successes;

[0030] When the task sequence number is less than a preset value and the number of task failures is equal to a preset number or the number of task successes is equal to 1, the task rule is updated to obtain an updated user interaction experiment scenario;

[0031] When the task sequence number is greater than or equal to a preset value, determining whether a total number obtained by adding the number of task failures to the number of task successes is greater than or equal to the preset number;

[0032] When the total number of times is greater than or equal to the preset number of times, multiple pressure curves are obtained and the pressure curves are used as reaction force curves; the pressure curves stop timing when the task is recorded as successful or failed.

[0033] Optionally, the feedback method further includes:

[0034] The success rate and reaction time of the subjects are calculated according to the reaction force curve and the corresponding task rules.

[0035] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0036] The present invention provides a combined visual-tactile feedback system and method for reaction force testing. Through a VR-based tactile and visual feedback system, a subject completes a short-term fingertip pressure task. The feedback system records the pressure value of the fingertip pressure and the duration of the pressure value that meets a set threshold range, and obtains a reaction force curve. According to the reaction force curve, objective behavioral data detection and evaluation of the subject are performed for user attention assessment and training, thereby improving the sensitivity of the reaction force test. Furthermore, based on the characteristics of the attention allocation mechanism in the tactile channel, the state of attention is indirectly reflected through behavioral data, thereby understanding the attention ability of children, and using the reaction force curve to evaluate children with ADHD. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 This is a flowchart of the application of the visual-tactile combined feedback system for reaction force testing provided by the present invention;

[0039] Figure 2 This is a flow chart of the visual-tactile combined feedback method for reaction force testing provided by the present invention. DETAILED DESCRIPTION

[0040] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0041] The purpose of the present invention is to provide a visual-tactile combined feedback system and method for reaction force testing, which can be used for user attention assessment and training.

[0042] Based on the characteristics of the tactile channel in the attention mechanism, the present invention designs a virtual reality game with combined visual and tactile feedback to evaluate children's attention state, thereby evaluating children with ADHD. The game as a whole consists of three subsystems: a hardware platform, a software interface, and a user interaction experimental scene. The game task basically includes a target appearing randomly at two symmetrical positions. The subject changes the height of the disc by adjusting the strength of the fingertip force. Only when the target is maintained for a certain period of time will a successful feedback be displayed, otherwise a failure feedback will be given. The game can obtain the subject's behavioral data by collecting the changes in the force curve in the game.

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Example 1

[0045] like Figure 1 As shown, the present invention provides a visual-tactile combined feedback system for reaction force testing, and the feedback system includes: a computer with an interactive experimental scene, a force acquisition platform and a VR helmet.

[0046] The computer is connected to the force acquisition platform and the VR helmet respectively; the computer is used to receive the fingertip pressure obtained by the force acquisition platform and record the duration of the fingertip pressure that meets the set threshold range, and determine the reaction force curve according to the fingertip pressure and the duration, and is also used to send the interactive experiment scene to the VR helmet.

[0047] The VR helmet is worn on the subject's head; the VR helmet is used to show the interactive experiment scene to the subject.

[0048] The force acquisition platform is used to collect the fingertip pressure of the subject according to the interactive experiment scenario.

[0049] In addition, the feedback system further comprises:

[0050] The analog-to-digital conversion module is connected to the force acquisition platform and is used to convert the fingertip pressure into a digital signal.

[0051] A communication module is connected to the analog-to-digital conversion module and the computer respectively, and is used to send the digital signal to the computer.

[0052] As a specific implementation, the computer is provided with Unity software; the Unity software is used to render the interactive experiment scene.

[0053] As a specific implementation, the force acquisition platform is a pressure sensor, and the sampling frequency of the force acquisition platform is 50 Hz.

[0054] Specifically, the force acquisition platform includes:

[0055] The middle finger pressure collection module is connected to the computer and is used to collect the pressure of the middle finger tip of the subject according to the interactive experiment scenario.

[0056] The index finger pressure collection module is connected to the computer and is used to collect the index fingertip pressure of the subject according to the interactive experiment scenario.

[0057] Furthermore, the middle finger pressure acquisition module includes a left middle finger pressure acquisition module and a right middle finger pressure acquisition module; the index finger pressure acquisition module includes a left index finger pressure acquisition module and a right index finger pressure acquisition module.

[0058] As a specific implementation, the computer is connected to the VR helmet via an HDMI interface.

[0059] In practical applications, the feedback system provided by the present invention mainly includes a hardware platform, a software interface and a virtual reality scene.

[0060] Among them, the hardware platform mainly includes pressure sensors (force acquisition platform), analog-to-digital conversion modules, communication modules and VR helmets.

[0061] The software interface was designed using Unity on a computer. Experimental parameters, such as the number of trials, subject name, and single trial duration, were modified within the software interface. This interface served as the interface before the task began; modifying the parameters led to the virtual reality scene.

[0062] The VR scene is designed using Unity on a computer and displayed to the subject through a VR headset. The amount of pressure applied by the user is reflected by the height adjustment of the target. The computer transmits the designed VR scene to the VR headset through a rendering pipeline. The basic process involves CPU calculations followed by GPU rasterization, resulting in a color matrix, which is then displayed through the VR headset's built-in hardware. Data transmission between the VR headset and the computer is achieved via an HDMI interface, using the HDMI standard.

[0063] The software interface is the interface that appears when Unity starts. You can modify it by adding properties to variables in C# scripts. Similarly, VR scenes are also created by writing C# scripts. So, design the software interface first, then the VR scenes.

[0064] After the software and VR interfaces are designed, the pressure data acquired by the pressure sensor is transmitted to the analog-to-digital conversion module, which converts the analog signal into a digital signal before inputting it into the computer via the communication module. After the pressure data is input into the computer, the Unity software in the computer receives the signal and, according to the pre-programmed software, adjusts the target's position in the VR scene proportionally to the pressure value, thereby achieving a linear correspondence between the target's height and the subject's visual feedback.

[0065] Specifically, this embodiment takes three tasks as an example to introduce the working process of the visual-tactile combined feedback system for reaction force testing provided by the present invention:

[0066] First, after putting on the helmet, press the spacebar (which indicates the start of the game) to start a task. After completing the first task, rest for 2 minutes before continuing the experiment and starting the next task. The subjects were presented with the user's interactive experimental scene by wearing the helmet, providing visual feedback. In the virtual environment, there are two translucent cylinders, each with a colored disc at the bottom. During the task, a gray cylindrical target of a fixed height will randomly appear above one of the colored discs. Because the height of the colored disc is proportional to the pressure detected by the pressure sensor, the subject quickly presses the corresponding finger button and controls the fingertip force to keep the colored disc within the gray cylinder for a period of time. This force is measured by the force sensor and involves the middle and index fingers of both hands. When one finger presses, the other fingers must be completely released. If the task is completed, the feedback system (the VR headset provides a visual display, so the feedback system is the VR headset itself) displays a sound effect and visuals indicating the achievement. If the duration (a timer in the C# script records the time; for example, as long as the pressure data is within the range of 0.9N to 1.1N, the timer will continue to count until 300ms, indicating success. If the pressure requirement is not met midway, the timer will reset to zero and restart from 0 again. If the timer fails to reach 300ms within the time limit set for each trial during the task, it is considered a failure) is less than the specified time and the response time is exceeded, a failure is displayed. Each subject is required to complete three tasks. The first task requires each trial to be completed within a time limit of 1.7s, and the second task requires each trial to be completed within a time limit of 2.5s. Failure to complete these tasks results in a failure display. Otherwise, the achievement is displayed and a real-time incentive is given. If two tasks are completed early, the subject will proceed to Task 3. The duration of each experiment is not fixed. The time limit for the third task is fixed at 2s, and all performances (success or failure) in the third task are marked as success. There is no incentive. Each of the three tasks consisted of 100 trials, each lasting approximately 5-8 minutes, with a 2-minute rest period between tasks, for a total duration of approximately 30 minutes. The pressure sensor sampling rate was 50 Hz throughout the entire task.

[0067] Through these three tasks, we obtained measurement data for the pressure curves of two tasks of different durations (the duration was not fixed, with the success or failure time as the stop time) and the success / failure status, as well as measurement data for the pressure curve of a fixed 2s task and the success / failure status. The measurement data was stored as a string of numbers in an Excel spreadsheet. Regarding success and failure, according to the previous task rules, the program automatically determines and calculates the statistics based on the collected pressure level and duration, recording success as 1 and failure as 0.

[0068] As a specific implementation method, the force acquisition platform can be replaced by other force sensors; similarly, the experimental paradigm of the game can also be changed to adjust different trial requirements or target heights in the virtual scene. At the same time, the interactive experimental scene can also be developed using other platforms such as C++, Unreal Engine UE, etc.

[0069] The main purpose of collecting measurement data and success / failure situations in the present invention is to obtain the success rate of the subject and various indicators of the reaction curve such as reaction time / error, etc., to provide objective data for subsequent judgment of the subject's attention situation.

[0070] This invention provides a more reliable visual-tactile virtual reality game for assessing the condition of children with ADHD than questionnaires and consultations. It is more appealing to children and can quickly capture behavioral characteristics, enabling objective and quantitative condition assessment, thus being used for children's attention assessment and training.

[0071] The present invention uses a force acquisition platform to collect mechanical signals to track behavioral data, providing objective data for subsequent data analysis. It can also interact with virtual reality scenes, changing the position of targets within them. The virtual reality scenes are presented through VR helmets, providing visual feedback. Virtual reality scenes are more engaging for children, facilitate experiments, and produce more sensitive reaction force test results.

[0072] Example 2

[0073] In order to implement the system corresponding to the above embodiment 1 and achieve the corresponding functions and technical effects, a visual-tactile combined feedback method for reaction force testing is provided below. Figure 2 As shown, the feedback method includes:

[0074] Step S101: Determine the user's interactive experimental scenario according to the task rules; the task rules include a task number, setting requirements, a setting threshold range, and a time threshold range; the setting requirements include the pressure on the left index finger, the pressure on the left middle finger, the pressure on the right index finger, and the pressure on the right middle finger.

[0075] Step S102: acquiring fingertip pressure according to the interactive experiment scenario.

[0076] Step S103: When the fingertip pressure meets the set requirement, it is determined whether the fingertip pressure meets the set threshold range.

[0077] Step S104: When the fingertip pressure does not meet the set requirement, the recording task fails and returns to S102.

[0078] Step S105: When the fingertip pressure meets the set threshold range, the duration of the fingertip pressure is obtained.

[0079] Step S106: When the fingertip pressure does not meet the set threshold range, the recording task fails and returns to S102.

[0080] Step S107: When the duration does not meet the time threshold range, the recording task fails and returns to S102.

[0081] Step S108: When the duration meets the time threshold range, the recording task is successful.

[0082] Step S109: Count the number of task failures and the number of task successes.

[0083] Step S110: When the task sequence number is less than a preset value and the number of task failures is equal to a preset number or the number of task successes is equal to 1, the task rules are updated to obtain an updated user interaction experiment scenario.

[0084] Step S111: When the task sequence number is greater than or equal to a preset value, it is determined whether the total number of task failures plus the number of task successes is greater than or equal to the preset number.

[0085] Step S112: When the total number of times is greater than or equal to the preset number of times, multiple pressure curves are obtained and the pressure curves are used as reaction force curves; the pressure curves stop timing when the task is recorded as successful or failed.

[0086] In addition, the feedback method further includes:

[0087] The success rate and reaction time of the subjects are calculated according to the reaction force curve and the corresponding task rules.

[0088] As a specific implementation, the preset number of times is 100 times.

[0089] As a specific implementation, each subject was required to complete three tasks. The first task required each trial to be completed within a time limit of 1.7 seconds, the second task required each trial to be completed within a time limit of 2.5 seconds, and the third task required each trial to be completed within a time limit of 2 seconds. Each task lasted approximately 5-8 minutes, with a 2-minute rest period between tasks. The total time required to complete all three tasks was approximately 30 minutes.

[0090] As a specific implementation, the threshold value is set to range from 0.9N to 1.1N of fingertip pressure.

[0091] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0092] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A visual-tactile combined feedback system for reaction force testing, characterized in that: The feedback system includes: a computer with an interactive experimental scene, a force acquisition platform and a VR helmet; The computer is connected to the force acquisition platform and the VR helmet respectively; the computer is used to receive the fingertip pressure acquired by the force acquisition platform and record the duration of the fingertip pressure that meets a set threshold range, and determine a reaction force curve based on the fingertip pressure and the duration, and is also used to send the interactive experiment scene to the VR helmet; The VR helmet is worn on the subject's head; the VR helmet is used to display the interactive experiment scene to the subject; The force acquisition platform is used to collect the fingertip pressure of the subject according to the interactive experiment scenario.

2. The visual-tactile combined feedback system for reaction force testing according to claim 1, characterized in that: The feedback system further comprises: an analog-to-digital conversion module, connected to the force acquisition platform, for converting the fingertip pressure into a digital signal; A communication module is connected to the analog-to-digital conversion module and the computer respectively, and is used to send the digital signal to the computer.

3. The visual-tactile combined feedback system for reaction force testing according to claim 1, characterized in that: The computer is provided with Unity software; the Unity software is used to render the interactive experiment scene.

4. The visual-tactile combined feedback system for reaction force testing according to claim 1, characterized in that: The force acquisition platform is a pressure sensor.

5. The visual-tactile combined feedback system for reaction force testing according to claim 1, characterized in that: The sampling frequency of the force acquisition platform is 50 Hz.

6. The visual-tactile combined feedback system for reaction force testing according to claim 1, characterized in that: The computer is connected to the VR helmet via an HDMI interface.

7. The visual-tactile combined feedback system for reaction force testing according to claim 1, characterized in that: The force acquisition platform includes: a middle finger pressure collection module, connected to the computer, for collecting the pressure of the subject's middle fingertip according to the interactive experiment scenario; The index finger pressure collection module is connected to the computer and is used to collect the index fingertip pressure of the subject according to the interactive experiment scenario.

8. The visual-tactile combined feedback system for reaction force testing according to claim 7, characterized in that: The middle finger pressure acquisition module includes a left middle finger pressure acquisition module and a right middle finger pressure acquisition module; the index finger pressure acquisition module includes a left index finger pressure acquisition module and a right index finger pressure acquisition module.

9. A visual-tactile combined feedback method for reaction force testing, characterized in that: The feedback method includes: Determine the user's interactive experiment scenario according to the task rules; the task rules include a task number, setting requirements, a setting threshold range, and a time threshold range; the setting requirements include the pressure of the left index finger, the pressure of the left middle finger, the pressure of the right index finger, and the pressure of the right middle finger; According to the interactive experimental scenario, obtaining fingertip pressure; When the fingertip pressure meets the set requirement, determining whether the fingertip pressure meets the set threshold range; When the fingertip pressure does not meet the set requirements, record the task failure and return to step "obtaining the fingertip pressure according to the interactive experiment scenario"; When the fingertip pressure meets a set threshold range, obtaining the duration of the fingertip pressure; When the fingertip pressure does not meet the set threshold range, record the task failure and return to step "obtaining the fingertip pressure according to the interactive experiment scenario"; When the duration does not meet the time threshold range, record the task failure and return to step "obtaining fingertip pressure according to the interactive experiment scenario"; When the duration meets the time threshold range, the task is recorded as successful; Count the number of task failures and the number of task successes; When the task sequence number is less than a preset value and the number of task failures is equal to a preset number or the number of task successes is equal to 1, the task rule is updated to obtain an updated user interaction experiment scenario; When the task sequence number is greater than or equal to a preset value, determining whether a total number obtained by adding the number of task failures to the number of task successes is greater than or equal to the preset number; When the total number of times is greater than or equal to the preset number of times, multiple pressure curves are obtained and the pressure curves are used as reaction force curves; the pressure curves stop timing when the task is recorded as successful or failed.

10. The visual-tactile combined feedback method for reaction force testing according to claim 9, characterized in that: The feedback method further includes: The success rate and reaction time of the subjects are calculated according to the reaction force curve and the corresponding task rules.

Images