Hand sensory function testing system, testing plate and testing method

Abstract

A hand sensory function test system, test plate and test method, wherein the system comprises: a test plate, an optical sensor, a system frame, a workstation and a human-computer interaction display, the test plate comprises a plurality of test touch parts protruding from the upper surface of the plate body, each test touch part has at least one notch with a directional pointing, the workstation determines a test scheme based on personal information using a large language model, the human-computer interaction display sends test guidance information under the control of the workstation, guiding the tester to put the correct test finger on the test touch part at the test starting position of the test plate, the optical sensor is arranged on the system frame, and the hand action image obtained is sent to the workstation, the workstation obtains real-time finger position and finger action through the hand action image, and obtains a test result according to the finger position and the finger action, the hand sensory function test system, the test plate and the test method can improve the objectivity and accuracy of the hand sensory function test.

Description

Technical Field

[0001] This application relates to the fields of medical devices and communication technology, and in particular to a hand sensory function testing system, test board and testing method. Background Technology

[0002] The assessment of hand sensory function is an important part of the diagnosis and treatment system for upper limb diseases. It is an important basis for judging the severity of the disease and an objective standard for evaluating the efficacy of different interventions such as surgery and rehabilitation therapy. It directly affects the scientificity and effectiveness of clinical decision-making and has important significance in scientific research, clinical follow-up, rehabilitation assessment, work injury identification, and medical insurance.

[0003] Currently, assessment of hand sensory function mainly relies on manual measurement and examination by professionally trained clinicians or rehabilitation therapists. This requires not only strict adherence to standard operating procedures by clinicians but also genuine feedback from patients, inevitably introducing subjectivity. In the field of hand surgery, the intrinsic sensory innervation areas of the radial, median, and ulnar nerve supply regions are often examined using monofilament tactile tools and two-point discrimination tools on the corresponding areas of the fingers.

[0004] However, existing assessment techniques suffer from the following drawbacks: There is a severe shortage of qualified hand surgeons and therapists, and medical resources are unevenly distributed, mainly concentrated in large cities. This severely limits the assessment, treatment, and rehabilitation guidance of hand sensory function for a large number of patients with upper limb dysfunction, significantly impacting the long-term therapeutic effects of related surgeries and rehabilitation. The inherent limitations of subjective evaluation and perception mean that errors caused by subjective judgment standards among different assessors are difficult to avoid. Furthermore, the subjective feedback from subjects is difficult to translate into objective, quantifiable indicators, hindering accurate and quantitative evaluation of hand sensory function. Existing measurement tools cannot convert patients' subjective feedback into convenient objective data signals, making it difficult to achieve self-assessment or intelligent assessment. Summary of the Invention

[0005] This application provides a hand sensory function testing system, test board, and testing method. By combining artificial intelligence, the system can determine the strength of a tester's hand sensory function by acquiring and utilizing the accuracy of the tester's touch on the test touch points on the test board, thereby improving the objectivity and accuracy of hand sensory function assessment.

[0006] One embodiment of this application provides a hand sensory function testing system, including:

[0007] This includes a test board, optical sensors, system framework, workstation, and human-computer interaction display;

[0008] The test board includes a board body and a plurality of test touch portions protruding from the upper surface of the board body, each of the test touch portions having at least one directional notch;

[0009] The workstation is used to acquire the tester's personal information and determine a test plan based on the personal information using a large language model. The test plan includes test guidance information, and the personal information includes the tester's age, gender, medical history, and test purpose.

[0010] The human-computer interaction display is connected to the workstation and is used to issue test guidance information under the control of the workstation, guiding the tester to place the correct test finger on the test touch part of the test board at the test start position.

[0011] The optical sensor is mounted on the system frame and connected to the workstation. It is used to acquire images of the test subject's hand movements and send the images to the workstation for analysis.

[0012] The workstation is used to obtain real-time finger position and finger movement through the hand motion image, and to obtain test results based on the finger position and finger movement.

[0013] One aspect of this application embodiment also provides a hand sensation function test board, including: a board body and a plurality of test touch parts protruding from the upper surface of the board body;

[0014] Each of the test touch portions has at least one directional notch.

[0015] This application also provides a method for testing hand sensory function, including: obtaining the tester's personal information and determining a test plan based on the personal information using a large language model, wherein the test plan includes test guidance information, and the personal information includes the tester's age, gender, medical history, and test purpose;

[0016] The human-computer interaction display is controlled to send out the test guidance information, which guides the tester to place the correct test finger on the test touch part at the test start position of the test board. The test board includes a board body and a plurality of test touch parts protruding from the upper surface of the board body. Each test touch part has at least one directional notch.

[0017] The test subject's hand movements are captured by an optical sensor. Real-time finger positions and movements are obtained from the hand movements, and test results are obtained based on the finger positions and movements.

[0018] As can be seen from the above embodiments of this application, this application obtains the tester's personal information through a workstation, determines the test plan based on this personal information using a large language model, displays test guidance information through a human-computer interaction display, guides the tester to place the correct test finger on the test touch part at the test starting position of the test board, acquires the tester's hand movement image, and obtains the real-time finger position and finger movement through the hand movement image. Based on the finger position and finger movement, the test result is obtained, which can transform the tester's subjective feedback into convenient objective data signals, improving the objectivity and accuracy of assessing the tester's hand sensory function. On the one hand, because this hand sensory function testing system has a simple architecture, with the workstation and human-computer interaction display working together to guide the tester to complete the test, the entire testing process has the advantages of convenience and speed. On the other hand, by combining artificial intelligence and utilizing a large language model, the test can retain both medical professionalism and standardization, while also possessing personalization, flexibility, and operability, providing a reliable basis for subsequent result analysis. Based on the above two aspects, this application can effectively solve the problem of uneven distribution of human and equipment resources that require professional functional assessment qualifications in the existing technology. In the context of uneven medical resources, this can realize a wider range of hand sensory function assessment, treatment and rehabilitation guidance, expand the application scope of hand sensory function testing, and improve the long-term treatment effect of related surgeries and rehabilitation. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A schematic diagram of the structure of a hand sensory function testing system provided in an embodiment of this application;

[0021] Figure 2 A schematic diagram of the structure of a hand sensory function testing board provided in an embodiment of this application;

[0022] Figure 3 A schematic diagram of the structure of a hand sensory function test board provided in another embodiment of this application;

[0023] Figure 4 A front view of the structure of a hand sensory function test board provided in another embodiment of this application;

[0024] Figure 5 A front view of the structure of a hand sensory function test board provided in another embodiment of this application;

[0025] Figure 6 Provided for an embodiment of this application and Figure 4 A three-dimensional structural diagram of the corresponding hand sensory function test board;

[0026] Figure 7 Provided for another embodiment of this application and Figure 4 A three-dimensional structural diagram of the corresponding hand sensory function test board;

[0027] Figure 8 For this application and Figure 3 A schematic diagram of the structure of a hand sensation function test board provided in a corresponding embodiment;

[0028] Figure 9 For this application and Figure 3 A schematic diagram of the structure of a hand sensation function test board provided in another corresponding embodiment;

[0029] Figure 10 A flowchart illustrating the implementation of a hand sensory function testing method according to an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "axial", "radial", "circumferential", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0032] In this application, unless otherwise clearly specified or limited, terms such as "installation", "connection", "linkage", "fixation" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal connection of two components or the interaction relationship between two components, unless otherwise clearly limited. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances. The technical solutions of this application will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0033] See Figure 1 , an embodiment of this application provides a hand sensory function test system for a tester to perform hand sensory tests. The tester is a person who needs to test the hand sensory function. The system includes: a hand sensory function test board (hereinafter referred to as the test board) 100, at least one optical sensor 10, a system framework 20, a workstation 30, and a human-computer interaction display 40.

[0034] An embodiment of this application also provides a hand sensory function test board, that is, the above test board 100. The structure of the test board 100 is as Figure 2 and Figure 3 shown:

[0035] The test board 100 includes: a board body 50 and a plurality of test touch parts 60 protruding from the upper surface of the board body 50. Each test touch part 60 has at least one notch 70 with a directional orientation.

[0036] The board body 50 and the test touch parts 60 can both be obtained by resin 3D printing; it can also be a non-detachable test board made of light-transmitting materials such as plexiglass or acrylic; it can also be a microarray matrix to generate protrusions in random directions according to preset dimensions.

[0037] Figure 2 As shown, adjacent test touch parts of the test board have the same size and different notch directions. The notch directions of the test touch parts are randomly set to the four directions of up, down, left, and right relative to the test touch part itself.

[0038] Specifically, the cross-sectional shape of the test touch part 60 can be C-shaped or similar to C-shaped, or E-shaped or similar to E-shaped. For example, the C-shaped can be formed by having a notch on the circumference of a circle, that is, a circle can be formed after removing the notch of the C-shaped; the E-shaped can be formed by missing one side of the Chinese character "日". For ease of description, the test touch parts 60 in the embodiments of this application are all exemplified by C-shaped cross-sections.

[0039] The thickness of the plate 50, the height of the test touch area 60, the size of the cross-section of the test touch area 60 (specifically, the diameter of the circle corresponding to the C-shape), and the width of the notch 70 (i.e., the size of the notch) can all be set to different values. See details... Figures 4-7 .

[0040] in, Figure 4 This is a top view of the test board in another embodiment. Figure 4 The width of the notch 70 in the test touch part 60 is 1mm, and the diameter of the circle corresponding to the C-shape is 5mm.

[0041] Figure 5 This is a top view of the test board in another embodiment, showing the dimensions of each part of the test board compared to... Figure 4 The differences are that the width of the notch 70 in the test touch part 60 is 1.5mm, and the diameter of the circle corresponding to the C-shape is 7.5mm.

[0042] Figure 6 and 7 They are respectively with Figure 4 Corresponding 3D view of the test board Figure 6 The C-ring height is 2mm, and the plate thickness is 5mm. Figure 7 The C-shaped ring in the middle is 5mm high, and the plate body 50 is 5mm thick.

[0043] It should be noted that the above Figures 4-7 The test touch portion 60 of each test board shown is the same size, but the test touch portion 60 of the same test board may also be different. For example, it may be divided into multiple groups of test touch portions from left to right, with the size of the test touch portion in each group decreasing. The number of test touch portions in each group may be the same or different.

[0044] The dimensions of each part of the test board can be set according to the actual application. For example, the width of the notch 70 of the test touch part 60 can also be 2mm, 2.5mm, 3mm, 3.5mm, 4mm, 4.5mm, 5mm, 5.5mm or 6mm, etc.

[0045] Based on the different widths of these notches 70, the diameters of the C-shaped circles corresponding to the different widths of the notches 70 are 10mm, 12.5mm, 15mm, 17.5mm, 20mm, 22.5mm, 25mm, 27.5mm and 30mm respectively.

[0046] When the width of the notch 70 is 2mm, 2.5mm, 3mm, 3.5mm or 4mm, the height of the C-ring and the thickness of the plate 50 can be set to 2mm for the C-ring height and 5mm for the plate 50 thickness, or 5mm for the C-ring height and 5mm for the plate 50 thickness.

[0047] When the width of the notch 70 is 4.5mm, 5mm, 5.5mm or 6mm, the height of the C-ring is 2mm and the thickness of the plate 50 is 8mm, or the height of the C-ring is 5mm and the thickness of the plate 50 is 8mm.

[0048] Understandably, the above-mentioned test board size specifications are only an example, and other sizes can be set in actual applications. This application embodiment will not elaborate on them one by one.

[0049] Furthermore, in another embodiment, Figure 3 The test board touch area includes a first touch area 61 located at the test start position and a second touch area 62 located at the test end position. A correct test path exists between the first touch area 61 and the second touch area 62, as shown below. Figure 3 The middle arrow indicates the various path-passing touch parts. The tester, following the preset test instructions, touches the second touch part 62 by touching the first touch part 61 with his finger along the correct test path based on the notch direction of the first touch part 61 and the notch direction of the multiple path-passing touch parts.

[0050] The size of the first touch portion 61 is larger than the size of the second touch portion 62.

[0051] Specifically, the structure of the test board can be as follows: Figure 3 and Figure 8 As shown, the notch of the first touch portion 61 points towards the first passing touch portion 631 closest to itself;

[0052] In the direction from the first touch part 61 to the second touch part 62, the notch direction of each of the touch parts along the correct test path points to the next adjacent touch part in sequence. For example, the notch direction of the third touch part 633 points to the fourth touch part 634 adjacent to itself.

[0053] The notch of the second path touch 632, which is close to the second touch 62, points towards the second touch 62.

[0054] The test command corresponding to the test board in this embodiment can be to touch the next test touch part in the direction indicated by the notch direction of each test touch part.

[0055] In another embodiment, the test board can also be structured as follows: Figure 9As shown, the notch of the first touch portion 61 points in the opposite direction to the first passing touch portion 631 closest to itself;

[0056] In the direction from the first touch part 61 to the second touch part 62, the notch direction of each of the touch parts along the correct test path points in the opposite direction to the next adjacent touch part. For example, the notch direction of the third touch part 633 points to the fourth touch part 634 adjacent to itself.

[0057] The direction of the notch of the second path touch 632, which is close to the second touch 62, points in the opposite direction to the second touch 62.

[0058] The test command corresponding to the test board in this embodiment can be to touch the next test touch part in the opposite direction to the direction indicated by the notch of each test touch part.

[0059] In other embodiments, the structure of the test board may also differ depending on the test instructions. For example, the next test touch point may be touched in the clockwise or counterclockwise direction indicated by the notch direction of each test touch point; or, the test instructions for the first few test touch points may be to touch the next test touch point in the direction indicated by the notch direction, and the test instructions for the remaining test touch points may be to touch the next test touch point in the opposite direction indicated by the notch direction.

[0060] See also Figure 1 The hand sensory function testing system shown:

[0061] Figure 1 The workstation 30 connects to the optical sensor 10 and the human-computer interaction display 40 via wired, wireless WiFi, or Bluetooth. The workstation 30 can specifically be a computer with image and other data processing capabilities. The optical sensor 10 is mounted on the system frame 20. The optical sensor 10 can specifically be a camera that can provide RGB (Red, Green, Blue) images, and there is at least one such camera. For ease of understanding... Figure 1 The example uses three optical sensors 10, but in practical applications, this is not the only possibility.

[0062] During the hand sensory function test, the test subject inputs personal information through the input device of the human-computer interaction display 40. The input device includes a keyboard, microphone, etc. The personal information may include, but is not limited to, the test subject's name, age, gender, medical history, current status of hand sensory function, and the purpose of the test. The purpose of the test may, for example, be: assessing the recovery of left or right hand finger function in stroke rehabilitation patients, or screening the tactile development of left or right hand fingers in children.

[0063] The human-computer interaction display 40 transmits the personal information input by the tester to the workstation 30. The workstation 30 uses a Large Language Model (LLM) to determine a test plan based on this personal information. This test plan includes test guidance information and test rules. The test guidance information includes at least one of the following: test guidance voice information, test guidance text information, and test guidance image information. The LLM can be trained based on a massive amount of user personal information, corresponding test plans, and test results in a cloud database. Optionally, after completing the test, the workstation 30 can also store the test process and results information (such as the generated test plan, captured images, acquired voice, test results, and annotations from medical experts on the test results) in the cloud database for periodic use in training the LLM. The cloud server periodically uses the data in the cloud database to retrain the LLM.

[0064] The test rules may include: test type, test path, and judgment criteria.

[0065] Specifically, the test types include: audible testing and silent testing. In audible testing, the tester verbally describes a specified feature of the object being touched. The workstation 30 performs speech recognition on this description and determines whether the tester's perception is correct based on the recognition result. In silent testing, the tester does not need to verbally describe anything; they simply follow the test guidance information and touch the test touch points with specified features on the test board in sequence. The workstation 30 determines whether the tester's perception is correct based on the captured images of the user's hand movements while touching the test board. It is understood that these hand movement images can be a static image sequence or a dynamic video.

[0066] This test path is used to determine the sequence of touches made by the tester on the test board, as well as to locate the start point, intermediate point, and end point of the touches. The start point of the touch can be the first touch area mentioned above, the intermediate point can be the intermediate touch area mentioned above, and the end point can be the second touch area mentioned above.

[0067] Optionally, the large language model can output test paths in the form of image-based images labeled with paths (e.g., the DALL·E plugin for GPT-4) via an embedded image generation plugin. Figure 3 , Figure 8 or Figure 9 (As shown by the arrow), or the output of the large language model can also be a labeling instruction used to describe the touch sequence and the characteristics of the test touch part corresponding to the start point, intermediate point and end point (e.g., whether it has a notch, notch direction, notch size, height or thickness of the test touch part, etc.).

[0068] Optionally, the test rules may also include: path adjustment rules. These path adjustment rules include: adjustment conditions and adjustment methods, used to dynamically adjust the test path according to the adjustment method when the workstation 30 detects a situation that meets the adjustment conditions during the test (e.g., when the user hesitates for more than a preset time, or when the user makes more than a preset number of errors). This adjustment method (e.g., replacing the test touch area corresponding to the current touch point and subsequent points with a test touch area with a larger notch) to reduce the difficulty of the test.

[0069] The judgment criteria may include: a preset algorithm for obtaining test results, normal range values, and / or abnormal range values.

[0070] Optionally, when there are multiple test panels, the testing rule may also include: identification information of the test panel that matches the tester's personal information.

[0071] In this way, by combining artificial intelligence and utilizing large language models, the testing can retain both medical professionalism and standardization while also being personalized, flexible, and operable, providing a reliable basis for subsequent result analysis. In the context of uneven distribution of medical resources, this can effectively expand the application scope of hand sensory function testing (such as community screening and home-based rehabilitation follow-up).

[0072] The hand sensory function testing system also includes a blocking device. After entering personal information, the tester uses the blocking device to block their view and avoid seeing the touch test part on the test board 100, thereby improving the accuracy and authenticity of the test.

[0073] Optionally, the covering device can be an object that directly covers the eyes, such as an eye mask. When the test begins, the workstation 30 sends out test guidance information through the human-computer interaction display 40. The test guidance information includes test guidance voice information, test guidance text information, and test guidance image information.

[0074] Optionally, the shielding device can also be an opaque enclosure with an opening that allows a hand to enter. The test board 100 and the system frame 20 are placed inside the enclosure, and the optical sensor 10 is mounted on the system frame 20. An infrared detector can also be mounted on the system frame 20 and connected to the workstation 30. When the tester inserts their hand through the opening, the infrared detector detects the hand entering the enclosure and triggers the workstation 30 to play test guidance voice information, or test guidance text information and / or test guidance image information, based on the test plan determined by LLM, through the speaker of the human-machine interface display 40. The test guidance voice information, the test guidance text information, and the test guidance image information all guide the tester to complete the hand sensory function test through voice, text, and image content, respectively.

[0075] Guided by the test instructions, the tester places their finger on the test touch area at the starting position on the test board. The optical sensor 10 acquires an image of the tester's hand movement and sends it to the workstation 30. The workstation 30 analyzes the real-time finger position and movement using built-in image processing and hand posture estimation algorithms. If the test finger is accurately placed on the test touch area at the starting position, a test command is issued through the human-computer interaction display 40 to start the test. This test command can also be in the form of voice, text, and image.

[0076] The test instruction includes: prompts for the test rules, instructions to start the test, information on interactions with the tester during the test, prompts to end the test, and prompts for the test results. This test instruction can be generated by the workbench 30 based on the test plan determined through LLM; for example, the prompts for the test rules can be generated based on the test rules within that test plan.

[0077] The test rule prompts guide the tester to determine the notch direction based on tactile feedback of the test touchpoint, and then follow the test rule to touch the next test touchpoint along that notch direction. Furthermore, the test rule prompts can also indicate the test type, such as whether verbal instructions are required. Additionally, the test rule prompts can also indicate the touch sequence or path.

[0078] The interaction with the tester during the test includes: the workstation 30 analyzes the hand motion image to determine the direction of the notch on the test touch area currently being touched by the tester, and judges whether the direction of the notch on the test touch area described by the tester is consistent with the determined direction of the notch. If they are consistent, the tester is notified that the result is correct and the tester proceeds to judge the direction of the notch on the next touch area. If they are inconsistent, the tester is notified that the result is incorrect and the tester continues to judge the direction of the notch on the next touch area. Alternatively, if the number of errors accumulates to a preset number, the tester is notified that the test is over.

[0079] After acquiring the hand movement image captured by the optical sensor 10, the workstation 30 automatically identifies and uses the image from the back of the hand based on the actual position of the test subject's fingers, and analyzes the coordinate data of key points of the fingers in real time to determine the movement state of the test subject's fingers. For example, the key points of the fingers may include the coordinate data of the thumb palmar key point, the index fingertip key point, and the little fingertip key point, which correspond to the sensory evaluation of the radial nerve, median nerve, and ulnar nerve, respectively.

[0080] Furthermore, the hand sensory function testing system also includes a microphone for collecting sound signals, which can be worn by the tester, integrated into the optical sensor 10, or placed in the human-computer interaction display 40.

[0081] In one embodiment, the tester used Figure 2 The test board shown is used for testing. The workstation 30 acquires the tester's spoken voice signal through the microphone and analyzes the voice signal to obtain the direction of the notch of the test touch part as perceived by the tester. The workstation 30 simultaneously captures the direction of the notch of the test touch part touched by the tester's finger through the optical sensor 10. The notch direction described by the tester is compared with the notch direction to determine whether the tester's perception is accurate, and to obtain the smallest notch size that the tester can accurately identify. The time for the tester to complete each test touch part test and the total time to complete the entire test board test are counted. Combining the perceived smallest notch size, perception time and perception result, the tactile perception evaluation of the tester's fingertip is judged. Specifically, the tactile evaluation result = score of the perceived smallest notch size - penalty for a single perception time - penalty for the number of perception errors.

[0082] Among them, the single perception time penalty refers to the score corresponding to the portion that exceeds the preset perception time when testing a certain test touch part; the perception error number penalty refers to the score corresponding to the sum of the number of touch test parts with incorrect perception notch direction.

[0083] Workstation 30 stores a table mapping the minimum perceptible notch size to scores, a table mapping the portion of a single perception time exceeding a preset perception time to scores, and a table mapping the number of perception errors to scores. It can obtain corresponding scores for the minimum perceptible notch size measured by the tester through the test board, the perception time of a single touch test part, and the number of perception errors on the touch test part, respectively. The tactile evaluation result is then obtained using the above formula. This allows for an objective and quantitative assessment of the tester's finger sensory function.

[0084] In one embodiment, a Large Language Model (LLM) can be constructed based on the aforementioned correspondence tables. This LLM can utilize raw data of speech, images, and touch pressure generated in different testing scenarios (timestamped to avoid cross-modal association confusion and ensure learnable cross-modal associations) as training data, and be trained using supervised learning. Specifically, the LLM can be constructed based on GPT-4V, Gemini Ultra, Llama 3.2Vision, AnyMAL, or other Multimodal Large Language Models (MLLMs). Optionally, the LLM can also be constructed based on MLLM combined with a CNN (Convolutional Neural Network), where the CNN is mainly used for front-end feature extraction, and the MLLM is used for back-end decision-making. Workstation 30 can use this LLM to generate personalized testing plans for different testers. Furthermore, workstation 30 can also use this LLM to obtain test results. Optionally, in a specific embodiment of this application, the LLM may include: a test plan generation module, a result analysis module, and a reporting and suggestion module.

[0085] The test plan generation module is used to call the built-in test rule database and generate a personalized test plan based on the tester's personal information. This test plan includes test guidance information and test rules. These test rules may include, but are not limited to, any combination of one or more of the following: test type, test path, judgment criteria, path adjustment rules, and identification information of the test board to be used that matches the tester's personal information.

[0086] It is understandable that different test panels, touch pressure, pattern type, test order, test path, etc., need to be used for different test groups. For example, for patients with diabetic peripheral neuropathy, test panels with larger patterns need to be selected to avoid loss of perception due to tactile impairment.

[0087] The test plan generation module specifically includes an input layer, a rule engine layer, an LLM interaction layer, and an output layer. The input layer transforms the input personal information into vectors using a feature extractor. The rule engine layer includes a test rule base related to personal information (e.g., age-related rules, disease-related rules, gender-related rules, test purpose-related rules, etc.) and a logical reasoning unit (used to generate initial test parameters based on the test rule base). The LLM interaction layer includes an encoder and a decoder. Its inputs are personal information and rule parameters (e.g., may include, but are not limited to, model behavior control, clinical standard constraint parameters for test plan configuration and result evaluation, data processing logic, and output format specifications), and its output is a test plan (e.g., a suggested test order: first touch the graphic with the opening to the left, then touch the graphic with the opening to the right).

[0088] This results analysis module is used to analyze the acquired images and / or speech, and obtain test results based on the analysis results. Specifically, this results analysis module may include: a speech data processing subnetwork, an image data processing subnetwork, a multimodal fusion layer, and an LLM inference layer.

[0089] The input to this speech data processing subnetwork is a speech waveform (such as the tester's speech). This subnetwork performs noise reduction, framing, Mel-spectrogram transformation or MFCC feature extraction (capturing spectral and temporal characteristics) on the speech, and outputs a speech semantic feature vector. This subnetwork can adopt a network architecture combining CNN and Transformer. The CNN (convolutional layer) is used to extract local spectral features (such as frequency distribution and energy changes), and the Transformer encoder is used to capture speech temporal dependencies (such as pauses between words and changes in speech rate) through self-attention.

[0090] The image data processing subnetwork takes the acquired image as input and performs image normalization, edge detection, and ROI (Region of Interest) extraction (focusing on the graphic region) on the object. It then outputs graphic category feature vectors and visual features (such as similarity scores and error type labels). This subnetwork can employ a network architecture combining CNN and object detection / segmentation (e.g., ResNet50+FPN). The CNN backbone (ResNet50) is used to extract spatial features of the graphic (such as contours, angles, and textures), the object detection head (e.g., YOLO / SSD) is used to locate and classify the graphic, and auxiliary branches are used to analyze the similarity between the graphic touched by the tester and the graphic indicated by the test guidance information (feature distance can be calculated using a Siamese network).

[0091] This multimodal fusion layer is used to fuse features from speech and images to capture cross-modal correlations (e.g., whether speech rate slows down when image recognition is delayed).

[0092] This LLM inference layer is used to output structured analysis results based on multimodal synthesis features and the above-mentioned corresponding tables.

[0093] The Report and Recommendations module is used to generate a structured report that conforms to clinical standards based on the structured analysis results output by the Results Analysis module. This report includes test conclusions, anomaly analysis, and functional repair recommendations.

[0094] The LLM can be configured on workbench 30 or a cloud server. Workbench 30 can output identification information of the test board to be used, matching the tester's personal information, through the human-computer interaction display 40, so that the user can select the correct test board for testing.

[0095] Furthermore, the system may also include a test board rack and a robotic arm. The workbench 30 can send control commands to the robotic arm containing identification information of the test boards to be used, so as to control the robotic arm to take out the corresponding test board from the test board rack and place it at the position specified by the control command. When multiple test boards are needed, the robotic arm can take out the corresponding test boards from the test board rack and place them at the designated positions in sequence according to the time specified by the control command.

[0096] Furthermore, the system may also include a pressure sensor configured on the test touch area to acquire the tactile force exerted by the test subject when touching the test touch area and transmit it to the worktable. This tactile force is used to identify whether the test subject is experiencing psychological stress rather than organic tactile impairment, thereby reducing misjudgment.

[0097] The LLM also includes a pressure data processing subnetwork to analyze pressure changes during the test subject's operation (such as the force of touching the test panel) to reflect reaction intensity and stability (such as large pressure fluctuations when tense). The input to this subnetwork is time-series data from the pressure sensor (such as pressure value, pressure change rate, and duration of action, with a sampling frequency of 100Hz). This subnetwork performs preprocessing on the data, including data denoising, normalization (to eliminate individual force differences), and key event extraction (such as pressure peak moments and pressure abrupt change points), and then outputs a pressure time-series feature vector (reflecting the correlation between force, stability, and reaction speed).

[0098] The aforementioned multimodal fusion layer is also used to fuse features such as speech, image, and touch pressure to capture cross-modal associations.

[0099] In another embodiment, the tester used Figure 3The test board is tested. The workstation 30 uses the optical sensor 10 to capture the direction of the notch of the test touch part touched by the tester's finger. The test touch part touched by the tester's finger according to the test instruction is compared with the correct test path of the test board to determine whether the tester's perception is accurate. The smallest notch size that the tester can accurately identify is obtained. The time for the tester to complete the test of each test touch part and the total time to complete the test of the entire test board are counted. Combining the minimum notch size perceived, the perception time, and the perception result, the tester's fingertip tactile perception evaluation is judged.

[0100] In this embodiment, by transforming the subjective sensation of the test subject's finger perception into an objective motion state that can be recorded and analyzed, and combining it with intelligent hand movement recognition technology, autonomous and intelligent sensory function evaluation can be achieved. This effectively reduces reliance on professional assessors, greatly alleviates the current shortage of clinical hand surgery medical personnel, and is especially beneficial for the rehabilitation needs of patients living in economically and health-inadequate areas. It also reduces misdiagnosis, missed diagnosis, and complications caused by the lack of access to medical resources during the course of their disease, and to a certain extent saves on medical costs.

[0101] The embodiments of this application can visualize and quantify the test subject's subjective feedback. By moving the finger (or verbally stating the four directions "up, down, left, right"), it avoids simple "yes or no" answers, effectively preventing the test subject from "guessing correctly" and improving the accuracy of the test.

[0102] The embodiments of this application use a unified evaluation standard to automatically assess test subjects by computer, which can reduce inter-individual differences caused by different physicians' professional levels and differences before and after repeated diagnoses and functional assessments by the same physician, thereby improving the accuracy and consistency of hand function evaluation and follow-up work.

[0103] See Figure 10 This application also provides a method for testing hand sensory function, which assesses hand sensory function using the aforementioned hand sensory function testing system. The method includes the following steps:

[0104] S301. Obtain the tester's personal information and determine the test plan based on the personal information using a large language model;

[0105] The testing plan includes testing guidance information, and the personal information includes the tester's age, gender, medical history, and testing purpose.

[0106] S302. Control the human-machine interface display to send test guidance information, instructing the tester to place the correct test finger on the test touch part at the test start position of the test board;

[0107] In the hand sensory function testing system, the workstation controls the human-computer interaction display to send test guidance information, so that the tester can place the correct test finger on the test touch part of the test board at the test starting position according to the test guidance information.

[0108] The optical sensor in the hand sensory function testing system can acquire images of the tester's hand movements. The workstation operator can determine whether the tester has placed the correct test finger on the test touch area at the test start position based on the hand movement image. If it has been placed correctly, the human-computer interaction display is controlled to issue a test command. The tester then begins the test according to the test rules and the test start command in the test command.

[0109] The test board includes a board body and a plurality of test touch portions protruding from the upper surface of the board body, each of the test touch portions having at least one directional notch. Its specific structure is described in the above embodiment and will not be repeated here.

[0110] S302. Acquire images of the test subject's hand movements using an optical sensor, obtain real-time finger positions and finger movements based on these images, and obtain test results based on the finger positions and finger movements.

[0111] The workstation acquires images of the test subject's hand movements using optical sensors. Based on these images, it obtains real-time finger positions and movements. Then, based on these finger positions and movements, it calculates the minimum size of the test touch portion that the test subject can perceive during the test, the single perception time penalty, and the perception error number penalty. Following a preset algorithm, the test result is obtained based on the minimum size of the perceptible test touch portion, the single perception time penalty, and the perception error number penalty.

[0112] Optionally, in another embodiment of this application, the workstation can input hand motion images into the aforementioned large language model, and use the large language model to recognize the hand motion images to obtain real-time finger positions and finger movements. Based on the test plan determined in step S301, the real-time finger positions and finger movements are evaluated, and the final test result is obtained based on the evaluation results.

[0113] Optionally, the large language model can also dynamically adjust the test path based on the path adjustment rules in the test plan during the evaluation process and output the adjusted test path. The workstation generates new test guidance information based on the adjusted test path and controls the human-computer interaction display to send out the new test guidance information, guiding the tester to continue the test according to the adjusted test path, and returns to step S302 until the test is completed.

[0114] Furthermore, the workstation can also display the test results through a human-computer interaction display, and / or send the test results to a server or a designated terminal (such as a medical staff member's mobile phone).

[0115] For details on the specific testing process, please refer to the above description of the hand sensory function testing system; it will not be repeated here.

[0116] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0117] In this application, unless otherwise expressly specified and limited, the first feature being "on" or "below" the second feature may be in direct contact with the first feature and the second feature, or indirect contact between the first feature and the second feature through an intermediate medium.

[0118] Furthermore, "above," "on top of," and "above" the first feature in relation to the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. Similarly, "below," "under," and "beneath" the first feature in relation to the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0119] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0121] The above is a description of the hand sensory function testing system, test board, and testing method provided in this application. For those skilled in the art, based on the ideas of the embodiments of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A hand sensory function testing system, characterized in that, This includes a test board, optical sensors, system framework, workstation, and human-computer interaction display; The test board includes a board body and a plurality of test touch portions protruding from the upper surface of the board body, each of the test touch portions having at least one directional notch; The workstation is used to acquire the tester's personal information and determine a test plan based on the personal information using a large language model. The test plan includes test guidance information, and the personal information includes the tester's age, gender, medical history, and test purpose. The human-computer interaction display is connected to the workstation and is used to issue test guidance information under the control of the workstation, guiding the tester to place the correct test finger on the test touch part of the test board at the test start position. The optical sensor is mounted on the system frame and connected to the workstation. It is used to acquire images of the test subject's hand movements and send the images to the workstation for analysis. The workstation is used to obtain real-time finger position and finger movement through the hand motion image, and to obtain test results based on the finger position and finger movement; The test touch portion includes a first touch portion located at the test start position and a second touch portion located at the test end position. There is a test correct path between the first touch portion and the second touch portion. The test correct path includes multiple passing touch portions, so that the tester, according to a preset test instruction, touches the second touch portion from the first touch portion by touching along the test correct path based on the notch direction of the first touch portion and the notch direction of the multiple passing touch portions.

2. The hand sensory function testing system according to claim 1, characterized in that, The notch of the first touch portion points in the opposite direction to the path of the touch portion closest to itself; In the direction from the first touch portion to the second touch portion, the direction opposite to the notch direction of each of the passable touch portions of the test correct path points sequentially to the next adjacent passable touch portion, and the direction opposite to the notch direction of the passable touch portion closest to the second touch portion points to the second touch portion.

3. The hand sensory function testing system according to claim 1, characterized in that, The notch of the first touch portion points towards the touch portion that is closest to it. In the direction from the first touch portion to the second touch portion, the notch direction of each of the passing touch portions of the test correct path points sequentially to the next adjacent passing touch portion, and the notch direction of the passing touch portion closer to the second touch portion points to the second touch portion.

4. The hand sensory function testing system according to claim 1, characterized in that, The size of the first touch portion is larger than the size of the second touch portion.

5. The hand sensory function testing system according to claim 1, characterized in that, The notch direction of the test touch part is randomly set to one of four directions relative to the test touch part itself: up, down, left, and right.

6. The hand sensory function testing system according to claim 1, characterized in that, The workstation is also used to obtain the minimum size of the test touch part that the test subject can perceive in this test, the single perception time penalty, and the perception error number penalty based on the finger position and the finger movement, and to obtain the test result based on the minimum size of the test touch part that can be perceived, the single perception time penalty, and the perception error number penalty according to a preset algorithm.

7. A hand sensory function testing board, characterized in that, The test board includes: The plate body and a plurality of test touch portions protruding from the upper surface of the plate body; Each of the aforementioned test touch portions has at least one directional notch; The test touch portion includes a first touch portion located at the test start position and a second touch portion located at the test end position. There is a test correct path between the first touch portion and the second touch portion. The test correct path includes multiple passing touch portions, so that the tester, according to a preset test instruction, touches the second touch portion from the first touch portion by touching along the test correct path with their finger based on the notch direction of the first touch portion and the notch direction of the multiple passing touch portions.

Images