Visual function auxiliary instrument and control method thereof

Abstract

The invention provides a visual function auxiliary instrument and a control method thereof, and belongs to the field of medical instruments. The visual function auxiliary instrument comprises a wearable structure, a distributed touch unit, an attitude sensor, an eye movement module, a binocular vision module and a processing module. The processing module controls the distributed tactile units to output different tactile signals according to the obtained spatial position information of the target object, and a patient can perceive the spatial position of the object according to the intensity and difference of the tactile signals at different positions. The invention further provides a control method of the visual function auxiliary instrument. Compared with an existing technical scheme, the method has the advantages that the patient can intuitively perceive the spatial position information of the object, the learning cost is reduced, the response is quicker, and a new scheme is provided for life assistance and visual function rehabilitation of the patient with the abnormal visual function.

Description

Technical Field

[0001] This invention relates to the field of medical devices, specifically to an assistive device and its control method for transmitting spatial location information to patients with visual impairments through tactile information. Background Technology

[0002] Binocular vision dysfunction refers to the inability of both eyes to coordinate, synchronize, and comfortably work together to form a single, clear, stereoscopic vision. The key aspect of binocular vision dysfunction is a problem with the two eyes adjusting their focus, affecting binocular coordination and making it impossible to accurately align with the same target point. Patients with stereopsis dysfunction cannot normally perceive depth, and their ability to perceive three-dimensional space is impaired, making it difficult to judge the location and distance of objects, causing inconvenience to their daily lives and work.

[0003] Current treatment options mainly include: directly correcting eye alignment and restoring the basis for binocular vision through methods such as eye muscle surgery; using optical correction devices, such as prism glasses and triangular prisms, to adjust parallax by changing light refraction and improve binocular coordination; and stimulating binocular coordination and improving fusion stereoscopic vision through visual training such as 3D visual training systems, red-green bars, and stereo cards.

[0004] In addition, to address the problem of being unable to judge the location and distance of objects due to stereopsis abnormalities, related technologies offer several auxiliary methods to convey this visual information to patients through other senses. A common method is using voice prompts; however, voice prompts suffer from low information content and slow comprehension, requiring patients to have a longer reaction time to verbal cues regarding spatial location. For patients with visual impairments, who rely more on auditory information in daily life and have a more sensitive auditory system, using voice prompts can interfere with their judgment of other auditory information. Related technologies also include tactile signals, such as multi-point vibration designs, to transmit spatial location information to patients. For these technologies, patients need to memorize the information corresponding to different tactile signals, increasing the learning cost and reaction time, and making it difficult to provide timely and accurate environmental information. Summary of the Invention

[0005] The purpose of this invention is to provide a visual function assistive device and its control method, which converts spatial position information such as the distance and orientation of objects in direct visual perception into tactile signals that patients can intuitively understand and perceive, thereby helping patients to easily and timely obtain this information.

[0006] The specific solution of the present invention is as follows: In a first aspect, the present invention provides a visual assistive device, comprising: Wearable structures are designed to be worn on the human body and to support other unit modules. A distributed tactile unit, which comes into contact with multiple symmetrically distributed parts of the human body, and transmits differentiated tactile signals to different parts of the human body; A posture sensor, located on the patient's head, is used to determine the orientation of the patient's head. An eye-tracking module, located around the eyes, is used to identify features that the patient is focusing on in real time; A binocular vision module, located within the wearable structure, can identify the distance and orientation of features and output the relevant information to the processing module; and The processing module is connected to the attitude sensor, eye-tracking module and binocular vision module and receives information to control the distributed tactile units.

[0007] The distributed tactile unit includes multiple tactile units symmetrically distributed on the human body. Different distributed tactile units output different signals to allow the patient to perceive direction.

[0008] The distributed tactile unit includes a left tactile unit and a right tactile unit, which are respectively connected to the processing module and receive different signals.

[0009] The left and right tactile units are symmetrically distributed on the human body.

[0010] In one embodiment, the number of left tactile units is one or more, and the number of right tactile units is one or more.

[0011] In one embodiment, the number of the left tactile units is the same as the number of the right tactile units.

[0012] In one embodiment, the left tactile unit is arranged on the left side of the human head, and the right tactile unit is arranged on the right side of the human head.

[0013] In one embodiment, the left tactile unit is arranged on the left side of the human body, and the right tactile unit is arranged on the right side of the human body.

[0014] The signals from the left and right tactile units are different and are independently controlled by the processing module.

[0015] Specifically, when the feature is oriented closer to one side, the signal of the distributed tactile units in that direction is strengthened, while the signal of the distributed tactile units in the opposite direction is weakened. The more the feature is oriented to one side, the more pronounced the signal strengthening and weakening become.

[0016] In one embodiment, the distributed tactile unit further includes an upper tactile unit. The upper tactile unit is located above the left and right tactile units.

[0017] The signals from the upper tactile unit are independently controlled by the processing module.

[0018] In one embodiment, the upper tactile unit is positioned on the upper side of the human head.

[0019] Preferably, the upper tactile unit is arranged on the forehead.

[0020] Specifically, when the feature is oriented upwards, the signal from the upper tactile unit is strengthened; when the feature is oriented downwards, the signal from the upper tactile unit is weakened. The more the feature is oriented in a different direction, the more pronounced the strengthening and weakening of the signal becomes.

[0021] In one embodiment, the distributed tactile unit further includes a lower tactile unit. The lower tactile unit is located below the left and right tactile units.

[0022] The signals from the lower tactile unit are independently controlled by the processing module.

[0023] In one embodiment, the lower tactile unit is positioned on the lower side of the human head.

[0024] Specifically, when the feature is oriented upwards, the signal of the lower tactile unit weakens; when the feature is oriented downwards, the signal of the lower tactile unit strengthens. The more the feature is oriented in a different direction, the more pronounced the weakening and strengthening of the signal becomes.

[0025] In one embodiment, the distributed tactile unit may also include an upper tactile unit and a lower tactile unit.

[0026] In one embodiment, the visual function assistive device adopts an integrated design.

[0027] In one embodiment, the wearable structure may take the form of glasses, headphones, hats, necklaces, headbands, or other similar structures.

[0028] In one embodiment, the visual assistive device adopts a split design, comprising multiple wearable structures. The wearable structures may be in the form of one or a combination of the above-described types.

[0029] The wearable structure is used to fix and integrate other functional modules, including a posture sensor, an eye-tracking module, a binocular vision module, and a processing module.

[0030] The posture sensor is fixed to the head via the wearable structure and remains relatively stationary with respect to the patient's head, thereby obtaining the patient's head position information.

[0031] In one embodiment, the attitude sensor is an inertial sensor, including one or more of an accelerometer, gyroscope, magnetometer, etc.

[0032] In one embodiment, other sensors can be used to obtain more specific and accurate spatial location information of the object.

[0033] In one embodiment, the posture sensor can also assist in scene identification by detecting head and body movements.

[0034] The eye-tracking module is fixed in front of the eyes via the wearable structure, thereby obtaining the patient's eye movement information.

[0035] The eye-tracking module detects eye movements and transmits the information to the processing module, thereby identifying the feature object that the patient is focusing on.

[0036] In one embodiment, the eye-tracking module can accurately identify the patient's gaze point and identify the feature object the patient is focusing on in real time by using a light source, an infrared camera, and an algorithm to track eye movements and gaze direction in real time.

[0037] In one embodiment, the binocular vision module is a module based on dual cameras to simulate human stereoscopic vision.

[0038] The binocular vision module calculates the distance and orientation of the feature object that the patient is focusing on by utilizing binocular parallax information.

[0039] The binocular vision module includes a left camera and a right camera. The left camera can be located on the left side of the wearable structure, and the right camera can be located on the right side of the wearable structure.

[0040] The processing module converts the calculated distance and orientation information into tactile signals that the patient can directly perceive.

[0041] The tactile signals should be based on ergonomic design so that patients can perceive them intuitively and make quick judgments.

[0042] Preferably, the intensity of the tactile signal is inversely correlated with the distance. The closer the feature is to the patient, the stronger the tactile signal, and vice versa.

[0043] The distributed tactile unit is fixed to the patient through the wearable structure and keeps in contact with the patient's skin, thereby outputting tactile signals.

[0044] The distributed tactile unit transmits tactile signals from the processing module to the patient through physical vibrations or pressure changes, allowing the patient to perceive the distance, orientation, and spatial location of the target object through touch instead of direct vision.

[0045] In one embodiment, the distributed tactile unit can output tactile perception to the patient based on one or more methods such as vibration tactile perception, force tactile perception, and electrotactile perception, thereby replacing visual perception to convey information to the patient.

[0046] Specifically, the distributed tactile unit can generate force by using actuators such as motors, electromagnetic or piezoelectric actuators through communication signals transmitted from the processing module, thereby producing vibrations of different frequencies or intensities.

[0047] Preferably, the closer the distance, the greater the intensity or vibration frequency.

[0048] In one embodiment, the distributed tactile unit includes a motor, a linkage mechanism, and a pressure pad. The motor drives the pressure pad to move via the linkage mechanism, applying different pressure forces to the patient.

[0049] In one implementation, the distributed tactile unit is based on force tactile sensation, simulating the mechanical sensation of real objects interacting with skin and muscles through controllable mechanical force stimulation.

[0050] In one embodiment, the distributed haptic unit includes a motor and a counterweight. The motor may be a rotor motor or a linear motor.

[0051] In one embodiment, the distributed tactile unit is based on vibration tactile sensation, with a motor and a vibrating head placed inside the wearable structure, and the motor driving the vibrating head to achieve vibration.

[0052] In one embodiment, the distributed tactile unit is based on electrical pulse technology, using electrical current to stimulate the skin.

[0053] Furthermore, the visual function assistive device provided by the present invention also includes a communication module, which is used for signal transmission between the various unit modules.

[0054] Furthermore, the visual function assistive device provided by the present invention also includes necessary electronic components such as a power supply.

[0055] Secondly, the present invention provides a control method for a visual assistive device, characterized by comprising the following steps: Step 1: The posture sensor acquires the posture information of the patient's head and transmits it to the processing module to establish a spatial coordinate system based on this information. Step 2: The eye-tracking module detects the movement of the human eye and transmits the information to the processing module to determine the feature object that the patient is focusing on, as well as its approximate range in the spatial coordinate system. Step 3: The binocular vision module calculates the distance L and orientation θ and α of the feature object that the patient is focusing on by using binocular parallax information, and transmits it to the processing module; Step 4: The processing module determines the reference intensity of the tactile signal based on the distance L of the focused feature. Based on the orientation of the focused feature, determine the adjustment value of the tactile signal. and ; Step 5, the processing module determines the reference intensity of the tactile signal. and adjustment value and Determine the signal strength of each distributed haptic unit. , The output is sent to the corresponding distributed haptic unit; Step 6: The distributed tactile unit outputs directly perceptible tactile sensations to the patient based on the received signal strength.

[0056] In step 1, the patient's head posture information includes pitch angle, roll angle, and yaw angle.

[0057] In step 1, the spatial coordinate system is a spatial polar coordinate system.

[0058] Furthermore, the specific steps of step 2 include: Step 2.1: Capture a sequence of dynamic eye images using a high-speed camera and analyze the characteristics of human eye movements; Step 2.2: The analyzed eye movement information is transmitted to the processing module. Based on information such as gaze persistence time and pupil changes, the characteristic object that the patient is focusing on is determined. Step 2.3: Map the coordinates of the feature in the image to a three-dimensional spatial coordinate system to determine its approximate range in the spatial coordinate system.

[0059] In step 2, deep learning algorithms such as AI image recognition can be used to analyze the patient's eye movement characteristics, thereby improving the reliability of the information obtained.

[0060] In step 3, the binocular vision module can obtain depth information by simultaneously capturing two images, left and right, and calculating the parallax. It can reconstruct three-dimensional scenes, measure object distances, and identify spatial relationships in real time, providing stereoscopic vision capabilities similar to those of humans.

[0061] Furthermore, step 3 includes the following specific steps: Step 3.1: Simultaneously capture two images using the left and right cameras, the images containing features identified by the eye-tracking module; Step 3.2: Locate the corresponding pixels of the feature in the two images, and calculate the distance L of the feature based on the principle of triangulation; Step 3.3: Calculate the horizontal deflection angle θ and vertical deflection angle α of the feature relative to the front of the user.

[0062] In step 3, existing binocular vision algorithms can also be used to calculate distance and orientation.

[0063] In step 4, the reference intensity of the tactile signal It is inversely correlated with distance L.

[0064] In one implementation, the reference strength is: Where A is a parameter that is adjusted based on the patient's sensitivity to touch.

[0065] In one implementation, the reference strength is: Where 'a' is a parameter, adjusted based on the patient's sensitivity to touch, and 0 <a<1。

[0066] Preferably, a distance L to a reference strength can be created. The mapping curve is calculated, and the mapping relationship is adjusted according to the patient's perception.

[0067] In step 4, the adjustment value of the tactile signal It is related to the orientation θ and α.

[0068] In one implementation, the adjustment value of the tactile signal is: In one implementation, the adjustment value of the tactile signal is: Where C is a parameter that is adjusted based on the patient's sensitivity to touch. These are parameters that can be adjusted according to the required field of view.

[0069] Preferably, adjustments can be made from the horizontal deflection angle θ and the vertical deflection angle α. and The mapping curve is calculated, and the mapping relationship is adjusted according to the patient's perception, with the optimal value being the one that the patient can perceive most intuitively.

[0070] In one implementation, considering the different perceptions of up and down and left and right by the human body, adjustments can be made from the horizontal deflection angle θ to the adjustment value. The mapping curve, and the vertical deflection angle α to the adjustment value. The mapping curves are adjusted according to the patient's sensory sensitivity.

[0071] In step 5, different signal strengths correspond to different distributed tactile units (i = 1, 2, ...). The signal strength is determined by the orientation of the focused feature.

[0072] Specifically, it should be determined based on the changes in the human body's perception of position after the corresponding distributed tactile units output tactile signals to the human body.

[0073] In step 5, the signal strength of each distributed tactile unit is compared with the reference strength. They show a positive correlation.

[0074] Preferably, the signal strength of each distributed tactile unit is equal to the reference strength. The relationship is linear.

[0075] In step 5, the signal strength and adjustment value of each distributed tactile unit are... and The relationship is determined by the position of the corresponding distributed tactile units on the human body.

[0076] Preferably, the signal strength and adjustment value of each distributed tactile unit and The relationship is linear.

[0077] In one implementation, the signal strength of each distributed tactile unit is: Where n is a constant, and Q is the transformation matrix, determined by the arrangement of the distributed haptic units. For each distributed haptic unit, there exists a unique [ ] is used to indicate its response to a signal.

[0078] Specifically, Determined based on the sensitivity of each distributed tactile unit, The arrangement is determined by the left and right layout of the distributed tactile units. The arrangement is determined by the vertical arrangement of the distributed tactile units.

[0079] Preferably, the signal strength from each distributed haptic unit can be created. Regarding the reference strength and adjustment value and The mapping relationship is established and adjusted based on the patient's perception.

[0080] In one implementation, the signal strength from the distance L and orientation θ and α of the feature to each distributed tactile unit can also be directly created. The mapping relationship.

[0081] The mapping relationship is determined based on the rules in steps 4 and 5.

[0082] Specifically, within a three-dimensional interval consisting of a certain distance range L and orientation ranges θ and α, several points are discretely selected, and the signal strength of each distributed tactile unit i is calculated to create a mapping table.

[0083] During control, for any distance L and orientation θ and α, the signal strength of the corresponding distributed tactile unit i can be obtained by directly looking up a table and fitting the data. .

[0084] Specifically, in step 6, the processing module circuit is connected to the distributed haptic units, and the signal strength of each distributed haptic unit is calculated. And control the physical tactile signals of the distributed tactile units.

[0085] The distributed haptic unit controls the internal mechanical control device to output haptic signals based on the received signal strength.

[0086] In one implementation, the distributed tactile unit in step 6 transmits the tactile signals from the processing module to the patient through mechanical movements such as physical vibration or pressure changes, allowing the patient to perceive the distance, orientation, and spatial location information of the target object through touch instead of direct vision.

[0087] The advantages or beneficial effects of the above technical solutions include at least the following: This invention discloses a visual assistive device that transmits spatial location information to eye patients through tactile information, enabling them to use tactile perception instead of direct visual perception. Unlike existing technologies that use voice prompts, this invention avoids the problem of voice information interfering with the patient's acquisition of other auditory information.

[0088] This invention provides a visual function assistive device that utilizes the difference in tactile signals between the left and right sides to simulate the binocular effect, providing a more intuitive reflection of spatial location information. Compared to traditional methods that use touch to transmit specific signals, this reduces the burden on patients who need to actively memorize tactile signals. Simultaneously, it improves the timeliness of patients' perception of spatial location information through touch.

[0089] The visual function assistive device of the present invention also adds upper and lower tactile signals, expanding the dimension of the patient's sensing space range.

[0090] The present invention provides a visual function assistive device that helps patients continuously verify the spatial position information of objects, which can help patients form correct spatial cognition and promote their rehabilitation.

[0091] The present invention provides a visual function assistive device with various wearable structural forms, which can discreetly provide assistance to patients with visual function abnormalities and meet the diverse usage needs and wearing habits of patients.

[0092] The present invention provides a visual function assistive device, which adopts a split design and includes multiple wearable structures. Patients can choose the type of wearable structure to meet their diverse needs.

[0093] The present invention discloses a control method for a visual function assistive device, which acquires the posture information of the patient's head through a posture sensor and establishes a coordinate system from the user's perspective, so that the given position information is closer to the patient's cognition, which helps the patient's cognition and rehabilitation.

[0094] The present invention discloses a control method for a visual function assistive device, which accurately captures the patient's target through an eye-tracking module, improving the accuracy of recognition, eliminating the need for the patient to consciously make selections, and reducing the patient's learning cost.

[0095] The present invention discloses a control method for a visual function assistive device, which uses a binocular vision module to determine the distance and orientation of a target feature. This method is closest to the working principle of the human eye, making it easier for patients to accept this assistive means.

[0096] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of this application will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0097] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0098] Figure 1 This is a schematic diagram of a visual function assistive device according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a visual function assistive device according to an embodiment of the present invention worn on the human body; Figure 3 This is a schematic diagram of a visual function assistive device (split type) according to an embodiment of the present invention; Figure 4 This is a control method for a visual function assistive device according to an embodiment of the present invention; Figure 5 This is a signal intensity diagram of the left and right tactile units in the control method of a visual function assistive device according to an embodiment of the present invention; Figure 6 This is a signal intensity diagram of the left and right tactile units in the control method of a visual assistive device according to another embodiment of the present invention.

[0099] Among them, 10 is the wearable structure, 20 is the distributed haptic unit, 21 is the left haptic unit, 22 is the right haptic unit, 23 is the upper haptic unit, 24 is the lower haptic unit, 30 is the posture sensor, 40 is the eye-tracking module, 50 is the binocular vision module, and 60 is the processing module. Detailed Implementation

[0100] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0101] Please refer to Figure 1 A preferred embodiment of the present invention provides a visual function assistive device, including a wearable structure 10, a distributed tactile unit 20, a posture sensor 30, an eye-tracking module 40, a binocular vision module 50, and a processing module 60.

[0102] The wearable structure 10 is worn on the human body and carries other unit modules.

[0103] In one embodiment, the visual function assistive device adopts an integrated design.

[0104] In one embodiment, the wearable structure 10 may take the form of a hat, helmet, headband, headband, or other head-mounted structure.

[0105] In one embodiment, the wearable structure 10 may take the form of glasses, headphones, or other similar structures.

[0106] In one embodiment, the wearable structure 10 may be in the form of a necklace or a neckband.

[0107] In one embodiment, the wearable structure 10 may take the form of a belt or clothing.

[0108] In one embodiment, the wearable structure 10 can take the form of a bracelet, watch, ring, or other structure distributed on the left and right sides of the body.

[0109] Preferably, in this embodiment, the wearable structure 10 adopts the structure of smart glasses.

[0110] The wearable structure 10 is used to fix and integrate other functional modules, including a posture sensor 30, an eye-tracking module 40, a binocular vision module 50, and a processing module 60.

[0111] The wearable structure 10 is designed to ensure that the functional modules maintain a rigid connection or close fit with the patient's head or other parts during use, thereby minimizing the relative movement between the two.

[0112] Please refer to Figure 2 The posture sensor 30 is located on the patient's head and is used to determine the patient's head orientation; the eye-tracking module 40 is located in front of the eyes and is used to identify the features that the patient is focusing on in real time; the binocular vision module 50 is located in the wearable structure 10 and can identify the distance and orientation of the features and output the relevant information to the processing module 60; the processing module 60 is connected to the posture sensor 30, the eye-tracking module 40 and the binocular vision module 50 and receives information to control the distributed tactile unit 20.

[0113] The wearable structure 10 is also used to support the distributed haptic unit 20 and provide support and reaction force for the distributed haptic unit 20.

[0114] The distributed tactile unit 20 comes into contact with the skin and transmits tactile signals to the human body.

[0115] The distributed tactile unit 20 includes multiple tactile units symmetrically distributed on the human body. Through the signal differences between different distributed tactile units, the patient can perceive direction.

[0116] The distributed tactile unit 20 includes a left tactile unit 21 and a right tactile unit 22, which are symmetrically distributed on the human body.

[0117] In this embodiment, there is one left tactile unit 21 and one right tactile unit 22.

[0118] In one embodiment, the left tactile unit 21 is arranged on the left side of the human head, and the right tactile unit 22 is arranged on the right side of the human head.

[0119] Preferably, the plurality of distributed tactile units 20 are arranged at the temples on the left and right sides of the human head.

[0120] In one embodiment, the plurality of distributed tactile units 20 are arranged behind the left and right ears of the human body.

[0121] Preferably, the plurality of distributed tactile units 20 are arranged behind the earlobes of the left and right ears of the human body, in the depressions close to the front of the mastoid process, so as to perceive tactile sensation more promptly and sensitively.

[0122] Please refer to Figure 3 In one embodiment, the visual function assistive device adopts a split design, including multiple wearable structures.

[0123] The left tactile unit 21 is arranged on the left side of the human body, and the right tactile unit 22 is arranged on the right side of the human body.

[0124] In one embodiment, the left tactile unit 21 is arranged on the left forearm, and the right tactile unit 22 is arranged on the right forearm.

[0125] In one embodiment, the left tactile unit 21 is arranged on the left shoulder, and the right tactile unit 22 is arranged on the right shoulder.

[0126] In one embodiment, the left tactile unit 21 is arranged on the left leg, and the right tactile unit 22 is arranged on the right leg.

[0127] The left tactile unit 21 and the right tactile unit 22 are respectively connected to the processing module 60 via a wireless module. The signals of the left tactile unit 21 and the right tactile unit 22 are different and are independently controlled by the processing module 60.

[0128] Specifically, when the feature is oriented to the left, the signal of the left tactile unit 21 is strengthened, and the signal of the right tactile unit 22 is weakened. When the feature is oriented even further to the left, the signal of the left tactile unit 21 is further strengthened, and the signal of the right tactile unit 22 is further weakened.

[0129] Specifically, when the feature is oriented to the right, the signal of the left tactile unit 21 weakens, while the signal of the right tactile unit 22 strengthens. When the feature is oriented even further to the right, the signal of the left tactile unit 21 weakens further, while the signal of the right tactile unit 22 strengthens further.

[0130] In one embodiment, the distributed tactile unit 20 further includes an upper tactile unit 23. The upper tactile unit 23 is located above the left tactile unit 21 and the right tactile unit 22.

[0131] In one embodiment, the upper tactile unit 23 is positioned on the upper side of the human head.

[0132] Preferably, the upper tactile unit 23 is arranged on the forehead.

[0133] In one embodiment, the upper tactile unit 23 is arranged at the crown of the head.

[0134] The upper tactile unit 23 is connected to the processing module 60, and the signal of the upper tactile unit 23 is independently controlled by the processing module 60.

[0135] Specifically, when the feature is oriented upwards, the signal of the upper tactile unit 23 is strengthened; when the feature is oriented even higher, the signal of the upper tactile unit 23 is further strengthened. When the feature is oriented downwards, the signal of the upper tactile unit 23 is weakened; when the feature is oriented even lower, the signal of the upper tactile unit 23 is further weakened.

[0136] In one embodiment, the distributed tactile unit 20 further includes a lower tactile unit 24. The lower tactile unit 24 is located below the left tactile unit 21 and the right tactile unit 22.

[0137] In one embodiment, the lower tactile unit 24 is positioned on the lower side of the human head.

[0138] In one embodiment, the lower tactile unit 24 is arranged at the philtrum.

[0139] In one embodiment, the lower tactile unit 24 is arranged at the tip of the nose.

[0140] In one embodiment, the lower tactile unit 24 is arranged on the body below the human head.

[0141] The lower tactile unit 24 is connected to the processing module 60, and the signal of the lower tactile unit 24 is independently controlled by the processing module 60.

[0142] Specifically, when the feature is oriented upwards, the signal of the lower tactile unit 24 weakens; when the feature is oriented even higher, the signal of the lower tactile unit 24 weakens further. When the feature is oriented downwards, the signal of the lower tactile unit 24 strengthens; when the feature is oriented even lower, the signal of the lower tactile unit 24 strengthens further.

[0143] In one embodiment, it may also include an upper tactile unit 23 and a lower tactile unit 24.

[0144] In one embodiment, the distributed tactile unit 20 is based on force tactile sensation, generating the mechanical sensation of real objects interacting with skin and muscles through controllable mechanical force stimulation.

[0145] In one embodiment, the distributed tactile unit 20 includes a motor, a linkage mechanism, and a pressure plate. The motor is connected to the linkage mechanism, and the motor provides power to drive the pressure plate to move and generate extrusion force through the linkage mechanism.

[0146] In one embodiment, the linkage mechanism is a four-bar linkage.

[0147] In one embodiment, the linkage mechanism comprises a driving arm, a driven arm, and a connecting rod forming a planar four-bar linkage system, which are connected by hinges.

[0148] In one embodiment, the active arm is 80mm long, the driven arm is 60mm long, the linkage rod is 50mm long, and the hinge pin is 4mm in diameter, generating a vertical travel of ±8mm and a horizontal travel of ±5mm, forming an elliptical motion trajectory with a major axis of 16mm and a minor axis of 10mm.

[0149] In one embodiment, the distributed haptic unit 20 includes a rotor motor and a counterweight.

[0150] The distributed tactile unit 20 is based on vibration tactile sensation, relies on magnetic force as its principle, and uses the rotation of a rotor motor to drive the counterweight to vibrate.

[0151] In one embodiment, the distributed haptic unit 20 includes a linear motor and a counterweight.

[0152] The distributed tactile unit 20 is based on vibration tactile sensation, relies on magnetic force as its principle, and uses a linear motor to rapidly shake the counterweight to vibrate.

[0153] In one embodiment, the distributed tactile unit 20 is based on vibration tactile sensation, with a motor and a pair of spherical vibrating heads placed inside the wearable structure 10, and the motor drives the vibrating heads to achieve vibration.

[0154] In one embodiment, the distributed tactile unit 20 is based on force tactile sensation, which simulates the mechanical sensation of real objects interacting with skin and muscles through controllable mechanical force stimulation.

[0155] In one embodiment, the distributed tactile unit 20 is based on electrical pulse technology and uses modulated low- to medium-frequency pulsed current to stimulate the skin.

[0156] In one embodiment, the distributed tactile unit 20 simulates a tactile effect by altering the way neurons send signals by sending small, low-voltage electrical pulses to electrodes on the skin.

[0157] In one embodiment, the distributed tactile unit 20 stimulates muscle contraction by providing small, low-voltage electrical pulses to the muscles.

[0158] The posture sensor 30 is fixed to the head via the wearable structure 10 and remains relatively stationary with respect to the patient's head, thereby obtaining the patient's head position information.

[0159] In one embodiment, the attitude sensor 30 is an inertial sensor, including one or more of an accelerometer, gyroscope, magnetometer, etc.

[0160] The accelerometer is used to measure the linear acceleration of the head on three orthogonal axes, including the gravitational acceleration component, and can be used to calculate attitude angles.

[0161] The gyroscope is used to measure the angular velocity of the head around three orthogonal axes and is a key component for real-time tracking of attitude changes.

[0162] The magnetometer is used to measure the components of the magnetic field strength in the environment in which the head is located on three orthogonal axes. When fused with accelerometer and gyroscope data, it can significantly improve the long-term accuracy and stability of attitude calculation and correct the cumulative drift error of the gyroscope.

[0163] In one embodiment, the processing module 60 performs real-time fusion calculations on one or more raw data from accelerometers, gyroscopes, and magnetometers.

[0164] In one implementation, other sensors can be used to obtain more specific and accurate spatial location information of the object.

[0165] Specifically, the other sensors may include optical sensors, ultrasonic sensors, ultra-wideband modules, etc.

[0166] The data acquired by the other sensors can be fused with the inertial data of the attitude sensor 30 in the processing module 60 to output head spatial pose information with higher accuracy and less drift.

[0167] In one embodiment, the posture sensor 30 can also assist in scene recognition by detecting head and body movements.

[0168] Specifically, the posture sensor 30 is arranged in the wearable structure 10 to detect the patient's head and body movement parameters in real time, including offset angle, pitch angle and angular velocity, to help identify the scene.

[0169] Preferably, the sampling frequency of the attitude sensor 30 is not less than 20Hz, or a higher frequency is used to meet the real-time tracking requirements.

[0170] Furthermore, the visual function assistive device provided by the present invention also includes a communication module, which is used for signal transmission between the various unit modules.

[0171] Furthermore, the visual function assistive device provided by the present invention also includes necessary electronic components such as a power supply.

[0172] The attitude sensor 30 is connected to the processing module 60 through the communication module.

[0173] The attitude sensor 30 is powered by the power supply.

[0174] The eye-tracking module 40 is fixed in front of the eye via the wearable structure 10, thereby obtaining the patient's eye movement information.

[0175] The eye-tracking module 40 detects the movement of the human eyeball and transmits the information to the processing module 60, thereby identifying the feature object that the patient is focusing on.

[0176] In one embodiment, the eye-tracking module 40 can accurately identify the patient's gaze point and identify the feature object the patient is focusing on in real time by using a light source, an infrared camera, and an algorithm to track eye movements and gaze direction in real time.

[0177] Preferably, the eye-tracking module 40 can emit invisible infrared light with a wavelength of 850-940nm to the patient's eye area, and the infrared camera captures the corneal reflection point and pupil outline.

[0178] Preferably, the infrared camera has a sampling frequency of not less than 50Hz to meet the real-time tracking requirements.

[0179] In one embodiment, the eye-tracking module 40 may include an optical filter assembly to filter out ambient light interference in non-emission bands and improve the signal-to-noise ratio.

[0180] Preferably, the eye-tracking module 40 can extract algorithms through methods such as pupil contour enhancement and Purkinje spot tracking, and dynamically calibrate the model using personalized eye parameter calculation and spatial mapping function.

[0181] The eye-tracking module 40 is connected to the processing module 60 via the communication module.

[0182] The eye-tracking module 40 is powered by the power supply.

[0183] The binocular vision module 50 is a module that simulates human stereoscopic vision based on dual cameras.

[0184] The binocular vision module 50 achieves three-dimensional spatial perception through two spatially separated cameras.

[0185] The binocular vision module 50 calculates the distance and orientation of the feature object that the patient is focusing on by utilizing binocular parallax information.

[0186] The binocular vision module 50 transmits the calculated distance and orientation information to the processing module 60, thereby determining the specific location of the feature.

[0187] The binocular vision module 50 includes a left camera and a right camera.

[0188] In one embodiment, the left camera may be located on the left side of the wearable structure 10, and the right camera may be located on the right side of the wearable structure 10.

[0189] In one embodiment, the left camera may be located on the left side of the human body, and the right camera may be located on the right side of the human body.

[0190] Preferably, the camera of the present invention has angle or focal length limitations, which reduces costs and minimizes the capture of invalid information.

[0191] The binocular vision module 50 is connected to the processing module 60 through the communication module.

[0192] The binocular vision module 50 is powered by the power supply.

[0193] The processing module 60 converts the calculated distance and orientation information into tactile signals that the patient can directly perceive.

[0194] The tactile signals should be based on ergonomic design so that patients can perceive them intuitively and make quick judgments.

[0195] Preferably, the intensity of the tactile signal is inversely correlated with the distance.

[0196] The closer the object is to the patient, the stronger the tactile signal. The farther the object is from the patient, the weaker the tactile signal.

[0197] Specifically, the processing module 60 will output different signal types corresponding to different types of distributed haptic units 20.

[0198] In one embodiment, the circuit board of the processing module 60 controls the generation of pulse current, and simulates different tactile effects by modulating the frequency, waveform and intensity of the pulse.

[0199] In one embodiment, the processing module 60 will output the control signals required for different types of motors.

[0200] In an optional embodiment, the processing module 60 integrates a CPU, GPU, or dedicated chip to run environmental perception, data fusion, and decision-making algorithms, enabling it to process data from the attitude sensor 30, eye-tracking module 40, etc., in real time, and coordinate functions such as tactile feedback.

[0201] The distributed tactile unit 20 is fixed to the patient through the wearable structure 10 and keeps in contact with the patient's skin, thereby outputting tactile signals.

[0202] The distributed tactile unit 20 transmits the tactile signals from the processing module 60 to the patient through physical vibration or pressure changes, allowing the patient to perceive the distance, orientation, and spatial location information of the target object through touch instead of direct vision.

[0203] In one embodiment, the distributed tactile unit 20 can output tactile perception to the patient based on one or more methods such as vibration tactile perception, force tactile perception, and electrotactile perception, thereby replacing visual perception to convey information to the patient.

[0204] Specifically, the distributed tactile unit 20 can generate force through actuators such as motors, electromagnetic or piezoelectric actuators.

[0205] In one embodiment, the distributed tactile unit 20 can generate vibrations of different frequencies or intensities through communication signals transmitted from the processing module 60.

[0206] Generally, the closer the distance, the greater the vibration frequency or intensity.

[0207] In one embodiment, the power supply is divided into a main battery module and a separate power supply module.

[0208] In one embodiment, the main battery is integrated into the body of the wearable structure 10 and uses a lithium battery to balance energy density and lightweight design.

[0209] In one embodiment, the separate power supply module can independently power key modules such as the attitude sensor 30 and the communication module, avoiding system failure caused by a single point of failure.

[0210] In one embodiment, the separate power supply module can be made into a removable battery.

[0211] In one implementation, the communication module is divided into a wired module and a wireless module, which are responsible for data transmission and interaction between systems.

[0212] In one embodiment, the wired module is mounted on the wearable structure 10 to provide signal transmission and interaction between the posture sensor 30, eye-tracking module 40, binocular vision module 50 and processing module 60.

[0213] In one embodiment, the wireless module mainly uses wireless communication technology to achieve a short-range connection between the processing module 60 and the distributed haptic unit 20, allowing the distributed haptic unit 20 to adjust the feedback intensity in real time.

[0214] In one implementation, the wireless communication technology can be Bluetooth or WiFi communication.

[0215] Secondly, please refer to Figure 4The present invention provides a control method for a visual assistive device, comprising the following steps: Step 1: Obtain the patient's head posture information through the posture sensor and transmit it to the processing module to establish a spatial coordinate system based on it; Step 2: The eye-tracking module detects the movement of the human eye and transmits the information to the processing module to determine the feature object that the patient is focusing on, as well as its approximate range in the spatial coordinate system. Step 3: The binocular vision module calculates the distance L and orientation θ and α of the feature object that the patient is focusing on by using binocular parallax information, and transmits it to the processing module; Step 4: The processing module determines the reference intensity of the tactile signal based on the distance L of the focused feature. Based on the orientation of the focused feature, determine the adjustment value of the tactile signal. and ; Step 5, the processing module determines the reference intensity of the tactile signal. and adjustment value and Determine the signal strength of each distributed haptic unit. , The output is sent to the corresponding distributed haptic unit; Step 6: The distributed tactile unit outputs directly perceptible tactile sensations to the patient based on the received signal strength.

[0216] In step 1, the patient's head posture information, including pitch angle, roll angle, and yaw angle, can intuitively represent the rotation angle of the head around three axes.

[0217] In step 1, the spatial coordinate system is a spatial polar coordinate system. Specifically, the center of the head is the origin, the top of the head is the Z-axis with upward as the positive direction; front and back are the X-axis with forward as the positive direction; and left and right are the Y-axis with left as the positive direction.

[0218] Furthermore, the specific steps of step 2 include: Step 2.1: Capture a sequence of dynamic eye images using a high-speed camera and analyze the characteristics of human eye movements; Specifically, this invention uses a near-infrared high-speed camera (wavelength range 780-950nm) to capture dynamic image sequences of the patient's eyes at a frame rate of 200-500fps and a resolution of not less than 1920×1080; and simultaneously triggers a ring-shaped infrared LED array (wavelength 850nm, irradiance ≤0.5mW / cm²). 2 Provides illumination to ensure the pupil outline is clear.

[0219] Specifically, this invention uses an ellipse fitting algorithm to calculate the pupil center coordinates with a positioning accuracy of ±0.1° angle of view; it detects at least 4 corneal reflection points to eliminate displacement errors caused by slight head movements (compensation accuracy ±0.5mm). In this embodiment, the eye-tracking module is used to detect gaze events. Specifically, a gaze event is defined as a visual displacement of less than 1° that lasts for 100-400ms.

[0220] Step 2.2: The analyzed eye movement information is transmitted to the processing module. Based on information such as gaze persistence time and pupil changes, the characteristic object that the patient is focusing on is determined. Specifically, eye-tracking data should include timestamps (accuracy ±1ms), pupil diameter, and gaze coordinates, and be transmitted to the processing module via a signal interface.

[0221] Specifically, the processing module of the present invention determines the presence of features in the current gaze area by calculating relevant conditions such as the change in pupil diameter and gaze duration.

[0222] Specifically, the processing module of the present invention merges continuous gaze points using a density clustering algorithm and excludes low-texture background regions with low image entropy.

[0223] Step 2.3: Map the coordinates of the feature in the image to a three-dimensional spatial coordinate system to determine its approximate range in the spatial coordinate system.

[0224] In step 3, the binocular vision module can obtain depth information by simultaneously capturing two images, left and right, and calculating the parallax. It can reconstruct three-dimensional scenes, measure object distances, and identify spatial relationships in real time, providing stereoscopic vision capabilities similar to those of humans.

[0225] Furthermore, step 3 includes the following specific steps: Step 3.1: Simultaneously capture two images using the left and right cameras, the images containing features identified by the eye-tracking module; Step 3.2: Locate the corresponding pixels of the feature in the two images, and calculate the distance L of the feature based on the principle of triangulation; Step 3.3: Calculate the horizontal deflection angle θ of the feature relative to the front of the user.

[0226] In step 3, existing binocular vision algorithms can be used to calculate distance and orientation.

[0227] In one implementation, step 3 may only calculate the horizontal deflection angle θ, without calculating the vertical deflection angle α.

[0228] In step 4, the reference intensity of the tactile signal It is inversely correlated with distance L.

[0229] In one implementation, the reference strength is: Where A is a parameter that is adjusted based on the patient's sensitivity to touch.

[0230] In one implementation, the reference strength is: Here, A and e are parameters that are adjusted based on the patient's sensitivity to touch.

[0231] In one implementation, the reference strength is: Where 'a' is a parameter, adjusted based on the patient's sensitivity to touch, and 0 <a<1。

[0232] Preferably, a distance L to a reference strength can be created. The mapping curve is calculated, and the mapping relationship is adjusted according to the patient's perception.

[0233] In step 4, the adjustment value of the tactile signal It is related to the orientation θ.

[0234] In one implementation, the adjustment value of the tactile signal is: In one implementation, the adjustment value of the tactile signal is: In one implementation, the adjustment value of the tactile signal is: Where C is a parameter that is adjusted based on the patient's sensitivity to touch. These are parameters that can be adjusted according to the required field of view.

[0235] Preferably, adjustments can be made from the horizontal deflection angle θ and the vertical deflection angle α. and The mapping curve is calculated, and the mapping relationship is adjusted according to the patient's perception, with the optimal value being the one that the patient can perceive most intuitively.

[0236] In one implementation, considering the different perceptions of up and down and left and right by the human body, adjustments can be made from the horizontal deflection angle θ to the adjustment value. The mapping curve, and the vertical deflection angle α to the adjustment value. The mapping curves are adjusted according to the patient's sensory sensitivity.

[0237] In one implementation, the adjustment value can be ignored. Simply set it to 0.

[0238] In step 5, different signal strengths correspond to different distributed tactile units (i = 1, 2, ...). The signal strength is determined by the orientation of the focused feature.

[0239] Generally, distributed tactile units closer to the focused feature have stronger signal strength, while distributed tactile units farther away from the focused feature have weaker signal strength.

[0240] In step 5, the signal strength of each distributed tactile unit is compared with the reference strength. They show a positive correlation.

[0241] Preferably, the signal strength of each distributed tactile unit is equal to the reference strength. The relationship is linear.

[0242] In step 5, the signal strength and adjustment value of each distributed tactile unit are... and They show a positive correlation.

[0243] Preferably, the signal strength and adjustment value of each distributed tactile unit and The relationship is linear.

[0244] In one implementation, the signal strength of each distributed tactile unit is: Where n is a constant. Q is the transformation matrix, determined by the arrangement of the distributed haptic units. For each distributed haptic unit, there exists a unique [ ] is used to indicate its response to a signal.

[0245] Specifically, Determined based on the sensitivity of each distributed tactile unit, The arrangement is determined by the left and right layout of the distributed tactile units. The arrangement is determined by the vertical arrangement of the distributed tactile units.

[0246] Preferably, the signal strength from each distributed haptic unit can be created. Regarding the reference strength and adjustment value and The mapping relationship is established and adjusted based on the patient's perception.

[0247] In one implementation, the signal strength from the distance L and orientation θ and α of the feature to each distributed tactile unit can also be directly created. The mapping relationship.

[0248] The mapping relationship should be determined based on the rules in steps 4 and 5.

[0249] In one implementation, without considering the upper and lower tactile units, the signal strength from the distance L and orientation θ of the feature to each tactile unit can be created. The mapping relationship.

[0250] Specifically, within a two-dimensional interval consisting of a certain distance range L and an orientation range θ, several points are discretely selected, and the signal strength of each distributed tactile unit i is calculated to create a mapping table.

[0251] During control, for any distance L and orientation θ, the signal strength of the corresponding distributed tactile unit i can be obtained by directly looking up a table and fitting the data. .

[0252] In one implementation, please refer to Figure 5 For any tactile unit, there is a corresponding signal intensity map. Figure 5 The upper part shows the signal intensity map of the left tactile sensor, and the lower part shows the signal intensity map of the right tactile sensor. The signal intensities of the left and right tactile sensors can be obtained by looking up a table, as follows: Select the corresponding curve based on the orientation angle of the feature (refer to the legend in the upper right corner). Based on the distance of the feature, the corresponding vertical line is intercepted in the horizontal direction. The intersection of the vertical line with the upper figure is the signal strength of the left tactile unit, and the intersection with the lower figure is the signal strength of the right tactile unit.

[0253] In one specific embodiment, assume there is a feature located at -60° and 0.4m away. According to the legend, -60° corresponds to the second legend (red square), which is the second curve from the top. 0.4m corresponds to the third data point. From the upper and lower graphs respectively, we can see that the signal strength of the left tactile unit is 6.22, and the signal strength of the right tactile unit is 0.45.

[0254] In one specific embodiment, assume there is a feature located at -20° and 0.55m away. Although there is no corresponding data point, by using difference fitting, it can be determined from the upper and lower images that the signal strength of the left tactile unit is 2.44 and the signal strength of the right tactile unit is 1.20.

[0255] In one implementation, please refer to Figure 6The signal strengths of the left and right tactile units can have different correspondences.

[0256] In one implementation, if there are upper or lower tactile units, then the corresponding diagrams are added according to the number of tactile units.

[0257] In step 6, the processing module circuit is connected to the distributed haptic units, and the signal strength of each distributed haptic unit is calculated. The physical tactile signals of the distributed tactile units are controlled.

[0258] The distributed tactile unit controls its internal mechanical control device to output tactile signals based on the received signal strength.

[0259] In one implementation, the distributed tactile unit in step 6 transmits the tactile signals from the processing module to the patient through mechanical movements such as physical vibration or pressure changes, allowing the patient to perceive the distance, orientation, and spatial location information of the target object through touch instead of direct vision.

[0260] The above-described specific embodiments are preferred embodiments of the present invention and are not intended to limit the present invention. Any other changes or equivalent substitutions made without departing from the technical solution of the present invention are included within the protection scope of the present invention.

Claims

1. A visual function assistive device, characterized in that, include: Wearable structures are designed to be worn on the human body and to support other unit modules. A distributed tactile unit, which comes into contact with multiple symmetrically distributed parts of the human body, and transmits differentiated tactile signals to different parts of the human body; A posture sensor, located on the patient's head, is used to determine the orientation of the patient's head. An eye-tracking module, located around the eyes, is used to identify features that the patient is focusing on in real time; A binocular vision module, located in the wearable structure, can identify the distance and orientation of features and output the relevant information to the processing module; The system also includes a processing module that is connected to the attitude sensor, eye-tracking module, and binocular vision module to receive information and control the distributed tactile units.

2. The visual function assistive device according to claim 1, characterized in that, The distributed tactile unit includes a motor, a linkage mechanism, and a pressure plate. The motor is connected to the linkage mechanism and is powered by the linkage mechanism to drive the pressure plate to move and generate extrusion force. The linkage mechanism includes an active arm, a driven arm, and a linkage rod forming a planar four-bar linkage system, which is connected by hinges.

3. The visual function assistive device according to claim 1, characterized in that, The distributed tactile unit includes a left tactile unit and a right tactile unit, which are respectively connected to the processing module and are symmetrically distributed on the human body.

4. The visual function assistive device according to claim 1, characterized in that, The distributed tactile unit includes an upper tactile unit or a lower tactile unit, and the signals of the upper tactile unit or the lower tactile unit are independently controlled by the processing module.

5. A visual function assistive device according to claim 1, characterized in that, The visual function assistive device adopts a split design.

6. A control method for a visual assistive device, characterized in that, Includes the following steps: Step 1: Acquire the patient's head posture information through the posture sensor and transmit it to the processing module to establish a spatial coordinate system based on this information; Step 2: The eye-tracking module detects the movement of the human eye and transmits the information to the processing module to determine the feature object that the patient is focusing on, as well as its approximate range in the spatial coordinate system. Step 3: The binocular vision module calculates the distance L and orientation θ and α of the feature object by utilizing binocular parallax information, and transmits it to the processing module; Step 4: The processing module determines the reference intensity of the tactile signal based on the distance of the feature. Based on the orientation of the focused feature, determine the adjustment value of the tactile signal. and ; Step 5, the processing module determines the reference intensity of the tactile signal. and adjustment value and Determine the signal strength of each tactile unit. , The output is sent to the corresponding distributed haptic unit; Step 6: The distributed tactile unit outputs directly perceptible tactile sensations to the patient based on the received signal strength.

7. The control method for a visual assistive device according to claim 6, characterized in that, In step 4, the reference intensity of the tactile signal and adjustment value and Calculated using the following method: Where A is a parameter and C is a parameter that is adjusted based on the patient's sensitivity to touch; These are parameters that can be adjusted according to the required field of view.

8. The control method for a visual assistive device according to claim 6, characterized in that, The signal strength of each distributed haptic unit mentioned in step 5: Where n is a constant, and Q is a transformation matrix determined by the arrangement of the distributed tactile units; for each distributed tactile unit, there exists a unique [ ] is used to indicate its response to a signal.

9. The control method for a visual assistive device according to claim 6, characterized in that, Within a three-dimensional interval defined by a certain distance L and orientation θ and α ranges, several points are discretely selected, and the signal intensity of each distributed tactile unit i is calculated to create a mapping table. During control, for any distance L and orientation θ and α, the signal intensity of the corresponding distributed tactile unit i can be obtained by directly looking up the table and fitting the data. .

10. The control method for a visual function assistive device according to claim 6, characterized in that, Create the signal strength from the distance L and orientation θ of the feature to each tactile unit. The mapping relationship is used to obtain the signal strength of the corresponding tactile unit i during control by directly looking up a table and fitting the data. .

Images