Simulated real scene building and rehabilitation training method and system based on visual adaptation

Abstract

The invention provides a simulated real scene construction and rehabilitation training method and system based on visual adaptation, and the method comprises the steps: obtaining the visual information of a patient facing a horizontal desktop in different postures in a real family scene, and constructing a family simulation basic scene adaptive to the visual angle height and range of the patient; the visual information comprises a visual height and a pitching angle relative to a reference plane; a home simulation basic scene is optimized, a plurality of induction grids are arranged on a horizontal desktop in the home simulation optimization scene, a patient touches the induction grids by using a virtual hand according to an instruction, and the motion trail of the virtual hand is obtained by combining a home simulation optimization scene visual image and the induction grids to touch special effect sounds. And performing rehabilitation analysis based on the motion trail. By acquiring the visual information of the patient in different postures in the real family scene, the family simulation scene adaptive to the visual angle of the patient is constructed, the problem that the individual difference and the family background are not considered in the VR application is solved, and the experience feeling of the patient is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of rehabilitation training, and in particular to a method for building a simulated real scene and conducting rehabilitation training based on visual adaptation. Background Art

[0002] In recent years, virtual reality (VR) technology has garnered significant attention for addressing motor dysfunction caused by impaired neurological function. Thanks to its unique environmental creation system and multi-sensor interaction between the subject and the simulator, VR allows for a near-realistic experience. VR technology is widely used in the rehabilitation of neurological conditions such as stroke and brain injury, enabling exercises that are difficult to achieve with traditional rehabilitation. Furthermore, gamification can enhance patient engagement and compliance, while multi-sensory stimulation and challenges can motivate patients and improve rehabilitation outcomes.

[0003] Currently, various VR technologies have been applied to medical scenario design, such as virtual training that simulates reaching, grabbing, and releasing a ball from a basket, and assisted treadmill training in different virtual environments to help patients with rehabilitation exercises. These solutions have achieved some success, but they have significant shortcomings in simulating home environments and adapting to individual patient differences.

[0004] Many VR applications use standardized medical environments and fail to fully consider the individual differences and family backgrounds of patients. At the same time, different patients have different visual perceptions, and most existing solutions are based on unified visual standards, which cannot truly fit the scenes seen by each patient. Existing VR scenes cannot be adjusted for such differences, causing patients to feel uncomfortable during the experience. This unfamiliar and inappropriate environment can easily cause anxiety and resistance in patients, which in turn affects their compliance and enthusiasm for treatment. Therefore, there is an urgent need for a VR scene and training task that can be designed based on the patient's real family background and fully considers individual visual differences, so as to reduce patients' environmental discomfort and improve treatment effectiveness. Summary of the Invention

[0005] To address these issues, the present invention proposes a method for constructing simulated real-world scenes and conducting rehabilitation training based on visual adaptation. By capturing visual information of patients in different postures in real-life home settings, a simulated home scene adapted to the patient's perspective is constructed. This addresses the issue of VR applications failing to consider individual differences and family backgrounds. Furthermore, the scene is adjusted based on the patient's visual characteristics, enhancing the fit and improving the patient experience.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a method for building a simulated real scene and conducting rehabilitation training based on visual adaptation, comprising: Acquire visual information of the patient facing a horizontal tabletop in different postures in a real home scene, and construct a home simulation basic scene adapted to the patient's viewing angle height and range; the visual information includes visual height and pitch angle relative to the reference plane; Optimizing the basic home simulation scene to obtain an optimized home simulation scene; Multiple sensing grids are set up on a horizontal table in the home simulation optimization scene. The patient uses a virtual hand to touch the sensing grids according to instructions. The movement trajectory of the virtual hand is obtained by combining the visual image of the home simulation optimization scene and the special sound effects of touching the sensing grids, and rehabilitation analysis is performed based on the movement trajectory.

[0007] In a second aspect, the present invention provides a system for simulating real-world scenes and rehabilitation training based on visual adaptation, comprising: A scene construction unit is configured to obtain visual information of a patient facing a horizontal table in different postures in a real home scene, and construct a home simulation basic scene adapted to the patient's viewing angle height and range; the visual information includes visual height and pitch angle relative to a reference plane; A scene optimization unit is configured to optimize the home simulation basic scene to obtain a home simulation optimized scene; The rehabilitation training unit is configured to set up multiple sensing grids on a horizontal table in a home simulation optimization scene. The patient uses a virtual hand to touch the sensing grids according to instructions, combines the visual image of the home simulation optimization scene and the special sound effects of touching the sensing grids to obtain the movement trajectory of the virtual hand, and performs rehabilitation analysis based on the movement trajectory.

[0008] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for building a simulated real scene and rehabilitation training based on visual adaptation described in the first aspect.

[0009] In a fourth aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the method for constructing a simulated real scene and rehabilitation training based on visual adaptation described in the first aspect are implemented.

[0010] Compared with the prior art, the present invention has the following beneficial effects: The present invention first obtains the patient's visual information of sitting and standing facing a horizontal table in a real home scene, accurately constructs a home simulation basic scene that adapts to the height and range of the viewing angle, and performs optimization processing such as denoising, color correction, and super-resolution reconstruction on the scene. During training, the virtual hand movement trajectory is obtained by combining visual images with the special sound effects of the sensor grid touch for rehabilitation analysis. The entire process is closely centered around the individual characteristics of the patient to create a VR rehabilitation training scene that is highly suitable for the patient, effectively improving the patient's training experience, stimulating their enthusiasm for participating in treatment, and bringing positive effects to rehabilitation treatment. By simulating a familiar home environment and providing a safe psychological space, patients can adapt and relax, alleviating their anxiety and fear about the medical process and environment. This can help patients feel more positive during treatment, thereby increasing their acceptance and compliance with treatment plans and ultimately improving treatment outcomes. Furthermore, customizing VR scenarios based on each patient's home environment and cultural background can enhance the individualized medical experience and improve patient satisfaction.

[0011] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their description are used to explain the present invention but do not constitute a limitation of the present invention.

[0013] Figure 1 This is a main flow chart of a method for simulating a real-world scenario and rehabilitation training based on visual adaptation provided by an embodiment of the present invention; Figure 2 A schematic diagram of a system for simulating real-world scenarios and conducting rehabilitation training based on visual adaptation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0014] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0015] Example 1 like Figure 1 As shown, this embodiment discloses a method for simulating a real scene and rehabilitation training based on visual adaptation, comprising the following steps: S1: Acquire visual information of the patient facing a horizontal table in different postures in a real home scene, and construct a home simulation basic scene adapted to the patient's viewing angle height and range; the visual information includes visual height and pitch angle relative to a reference plane; S2: Optimize the basic home simulation scenario to obtain an optimized home simulation scenario; S3: Multiple sensing grids are set up on a horizontal table in the home simulation optimization scene. The patient uses a virtual hand to touch the sensing grids according to instructions. The movement trajectory of the virtual hand is obtained by combining the visual image of the home simulation optimization scene and the special sound effects of touching the sensing grids, and rehabilitation analysis is performed based on the movement trajectory.

[0016] Next, combine Figure 2 , a method for building a simulated real scene and rehabilitation training based on visual adaptation disclosed in this embodiment is described in detail.

[0017] In S1, in order to improve patients' sense of security and comfort during rehabilitation training, a simulated training scenario was constructed based on the patient's real home scenario.

[0018] (1) Equipment selection and deployment Select a suitable 360° fisheye camera and ensure that its resolution and image quality meet the requirements of subsequent processing.

[0019] Plan your photoshoot, ensure your patient's home is clean and tidy, and shoot in good lighting conditions.

[0020] (2) Image acquisition Place fisheye cameras in the patient's home environment, ensuring that they cover key areas of the patient's daily life (such as the coffee table in front of the sofa or the dining table).

[0021] Acquire visual information from the patient in a real-life home setting, in various postures (sitting and standing) facing a horizontal tabletop. This information includes visual height and pitch angle relative to a reference plane. Measure the patient's sitting / standing height and, based on this acquired visual information, set the fisheye camera height to ensure that the captured image information meets the patient's viewing angle and range.

[0022] Take multiple shots to obtain images from different angles so that the best image can be selected for stitching.

[0023] (3) Image preprocessing The collected fisheye images are dedistorted to correct lens distortion and improve stitching quality.

[0024] Use image enhancement techniques (such as brightness and contrast adjustment) to optimize image quality. Use existing image processing libraries (such as OpenCV) to implement this functionality.

[0025] The undistortion function uses the OpenCV cv2.fisheye.undistortImage() method to undistort the input fisheye image. The undistortion process uses the camera intrinsic parameter matrix K and the distortion coefficient D to correct the image.

[0026] Furthermore, feature matching is performed by using the feature_matching() function to detect and match key points in the two images using the ORB feature extractor, and matching is performed using brute force matching (cv2.BFMatcher).

[0027] (4) Image stitching The two fisheye images are stitched together into a rectangular panorama using a specific algorithm (such as a panoramic stitching algorithm). During the stitching process, the overlapping areas are ensured to be seamless, avoiding visible visual artifacts at the seams.

[0028] Among them, image stitching uses the image_stitching() function to calculate the homography matrix based on the feature matching results, and then uses the cv2.warpPerspective() function to perform perspective transformation, and merges the second image into the first image to finally obtain the stitching result.

[0029] (5) Generate a panoramic image Output the stitched panorama to ensure it is distortion-free and meets VR display requirements. At the same time, perform quality inspections to confirm the panorama's clarity and detail.

[0030] (6) Constructing a basic scenario for family simulation According to the previously acquired patient visual information, a home simulation basic scene adapted to the patient's viewing angle height and range is constructed based on the panoramic image. The scene includes a horizontal desktop.

[0031] In S2, during the construction of a simulated home scene based on visual adaptation, the patient's visual height and pitch angle relative to a reference plane, as viewed from a horizontal tabletop in different postures in a real home setting, were obtained. This was to construct a basic home simulation scene that matched the patient's viewing angle and range. However, the introduction of this critical information led to image distortion.

[0032] Due to the diversity of patient postures, different viewing heights and pitch angles can cause distortion in the lens image during image capture. For example, when a patient is standing, the pitch of the camera angle may cause the horizontal tabletop in the image to appear tilted or stretched. Meanwhile, when sitting, the lower viewing height may cause the proportions of objects in the image to deviate from their actual proportions. These distortions not only affect the realism of the scene but also may interfere with the patient's visual judgment during rehabilitation training, thereby affecting the accuracy of rehabilitation analysis. Therefore, optimization of the basic home simulation scene is necessary.

[0033] Specifically, scene optimization can be performed through denoising, color correction, and super-resolution reconstruction.

[0034] In S3, multiple sensing grids are set up on a horizontal table in the home simulation optimization scene. The patient uses a virtual hand to touch the sensing grids according to the rehabilitation training instructions. The movement trajectory of the virtual hand is obtained by combining the visual image of the home simulation optimization scene and the special sound effects of touching the sensing grids, and rehabilitation analysis is performed based on the movement trajectory.

[0035] Specifically, the scene module imports the processed panorama into the VR development platform Unity3D to build the basic scene of the virtual environment.

[0036] The patient is equipped with a virtual hand. The motion sensor Leap Motion Controller 2 scans the infrared reflection signal of the virtual hand through its infrared camera, and tracks the patient's hand grasping, movement and other movements in real time. By calculating the relative position and angle of the hand, it can determine whether the fingers are performing certain actions (such as grasping, pointing, waving, etc.).

[0037] For example, by tracking the specific position of each finger, the motion sensor can determine whether the hand is moving or resting. If the finger moves on the x-axis, y-axis, or z-axis, it is judged as "hand movement." The motion sensor can also use hand.grab_strength and hand.pinch_strength, as well as the corresponding thresholds, to determine whether the hand is grasping or pinching.

[0038] As an implementation, the threshold is set to 0.8: when grab_strength > 0.8, the hand is judged to be performing a grab gesture; when pinch_strength > 0.8, the hand is judged to be performing a pinch gesture. It should be understood that those skilled in the art can set the threshold as needed.

[0039] Furthermore, set up the training scenario: Based on the patient's training needs and actual posture, the setting of the training scene needs to take into account the patient's sitting or standing position in virtual reality. The system will automatically adjust the height and angle of the training task plane in the VR panorama according to the patient's sitting or standing posture to ensure that the virtual task matches the patient's actual posture, improving the interactive experience and the smoothness of the movement. Specifically: (1) Sitting mode setting: When the patient is in a sitting position, the VR system automatically adjusts the height of the task plane to ensure that it is aligned with the patient's line of sight and hand range of motion, avoiding training difficulties caused by the mismatch between the task area and the patient's actual range of motion.

[0040] (2) Standing mode setting: When the patient is in a standing position, the system will automatically adjust the height and position of the task plane according to the patient's height and the type of task, ensuring that the patient can perform training tasks within the natural range of hand movements, while providing a larger movement space to promote the body's standing balance and movement coordination.

[0041] It should be understood that the conversion of visual images into spatial coordinates involved can be implemented by those skilled in the art.

[0042] In this embodiment, by collecting information such as the visual height and pitch angle of the patient's sitting and standing postures in real home scenes, a basic home simulation scene that is highly consistent with their perspective, such as a living room or dining room environment, is constructed. This can greatly enhance the patient's familiarity and sense of security with the training scene, alleviate the anxiety and resistance caused by the unfamiliar medical environment, and improve treatment compliance.

[0043] The optimized scene eliminates image distortion caused by posture differences through denoising and color correction, bringing the virtual scene closer to real-world visual perception, reducing interference with visual judgment, and ensuring the accuracy of rehabilitation analysis. During training, the task plane height and angle are automatically adjusted based on the patient's posture, such as aligning the line of sight and hand range of motion when sitting, and matching height and range of motion when standing. This ensures smooth and natural interaction, promotes balance and coordination training, and allows patients to efficiently complete rehabilitation tasks in a comfortable, personalized virtual environment, significantly improving the training experience and treatment effectiveness.

[0044] VR training tasks are designed based on daily living activities to help patients improve their motor function and daily living abilities. In this embodiment, rehabilitation training is conducted in the training module based on the "wiping table" task.

[0045] First, the training props were set up. A VR home desktop in a panoramic scene was placed in front of the patient's line of sight. A translucent surface composed of multiple squares was placed on the desktop. Each square was assigned a different color or transparency to distinguish between completed and unfinished areas. (The translucent surface, 150 cm long and 100 cm wide, consisted of 150 squares.) Afterwards, the patient is given a virtual hand (containing a motion sensor for hand tracking) and a head-mounted VR goggles (providing the patient with a virtual home scene that suits their perspective). The virtual hand and head-mounted VR goggles are connected to a computer respectively.

[0046] The patient is given a square elimination instruction, and each square will gradually disappear after the patient touches it with the virtual hand, simulating the cleaning action. The process of obtaining the motion trajectory of the virtual hand is as follows: (1) Multi-source data acquisition and time calibration Data collection: The Leap Motion Controller 2 (virtual hand) captures real hand motion data at a sampling rate of 120Hz and generates a three-dimensional space coordinate sequence {X, Y, Z, }, establish the xy axis coordinate system with the desktop grid, z is the height perpendicular to the desktop plane, and Z=0 when it touches the desktop.

[0047] VR scene record: The VR scene rendering engine synchronously records the contact events of the virtual hand and the grid collision detection, and generates a contact timestamp set { When the rehabilitation patient clicks a square with his virtual hand, the system records the duration of the contact.

[0048] Sound feedback collection: When the grid is touched, a special effect sound will be generated. The starting time point of the special effect sound is extracted through time-frequency analysis { }, establish a millisecond time base.

[0049] (2) Time synchronization and error correction Time synchronization: To ensure consistent timing between the virtual hand, VR screen, and sound systems, the Leap Motion, audio device, and graphics rendering engine are synchronized at the hardware level using the NTP protocol, with the calibration error controlled within ±8ms.

[0050] Determine effective touch and establish a time correlation model: If the time of the hand position data Contact time recorded by VR The difference does not exceed the preset millisecond threshold (| - |≤15ms), it means that this is a valid touch, for example, it actually touches the square. That is, when | - When |≤Δt (Δt=15ms), it is determined to be a valid touch event.

[0051] If the time of the hand position data and sound time Difference of more than 30 milliseconds (| - |≥30ms), indicating that there is an error. That is, when | - When the error is ≥ 30ms, the anomaly detection algorithm is started.

[0052] Since the rehabilitation personnel may accidentally touch the grid boundary (non-valid touch area) or touch the wrong grid (such as touching grid 2 when the target is grid 1), corrections need to be made in combination with the physical feedback of the sensing grid.

[0053] When the sensing grid is effectively touched, the built-in sensor triggers the corresponding special effect sound (such as a high-frequency prompt sound for a correct touch and a low-frequency warning sound for an incorrect touch), and records the time when the special effect sound is emitted. . Through the time synchronization algorithm, t is synchronized with the timestamp of the virtual hand recognition Matching, calculate the time difference ΔT=| - |, when ΔT is within the preset threshold, As the corrected touch time, invalid data of boundary touch or error touch is eliminated.

[0054] Smoothing: A sliding window algorithm was used to perform temporal smoothing on continuous touch events, and the window length was dynamically adjusted according to the patient's movement speed (range 50-200ms).

[0055] (3) Motion trajectory reconstruction Based on the Kalman filter, the spatial coordinates and time correction data are fused to generate a four-dimensional trajectory data packet (X, Y, Z, Δt) with confidence. Establish a mapping relationship between contact events and grid positions, and use the Bresenham algorithm to complete the linear motion trajectory between adjacent contact points; Generate a motion heatmap containing time dimension information, where the chromaticity represents the contact duration and the brightness represents the contact pressure estimate.

[0056] (4) Adaptive training feedback mechanism Real-time analytics: Calculate the effective contact area percentage per unit time: S = Σ(number of effective contact squares) / total number of contactable squares × 100%. For example, if a patient is supposed to touch 10 squares and actually touches 8 correctly, S is 80%. If S falls below a set threshold (e.g., 70%), indicating difficulty with the user, the system will automatically brighten the squares and increase the volume to make it easier for the patient to perceive.

[0057] Calculate "Motion Continuity" (C), a dynamic assessment of movement continuity: C = 1 - (number of abnormal time intervals / total number of touches). If the patient frequently interrupts their movements (e.g., frequent pauses), the C value will decrease. When C falls below the threshold, the system adjusts the grid density, such as by highlighting the direction of the patient's hand movement in advance, to help the patient complete the movement more smoothly.

[0058] Automatic adjustment feedback mechanism: When S < threshold When , the visual contrast of the disappearing grid is automatically reduced and the intensity of the special effect audio is enhanced; When C < threshold When the grid is generated, the density of the grid is adjusted to pre-highlight the prompt area in the predicted direction of the motion trajectory.

[0059] In this embodiment, the motion trajectory is obtained based on the recognition module. Among them, rehabilitation personnel are prone to accidentally touch the grid boundary or touch the wrong square during operation. By combining the physical feedback of the sensing grid and the time synchronization algorithm, the touch time is corrected with the special effect sound time, invalid data is eliminated, and the effective touch is accurately defined, so that the motion trajectory is more in line with the real rehabilitation movement, providing a reliable basis for subsequent training evaluation. In terms of rehabilitation training effect, smoothing processing makes the trajectory more continuous and stable. A sliding window algorithm is used to dynamically adjust the window length according to the patient's movement speed, smooth the time domain of continuous touch events, avoid trajectory fluctuations caused by hand shaking, etc., help rehabilitation personnel form standardized and smooth movement patterns, and improve training quality. At the same time, accurate trajectory data provides support for adaptive training feedback. Based on the accurate trajectory, the effective contact area ratio and movement continuity index are calculated. The system adjusts the visual and auditory feedback in a timely manner to enhance patient perception, improve training efficiency, and better achieve rehabilitation training goals.

[0060] After each training session, the system automatically generates a detailed training report, including the total time to complete the task, the number of squares eliminated, the remaining squares, the average movement accuracy, the movement frequency, etc. This data will help doctors or therapists assess the patient's recovery progress and adjust subsequent training plans accordingly.

[0061] Many VR applications use standardized medical environments, ignoring individual patient differences and family backgrounds. Furthermore, their visual standards are uniform, and the scenes fail to align with the patient's visual perception. This invention makes targeted improvements by collecting information such as the patient's visual height and pitch angle in different postures in real home scenes, constructing and optimizing an adaptive home simulation basic scene. This allows patients to train in familiar home scenes, fully accounting for individual visual differences, greatly improving scene alignment, effectively alleviating patients' anxiety and resistance, and significantly improving treatment compliance and motivation.

[0062] Example 2 This embodiment provides a system for simulating real-world scenarios and rehabilitation training based on visual adaptation, including: A scene construction unit is configured to obtain visual information of a patient facing a horizontal table in different postures in a real home scene, and construct a home simulation basic scene adapted to the patient's viewing angle height and range; the visual information includes visual height and pitch angle relative to a reference plane; A scene optimization unit is configured to optimize the home simulation basic scene to obtain a home simulation optimized scene; The rehabilitation training unit is configured to set up multiple sensing grids on a horizontal table in a home simulation optimization scene. The patient uses a virtual hand to touch the sensing grids according to instructions, combines the visual image of the home simulation optimization scene and the special sound effects of touching the sensing grids to obtain the movement trajectory of the virtual hand, and performs rehabilitation analysis based on the movement trajectory.

[0063] Example 3 This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the method for building a simulated real scene and rehabilitation training based on visual adaptation as described in the first embodiment above are implemented.

[0064] Example 4 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the method for constructing a simulated real scene and conducting rehabilitation training based on visual adaptation as described in the first embodiment above are implemented.

[0065] The steps or modules involved in Examples 2 to 4 above correspond to those in Example 1. For detailed implementations, please refer to the relevant description of Example 1. The term "computer-readable storage medium" should be understood to mean a single medium or multiple media that includes one or more instruction sets; it should also be understood to include any medium that can store, encode, or carry an instruction set for execution by a processor and cause the processor to perform any method of the present invention.

[0066] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for simulating real-world scenes and rehabilitation training based on visual adaptation, characterized in that: include: Acquire visual information of the patient facing a horizontal tabletop in different postures in a real home scene, and construct a home simulation basic scene adapted to the patient's viewing angle height and range; the visual information includes visual height and pitch angle relative to the reference plane; Optimizing the basic home simulation scene to obtain an optimized home simulation scene; Multiple sensing grids are set up on a horizontal table in the home simulation optimization scene. The patient uses a virtual hand to touch the sensing grids according to instructions. The movement trajectory of the virtual hand is obtained by combining the visual image of the home simulation optimization scene and the special sound effects of touching the sensing grids, and rehabilitation analysis is performed based on the movement trajectory.

2. A method for simulating a real scene and rehabilitation training based on visual adaptation as claimed in claim 1, characterized in that: The different postures include sitting and standing; the basic home simulation scene is a living room scene or a dining room scene.

3. The method for simulating a real scene and rehabilitation training based on visual adaptation according to claim 1, characterized in that: The method of obtaining visual information of the patient facing a horizontal table in different postures in a real home scene and constructing a home simulation basic scene adapted to the patient's viewing angle height and range specifically includes: Multiple sets of image data were collected when the patient was sitting and standing at home facing a horizontal table; Using image recognition and posture estimation algorithms, the patient's head position is identified from the captured image to obtain the visual height; by analyzing the angular relationship between the patient's head and the reference plane, the pitch angle relative to the reference plane is determined; Summarize and analyze the collected visual height and pitch angle data to determine the patient's visual height range and pitch angle range in different postures; According to the determined visual range, a basic home simulation scene is constructed in the virtual reality modeling software, and the position and height of objects in the scene are adjusted to obtain a basic home simulation scene that is adapted to the height and range of the patient's viewing angle.

4. The method for simulating a real scene and rehabilitation training based on visual adaptation according to claim 1, characterized in that: The home simulation basic scene is optimized to obtain a home simulation optimized scene; the optimization includes denoising, color correction and super-resolution reconstruction.

5. The method for simulating a real scene and rehabilitation training based on visual adaptation according to claim 1, characterized in that: The horizontal desktop is a translucent plane; the sensing grid presents different colors or transparencies to distinguish completed and unfinished areas, and emits special sound effects when the sensing grid is touched.

6. The method for simulating a real scene and rehabilitation training based on visual adaptation according to claim 1, characterized in that: The method combines the visual image of the home simulation optimization scene and the special sound effect of the sensing grid touch to obtain the movement trajectory of the virtual hand; specifically includes: Using a head-mounted VR glasses camera to obtain a visual image of the home simulation optimization scene, and using a virtual hand to obtain the position data of the touched squares; the position data is the coordinate information and timestamp of the virtual hand movement; Get the time point when the special effect sound of the sensor grid is touched, match it with the touch action timestamp, and get the corrected time; With the touch event as the trigger point, the coordinate information and the correction time are associated to form motion trajectory data with time information.

7. The method for simulating a real scene and rehabilitation training based on visual adaptation according to claim 6, characterized in that: The method of identifying the position data of the virtual hand touching the square based on visual images specifically includes: establishing a mapping relationship between the plane coordinate system of the virtual hand and the virtual scene coordinate system of the virtual hand, converting the pixel coordinates of the virtual hand and the square into actual coordinates in the virtual scene, and recording the timestamp of the touch action.

8. A system for simulating real-world scenes and rehabilitation training based on visual adaptation, characterized in that: include: A scene construction unit is configured to obtain visual information of a patient facing a horizontal table in different postures in a real home scene, and construct a home simulation basic scene adapted to the patient's viewing angle height and range; the visual information includes visual height and pitch angle relative to a reference plane; A scene optimization unit is configured to optimize the home simulation basic scene to obtain a home simulation optimized scene; The rehabilitation training unit is configured to set up multiple sensing grids on a horizontal table in a home simulation optimization scene. The patient uses a virtual hand to touch the sensing grids according to instructions, combines the visual image of the home simulation optimization scene and the special sound effects of touching the sensing grids to obtain the movement trajectory of the virtual hand, and performs rehabilitation analysis based on the movement trajectory.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by the processor, the steps of the method for simulating real scene construction and rehabilitation training based on visual adaptation as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method for simulating real scene construction and rehabilitation training based on visual adaptation as described in any one of claims 1 to 7 are implemented.

Images