Visually impaired person behavior ability assessment method and system

Abstract

The invention discloses a visual impaired person behavior ability assessment method and system, and relates to the technical field of visual rehabilitation, and the method comprises the steps: obtaining a dynamic behavior video of a visual impaired person; analyzing the dynamic behavior video by using a human body posture recognition library and a deep learning algorithm, and determining a head rotation direction and rotation times of the visually impaired person; and comparing and analyzing the head rotation direction and the rotation frequency of the visually impaired person with a pre-established standardized reference table, evaluating the actual use effect of the magnifying visual aid or the visual field expanding visual aid worn by the visually impaired person, and generating a personalized rehabilitation scheme according to the actual use effect. According to the method and the system provided by the invention, the visual behavior capability of a low-vision group is reflected more comprehensively and accurately by evaluating the head rotation frequency index of the visually impaired person in dynamic behaviors such as obstacle avoidance and search, so that powerful support is provided for formulating a personalized rehabilitation scheme.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual rehabilitation, in particular to a method and system for assessing the behavior ability of visually impaired persons. BACKGROUND

[0002] With the intensification of population aging, the number of low-vision groups is increasing year by year, which brings great challenges to their daily life, work and social interaction. Low vision not only affects the individual's visual perception ability, but also directly leads to the decline of their action ability in completing daily tasks. Especially in complex environments, low-vision groups often have difficulty accurately judging the situation around them and cannot respond in time, thereby increasing the risk of accidents.

[0003] Currently, there are few methods for assessing the visual behavior ability of low-vision groups, and existing visual behavior assessment techniques mostly focus on static visual tests, ignoring the dynamic response related to vision in the behavior process. For example: in the process of obstacle avoidance, low-vision groups often confirm the situation of the surrounding environment by turning their heads. However, existing technologies have not been able to accurately capture and quantify these dynamic behaviors, resulting in a significant reduction in the accuracy and scientificity of the assessment results.

[0004] In addition, although visual aids (such as magnifying aids and field expansion aids) have been widely used in low-vision rehabilitation, their applicability and effectiveness are still difficult to accurately determine, mainly because of the lack of targeted and personalized visual behavior assessment standards. Therefore, how to accurately and objectively assess the visual behavior ability of low-vision groups, especially their performance in dynamic behaviors (such as obstacle avoidance), has become a problem that needs to be solved in the current visual rehabilitation field. SUMMARY

[0005] To this end, the present application provides a method and system for assessing the behavior ability of visually impaired persons to solve the problems of low accuracy in assessing the visual behavior ability of low-vision groups and the lack of targeted and personalized visual behavior assessment standards in existing technologies.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0007] In a first aspect, a method for assessing the behavior ability of visually impaired persons includes:

[0008] Step 1: Obtain a dynamic behavior video of a visually impaired person;

[0009] Step 2: Analyze the dynamic behavior video using a human pose recognition library and a deep learning algorithm, and determine the head turning direction and number of turns of the visually impaired person;

[0010] Step 3: The head rotation direction and rotation times of the visually impaired person are compared and analyzed with a pre-established standardized reference table, and the actual use effect of the magnifying aid or field expansion aid worn by the visually impaired person is evaluated, and a personalized rehabilitation plan is generated according to the actual use effect;

[0011] In step 2, the method specifically comprises:

[0012] Step 201: Frame data in the dynamic behavior video is obtained frame by frame, and human body posture key point position coordinates and confidence scores, face key point position coordinates and confidence scores, and head region position coordinates in each frame are extracted by using a human body posture recognition library and a deep learning algorithm;

[0013] Step 202: The average value of the confidence of the face key points in each frame is calculated, and a confidence waveform graph is drawn according to the average value of the confidence of the face key points in each frame;

[0014] Step 203: A first threshold value is determined according to the confidence waveform graph, and whether the visually impaired person in the dynamic behavior video is in a front mode or a back mode is determined according to the first threshold value;

[0015] Step 204: The head rotation direction and rotation times of the visually impaired person in the front mode or the back mode are determined according to the human body posture key point position coordinates and confidence scores, the face key point position coordinates and confidence scores, and the head region position coordinates.

[0016] Optionally, in step 201, the deep learning algorithm adopts a YOLOv5 target detection algorithm.

[0017] Optionally, in step 204, when the head rotation direction and rotation times of the visually impaired person in the front mode are determined, the method specifically comprises:

[0018] determining the horizontal direction state of the head according to the confidence difference between the right ear confidence and the left ear confidence;

[0019] determining the vertical direction state of the head according to the relative position coordinates of the nose, the ear and the shoulder;

[0020] determining the head rotation direction and rotation times according to the horizontal direction state and the vertical direction state.

[0021] Optionally, when the horizontal direction state of the head is determined according to the difference between the right ear confidence and the left ear confidence, the method specifically comprises: calculating the confidence difference between the right ear confidence and the left ear confidence, if the confidence difference is greater than a second threshold value, it is determined that the head rotates to the left, if the confidence difference is less than a negative second threshold value, it is determined that the head rotates to the right, and if the confidence difference is between the second threshold value and the negative of the second threshold value, it is determined that the head is in the middle.

[0022] Optionally, when judging the vertical direction state of the head according to the relative position coordinates of the nose, ear and shoulder, the method specifically comprises:

[0023] calculating the distance from the shoulder center to the ear as a reference;

[0024] calculating the vertical proportion between the nose and the shoulder center;

[0025] if the vertical proportion is greater than a preset upward threshold, the head is determined to be tilted upward; if the vertical proportion is less than a preset downward threshold, the head is determined to be tilted downward; and if the vertical proportion is within a reference range, the head is determined to be in a central position.

[0026] Optionally, when determining the head rotation direction and rotation times of the visually impaired person in the back mode in step 204, the method specifically comprises:

[0027] judging the horizontal direction state of the head according to the confidence difference between the right ear confidence and the left ear confidence;

[0028] determining a target region according to the head region position coordinates, and judging the vertical direction state of the head by tracking feature points in the target region using an optical flow method;

[0029] determining the head rotation direction and rotation times according to the horizontal direction state and the vertical direction state.

[0030] Optionally, when judging the vertical direction state of the head by tracking feature points in the target region using the optical flow method, the method specifically comprises: if the vertical component of the optical flow vector is greater than a preset upward threshold, the head is determined to be tilted upward; if the vertical component of the optical flow vector is less than a preset downward threshold, the head is determined to be tilted downward; and if the vertical component of the optical flow vector is within a preset range, the head is determined to be in a central position.

[0031] Optionally, when determining the head rotation direction and rotation times of the visually impaired person in the front mode or the back mode in step 204, the method detects whether the head state changes according to a debounce strategy; if the head state changes and reaches a debounce frame number, the head rotation times are counted.

[0032] Optionally, in step 3, the construction process of the standardized reference table specifically comprises:

[0033] collecting the head rotation times of a plurality of visually impaired persons in a standardized obstacle avoidance and search task scenario to obtain real samples;

[0034] collecting the head rotation times of a simulated visually impaired person in a standardized obstacle avoidance and search task scenario to obtain simulation samples;

[0035] Integrate real samples and simulation samples, and count the number of head rotations and fluctuation ranges in the standardized obstacle avoidance and search task scenarios of different field of view range groups to obtain a standardized reference table.

[0036] In a second aspect, a behavior assessment system for the visually impaired includes:

[0037] A data acquisition module for acquiring dynamic behavior videos of the visually impaired;

[0038] A head rotation statistics module for analyzing the dynamic behavior videos using a human pose recognition library and deep learning algorithms to determine the head rotation direction and number of rotations of the visually impaired;

[0039] A visual aid effect evaluation module for comparing and analyzing the head rotation direction and number of rotations of the visually impaired with a pre-established standardized reference table and evaluating the actual use effect of the magnifying visual aid or field of view expanding visual aid worn by the visually impaired, and generating an individualized rehabilitation plan according to the actual use effect;

[0040] The head rotation statistics module specifically includes:

[0041] Frame-by-frame acquisition of frame data in the dynamic behavior videos, and extraction of human pose key point position coordinates and confidence scores, face key point position coordinates and confidence scores, and head region position coordinates in each frame using a human pose recognition library and deep learning algorithms;

[0042] Calculate the average confidence of the face key points in each frame, and draw a confidence waveform graph according to the average confidence of the face key points in each frame;

[0043] Determine a first threshold value according to the confidence waveform graph, and determine whether the visually impaired person in the dynamic behavior video is in a front mode or a back mode according to the first threshold value;

[0044] Determine the head rotation direction and number of rotations of the visually impaired person in the front mode or back mode according to the human pose key point position coordinates and confidence scores, face key point position coordinates and confidence scores, and head region position coordinates.

[0045] Compared with the prior art, the present application has at least the following beneficial effects:

[0046] The application provides a visual impairment behavior ability evaluation method, which comprises the following steps: acquiring a dynamic behavior video of a visual impairment person; analyzing the dynamic behavior video by using a human body posture recognition library and a deep learning algorithm, and determining the head rotation direction and rotation times of the visual impairment person; comparing and analyzing the head rotation direction and rotation times of the visual impairment person with a pre-established standardized reference table, and evaluating the actual use effect of a magnifying visual aid or a visual field expansion visual aid worn by the visual impairment person, and generating a personalized rehabilitation scheme according to the actual use effect. The method provided by the application can more comprehensively and accurately reflect the visual behavior ability of the low vision group by evaluating the head rotation times of the visual impairment person in dynamic behaviors such as obstacle avoidance and searching, thereby providing strong support for formulating a personalized rehabilitation scheme. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more intuitively illustrate the prior art and the application, exemplary drawings are given below. It should be understood that the specific shapes, structures shown in the drawings should not be regarded as limiting conditions in the implementation of the application; for example, based on the technical concepts disclosed in the application and the exemplary drawings, those skilled in the art can easily make routine adjustments or further optimizations to some units (components) in terms of increase / decrease / assignment, specific shape, positional relationship, connection mode, size ratio relationship, etc.

[0048] Figure 1 A visual impairment group behavior ability evaluation method flow chart provided by the embodiment one of the application;

[0049] Figure 2 A visual impairment group behavior ability evaluation method structure schematic diagram provided by the embodiment one of the application;

[0050] Figure 3 A head rotation calculation algorithm flow chart provided by the embodiment one of the application;

[0051] Figure 4 A human body posture key point position and face key point schematic diagram provided by the embodiment one of the application;

[0052] Figure 5 A head horizontal rotation direction judgment and counting algorithm flow chart in a front mode provided by the embodiment one of the application;

[0053] Figure 6 A visual impairment person personalized rehabilitation flow chart provided by the embodiment one of the application;

[0054] Figure 7 A head-mounted binocular adjustable aperture device schematic diagram provided by the embodiment one of the application. DETAILED DESCRIPTION

[0055] The application will be further described in detail below with reference to the accompanying drawings.

[0056] In the description of the present application: unless otherwise specified, the meaning of "a plurality" is two or more. The terms "first", "second", "third" and the like in the present application are intended to distinguish the objects referred to, and do not have a special meaning in the technical connotation aspect (for example, it should not be understood as emphasizing importance or order, etc.). The expressions "include", "contain", "have" and the like also mean "not limited to" (some units, components, materials, steps, etc.).

[0057] The terms such as "upper", "lower", "left", "right", "intermediate" and the like referred to in the present application are generally indications made for the purpose of intuitive understanding with reference to the drawings, and are not absolute limitations on the positional relationship in the actual product.

[0058] Embodiment one

[0059] Please refer to Figure 1 and Figure 2 The embodiment provides a visual impairment group behavior ability evaluation method, which comprises the following steps:

[0060] S1: acquiring a dynamic behavior video of a visually impaired person;

[0061] Specifically, this step acquires a dynamic behavior video (including obstacle avoidance, search and other dynamic behaviors) of a visual impairment group in real time through an industrial camera, aiming to meet the demand for high-quality video acquisition. The resolution of the selected industrial camera is not less than 1920x1080, thereby ensuring the definition of video acquisition, and the video acquisition frame rate is set to be between 30 and 40 FPS. The industrial camera is supported through a fixing device on a camera support, the aperture and focal length of the industrial camera are first adjusted to ensure that the person is clearly presented in the camera field of view. Then, the position of the industrial camera on the camera support is adjusted so that the person can be completely captured by the industrial camera. The industrial camera is connected with a host through a USB3.0 industrial data line, and video data acquisition and saving are completed for subsequent analysis and processing.

[0062] This step is to ensure that the visually impaired person can be clearly presented in the video picture, and special attention is paid to the layout and angle adjustment of the industrial camera during design, so that the movements of the visually impaired person can be accurately captured and recorded in different environments and scenes. Through the high-resolution industrial camera and the stable camera support, the picture blur caused by the motion trajectory or posture change of the visually impaired individual can be effectively reduced, the definition and usability of the video content are ensured, and reliable data support is provided for subsequent behavior ability evaluation.

[0063] S2: analyzing the dynamic behavior video by using a human body posture recognition library and a deep learning algorithm, and determining the head rotation direction and rotation times of the visually impaired person;

[0064] Referring to Figure 3 , this step processes and analyzes each frame in the dynamic behavior video of the visually impaired person in the obstacle avoidance, search and other dynamic processes. By extracting the human posture key point, face key point position coordinates and confidence, and the head region position coordinates, based on these data, it can be judged whether the visually impaired person is facing the camera or facing away from the camera, and combined with the algorithm in different modes, the head rotation direction is further analyzed and the rotation times are counted. Specifically, it includes:

[0065] S201: frame-by-frame acquisition of frame data in the dynamic behavior video, and extraction of human posture key point position coordinates and confidence score, face key point position coordinates and confidence score, and head region position coordinates in each frame using a human posture recognition library and a deep learning algorithm;

[0066] Specifically, referring to Figure 4 , this step performs 2D human whole body posture recognition by calling the OpenPose human posture recognition library, and uses its deep learning algorithm (preferably YOLOv5 target detection algorithm) to process each frame in the video collected by the visually impaired person in the obstacle avoidance, search and other tasks. Extract and save the position coordinates (x, y) of the human posture key point Body_KeyPoint and the face key point Face_KeyPoint of the visually impaired person in each frame and the corresponding confidence score Score.

[0067] S202: calculate the average confidence of the face key point in each frame, and draw a confidence waveform graph according to the average confidence of the face key point in each frame;

[0068] Specifically, this step averages the confidence scores Score of the face key points Face_KeyPoint of each frame, calculates the average confidence of the face key points of all frames, and draws a waveform graph of the average confidence.

[0069] S203: determine a first threshold value according to the confidence waveform graph, and determine whether the visually impaired person in the dynamic behavior video is in a front mode or a back mode according to the first threshold value;

[0070] Specifically, this step selects a suitable first threshold value according to the waveform graph, and the first threshold value must ensure that the visually impaired person in each frame of the video can be accurately distinguished as facing the camera or facing away from the camera, so as to divide the characters in the video into a front mode Front_Mode and a back mode Back_Mode.

[0071] S204: determine the head rotation direction and rotation times of the visually impaired person in the front mode or back mode according to the human posture key point position coordinates and confidence score, face key point position coordinates and confidence score, and head region position coordinates.

[0072] Referring to Figure 5 , the step of determining the head rotation direction and rotation times of the visually impaired person in the front mode specifically includes:

[0073] S2041: determining the horizontal direction state of the head according to the confidence difference between the right ear confidence and the left ear confidence;

[0074] Specifically, in the algorithm initialization stage, two independent state machines, a horizontal direction state machine and a vertical direction state machine, are created. The horizontal direction state machine is used to track the horizontal direction of the head (left turn, right turn, front), and the vertical direction state machine is used to track the vertical direction of the head (upward tilt, downward tilt, front). Each state is represented by a set of predefined state values, where the horizontal direction state: Horizontal_Center represents that the head is in the center, Horizontal_Left represents that the head is turning to the left, and Horizontal_Right represents that the head is turning to the right. The vertical direction state: Vertical_Center represents that the head is in the center, Vertical_Up represents that the head is tilted upward, and Vertical_Down represents that the head is tilted downward. The current state is initialized to Horizontal_Center.

[0075] This step is based on the confidence difference dif between the right ear confidence Right_Ear and the left ear confidence Left_Ear to determine whether the head is turning. Specifically, the confidence difference dif between the right ear confidence and the left ear confidence is calculated. If the confidence difference dif is greater than the second threshold, it is determined that the head is turning to the left, and the state changes to Horizontal_Left. If the confidence difference dif is less than the negative second threshold, it is determined that the head is turning to the right, and the state changes to Horizontal_Right. If the confidence difference dif is between the second threshold and the negative of the second threshold, it is determined that the head is in the middle, and the state changes to Horizontal_Center.

[0076] It should be noted that when determining the horizontal direction (left turn, right turn, front) of the head based on the confidence difference between the left and right ears in the horizontal direction, smoothing data can be used to reduce noise.

[0077] S2042: determining the vertical direction state of the head according to the relative position coordinates of the nose and the ears and the shoulders;

[0078] The vertical state of this step is determined by the relative positions of the nose and the shoulder and the ear, and the relative proportion method is used to determine whether the head is tilted up, tilted down or straight. Specifically, the distance from the center of the shoulder to the ear is calculated as a reference; then the vertical ratio between the nose and the center of the shoulder is calculated, i.e. the relative distance of the nose to the shoulder to the distance from the shoulder to the ear. In the initial stage, the base ratio is determined by collecting vertical ratio data for a period of time, and the standard deviation is calculated according to the data, and then a dynamic threshold is set.

[0079] If the vertical_ratio is greater than the set tilt-up threshold, it indicates that the head is tilted up, and the state changes to Vertical_Up; if the vertical_ratio is less than the set tilt-down threshold, it indicates that the head is tilted down, and the state changes to Vertical_Down; if the vertical_ratio is within the base range, it indicates that the head is in the center, and the state changes to Vertical_Center.

[0080] It should be noted that this step determines the vertical state of the head (tilted up, tilted down, straight) by the relative positions of the nose and the shoulder in the vertical direction, and sets the reference value by calibration for a period of time in the initial stage.

[0081] S2043: Determine the head rotation direction and rotation times according to the horizontal and vertical states.

[0082] In this embodiment, in order to avoid errors caused by instantaneous noise or unstable data, the algorithm introduces a "debounce" mechanism. The debounce frame number debounce_frames (default is 5) indicates how many frames of stable data the algorithm will wait before confirming the head rotation or change.

[0083] This step updates the state machine by calculating the horizontal and vertical states of the current frame, and detects whether a rotation has occurred according to the debounce strategy. When the state changes and reaches the debounce frame number, the number of head rotations is counted, which specifically includes:

[0084] When a new state is detected that is different from the current state:

[0085] Initialize the pending state: if there is no pending state in this direction, set the pending state to the new state and the counter to 1; if there is a pending state and it is the same as the new state, increase the counter by 1; if there is a pending state but it is different from the new state, update the pending state to the new state and reset the counter to 1.

[0086] State confirmation: when the counter is greater than or equal to debounce_frames, update the current state to the pending state, clear the pending state, and reset the counter to 0; otherwise, keep the current state unchanged.

[0087] When the detected new state is the same as the current state: clear the pending state and set the counter to 0.

[0088] After each input of new key point data, the algorithm calculates the horizontal and vertical states of the current frame. If a state update is detected, the rotation count is triggered:

[0089] Horizontal rotation: if the horizontal state is updated from Horizontal_Center to Horizontal_Left, a left turn is recorded; if the horizontal state is updated from Horizontal_Center to Horizontal_Right, a right turn is recorded.

[0090] Vertical rotation: if the vertical state is updated from Vertical_Center to Vertical_Up, a head-up is recorded; if the vertical state is updated from Vertical_Center to Vertical_Down, a head-down is recorded.

[0091] This step determines the head rotation direction and rotation count of the visually impaired person in the back mode, and specifically includes:

[0092] S2041: Determine the horizontal state of the head according to the confidence difference between the right ear confidence and the left ear confidence;

[0093] Specifically, for the back mode, the method used to count the left and right rotation of the head of the visually impaired person is consistent with that of the front mode. Since the back mode cannot obtain the accurate coordinates of the face key points, the optical flow method is used to count the up and down rotation of the head. Only the vertical state calculation is different from that of the front mode, but the basic idea of counting the head rotation of the visually impaired person in the back mode is consistent with that of the front mode.

[0094] S2042: Determine the target area according to the head region position coordinates, and use the optical flow method to track the feature points in the target area to determine the vertical state of the head;

[0095] The state in the vertical direction of this step is determined by the position coordinates of the head region of the person, and the motion of the head is determined by tracking feature points in the region using the optical flow method. Specifically, the rotation direction of the head is determined by analyzing the vertical component of the optical flow vector: if the vertical component of the optical flow vector exceeds the set upward threshold, it indicates that the head is tilted upward, and the state changes to Vertical_Up; if the vertical component of the optical flow vector is lower than the set downward threshold, it indicates that the head is tilted downward, and the state changes to Vertical_Down; if the vertical component of the optical flow vector is within the set range, it indicates that the head is in the center, and the state changes to Vertical_Center.

[0096] S2043: Determine the head rotation direction and rotation times according to the horizontal direction state and the vertical direction state.

[0097] Specifically, in the back mode, the embodiment also updates the state machine by calculating the horizontal and vertical states of the current frame, and detects whether rotation has occurred according to the debounce strategy. When the state changes and reaches the debounce frame number, the number of head rotations is counted. The principle of this process is the same as that in the front mode, and will not be repeated here.

[0098] S3: Compare and analyze the head rotation direction and rotation times of the visually impaired person with the pre-established standardized reference table, and evaluate the actual use effect of the magnifying aid or the field expansion aid worn by the visually impaired person, and generate a personalized rehabilitation plan according to the actual use effect.

[0099] See Figure 6 , this step aims to analyze the changes in head rotation behavior of visually impaired persons during the execution of dynamic tasks (such as obstacle avoidance and searching) while wearing aids, to evaluate the actual use effect of magnifying aids and field expansion aids in assisting visually impaired persons in daily life, and to provide precise rehabilitation and assistance solutions for visually impaired persons.

[0100] The construction process of the standardized reference table specifically includes:

[0101] S301: Collect the number of head rotations of a plurality of visually impaired persons in a standardized obstacle avoidance and searching task scenario to obtain real samples;

[0102] Specifically, for visually impaired individuals with known field of view limitations (such as a field of view range of 10°, 20°, 30°, etc.), the head rotation statistics module is used to collect the number of head rotations in a standardized obstacle avoidance and searching task scenario to establish a basic data set.

[0103] S302: Collect the number of head rotations of a plurality of visually impaired persons in a standardized obstacle avoidance and searching task scenario to obtain real samples;

[0104] Specifically, the present step combines Figure 7 The head-mounted binocular adjustable aperture device shown simulates different degrees of visual field restriction (such as 10°, 20°, 30°) by adjusting the aperture size, thereby simulating the visual impairment effect in healthy individuals. In the same standardized task scenario, the task execution process video of the simulated visual impairment individual is collected, and the head rotation data is recorded, so as to expand the sample quantity and improve the statistical reliability and representativeness of the evaluation result.

[0105] S303: Integrate real samples and simulated samples, and count the head rotation times and fluctuation ranges of different visual field range groups in the standardized obstacle avoidance and search task scenarios to obtain a standardized reference table.

[0106] Specifically, the present step integrates real sample and simulated sample data, counts the average head rotation times and fluctuation ranges of different visual field range groups (such as 10°, 20°, 30°) in the obstacle avoidance and search task, and constructs a standardized reference table, as shown in Table 1:

[0107] Table 1

[0108]

[0109] In the same standardized obstacle avoidance and search task environment, the head rotation data of the visual impairment individual wearing the magnifying aid or the visual field expanding aid is collected, and compared and analyzed with the reference table, so as to calculate the equivalent visual field range of the individual after wearing the aid. Based on the result, the improvement degree of the visual field range and the behavior ability of the aid in actual use can be evaluated.

[0110] Based on the actual effect of the aid and the equivalent visual field range of the individual evaluated by the table lookup method, a differentiated personalized rehabilitation plan can be developed, specifically including:

[0111] If the evaluation result shows that the effect of the aid is significantly lower than the theoretical expectation, it means that the rehabilitation difficulty of the visual impairment individual is high, and the aid has limited improvement on the behavior ability. It is suggested to adjust the parameters of the aid (such as increasing the magnification or expanding the visual field range), and cooperate with high-intensity behavior training to improve the rehabilitation effect.

[0112] If the evaluation result is slightly lower than the theoretical expectation, it means that the rehabilitation condition is good, and the aid has a certain effect on improving the behavior ability. It is suggested to moderately optimize the aid, and cooperate with medium-intensity behavior training to further improve the functional performance.

[0113] If the evaluation result is close to or reaches the theoretical expectation, it means that the rehabilitation effect is good, and the aid has a significant improvement on the behavior ability. It is suggested to continue using the current device, and cooperate with low-intensity behavior training to realize stable rehabilitation support.

[0114] The method for evaluating the behavior ability of the visually impaired group provided by the embodiment collects video data of the visually impaired person in the process of obstacle avoidance and searching in real time through a high-resolution industrial camera, ensures clear images and reduces motion blur, and analyzes the rotation direction and number of the head of the visually impaired person by extracting the human posture and facial key point information in the video, judges whether the person faces or backs the camera, and calculates the horizontal and vertical changes of the head rotation; finally, by analyzing the head rotation behavior of the visually impaired individual in the standardized obstacle avoidance and searching task, and combining real and simulated sample data, a reference table between the field of view range and the behavior performance is established. The table can calculate the equivalent field of view range after wearing the magnifying or field of view expanding visual aid, quantify the actual assistance effect of the visual aid, and develop a personalized rehabilitation program according to the evaluation results. According to the evaluation level, the parameters of the visual aid can be optimized or different intensity behavior training can be recommended to improve the rehabilitation efficiency and self-care ability. The method not only realizes the objective evaluation of the behavior ability of the visually impaired individual, but also provides fine and differentiated assistance strategies according to the equivalent field of view performance, effectively improving the life quality and rehabilitation effect of the visually impaired group.

[0115] In summary, the method for evaluating the behavior ability of the visually impaired group provided by the embodiment overcomes the problem that the existing visual behavior evaluation method mainly focuses on static visual test and ignores the dynamic reaction related to vision in the behavior process. The method aims to evaluate the head rotation frequency index of the visually impaired group in dynamic behaviors such as obstacle avoidance and searching, more comprehensively and accurately reflect the visual behavior ability of the low vision group, and provide strong support for developing personalized rehabilitation programs.

[0116] Embodiment Two

[0117] The embodiment provides a behavior ability evaluation system for the visually impaired, comprising:

[0118] A data acquisition module is configured to acquire dynamic behavior videos of the visually impaired person.

[0119] A head rotation statistics module is configured to analyze the dynamic behavior videos by using a human posture recognition library and a deep learning algorithm, and determine the head rotation direction and rotation frequency of the visually impaired person.

[0120] A visual aid effect evaluation module is configured to compare and analyze the head rotation direction and rotation frequency of the visually impaired person with a pre-established standardized reference table, evaluate the actual use effect of the magnifying visual aid or field of view expanding visual aid worn by the visually impaired person, and generate a personalized rehabilitation program according to the actual use effect.

[0121] The head rotation statistics module specifically comprises:

[0122] Frame data in the dynamic behavior video is acquired frame by frame, and human body posture key point position coordinates and confidence scores, face key point position coordinates and confidence scores, and head region position coordinates in each frame are extracted by using a human body posture recognition library and a deep learning algorithm;

[0123] An average value of the confidence of the face key point in each frame is calculated, and a confidence waveform graph is drawn according to the average value of the confidence of the face key point in each frame;

[0124] A first threshold is determined according to the confidence waveform graph, and it is determined whether the visually impaired person in the dynamic behavior video is in a front mode or a back mode according to the first threshold;

[0125] The head rotation direction and rotation times of the visually impaired person in the front mode or the back mode are determined according to the human body posture key point position coordinates and confidence scores, the face key point position coordinates and confidence scores, and the head region position coordinates.

[0126] In the behavior ability evaluation system for the visually impaired person provided in the embodiment, the data acquisition module acquires video data of the visually impaired population in the obstacle avoidance and search process in real time through a high-resolution industrial camera, ensures clear images and reduces motion blur. The head rotation statistics module extracts human body posture and face key point information in the video by using OpenPose and YOLOv5 algorithms, analyzes the rotation direction and times of the head of the visually impaired person, judges whether the head faces or backs the camera, and calculates the changes in the horizontal and vertical directions of the head rotation on this basis. The visual aid effect evaluation module judges the difference between the actual use effect and the theoretical effect of the visual aid worn by the visually impaired person according to the table lookup method, and formulates a personalized rehabilitation scheme for the visually impaired person. The system not only can accurately evaluate the behavior ability of the visually impaired population, but also can provide customized rehabilitation schemes for individuals.

[0127] The specific implementation contents of each module in the behavior ability evaluation system for the visually impaired person can be referred to the limitation of the behavior ability evaluation method for the visually impaired person in the above, which will not be described here.

[0128] The technical features of the above embodiments can be combined in any manner (as long as the combination of the technical features does not exist contradiction), in order to make the description simple, not all possible combinations of the technical features in the above embodiments are described; these embodiments not explicitly written, should also be considered as the scope of the present disclosure.

Claims

1. A method for assessing the behavior of a visually impaired person, characterized in that, The application comprises the following steps: Step 1: obtaining a dynamic behavior video of a visually impaired person; Step 2: analyzing the dynamic behavior video by using a human body posture recognition library and a deep learning algorithm, and determining the head rotation direction and rotation times of the visually impaired person; Step 3: comparing and analyzing the head rotation direction and rotation times of the visually impaired person with a pre-established standardized reference table, and evaluating the actual use effect of the magnifying aid or the field of view expanding aid worn by the visually impaired person, and generating a personalized rehabilitation scheme according to the actual use effect; Step 2 specifically comprises the following steps: Step 201: obtaining frame data in the dynamic behavior video frame by frame, and extracting human body posture key point position coordinates and confidence scores, face key point position coordinates and confidence scores, and head region position coordinates in each frame by using a human body posture recognition library and a deep learning algorithm; Step 202: calculating the average confidence of the face key points in each frame, and drawing a confidence waveform graph according to the average confidence of the face key points in each frame; Step 203: determining a first threshold value according to the confidence waveform graph, and determining whether the visually impaired person in the dynamic behavior video is in a front mode or a back mode according to the first threshold value; Step 204: determining the head rotation direction and rotation times of the visually impaired person in the front mode or the back mode according to the human body posture key point position coordinates and confidence scores, the face key point position coordinates and confidence scores, and the head region position coordinates.

2. The method of claim 1, wherein, In step 201, the deep learning algorithm adopts a YOLOv5 target detection algorithm.

3. The method of claim 1, wherein, In step 204, when determining the head rotation direction and rotation times of the visually impaired person in the front mode, the method specifically comprises the following steps: judging the horizontal direction state of the head according to the confidence difference between the right ear confidence and the left ear confidence; judging the vertical direction state of the head according to the relative position coordinates of the nose, the ear and the shoulder; determining the head rotation direction and rotation times according to the horizontal direction state and the vertical direction state.

4. The method of claim 3, wherein, When judging the horizontal direction state of the head according to the confidence difference between the right ear confidence and the left ear confidence, the method specifically comprises the following steps: calculating the confidence difference between the right ear confidence and the left ear confidence, if the confidence difference is greater than a second threshold value, it is judged that the head is turning to the left; if the confidence difference is less than a negative second threshold value, it is judged that the head is turning to the right; if the confidence difference is between the second threshold value and the negative of the second threshold value, it is judged that the head is in the middle.

5. The method of claim 3, wherein, When judging the vertical direction state of the head according to the relative position coordinates of the nose, the ear and the shoulder, the method specifically comprises the following steps: calculating the distance from the shoulder center to the ear as a reference; calculating the vertical proportion between the nose and the shoulder center; if the vertical proportion is greater than a preset upward threshold value, it is judged that the head is upward; if the vertical proportion is less than a preset downward threshold value, it is judged that the head is downward; if the vertical proportion is within the reference range, it is judged that the head is in the center position.

6. The method of claim 1, wherein, In step 204, when determining the head rotation direction and rotation times of the visually impaired person in the back mode, the method specifically comprises the following steps: judging the horizontal direction state of the head according to the confidence difference between the right ear confidence and the left ear confidence; A target region is determined according to the head region position coordinates, and a feature point in the target region is tracked by using an optical flow method to determine the vertical direction state of the head; The head rotation direction and rotation times are determined according to the horizontal direction state and the vertical direction state.

7. The method of claim 6, wherein, When the vertical direction state of the head is determined by tracking the feature point in the target region by using the optical flow method, the following steps are specifically included: if the vertical component of the optical flow vector is greater than a preset upward threshold, it is determined that the head is tilted upward; if the vertical component of the optical flow vector is less than a preset downward threshold, it is determined that the head is tilted downward; and if the vertical component of the optical flow vector is within a preset range, it is determined that the head is in a central position.

8. The method of claim 1, wherein, In step 204, when the head rotation direction and rotation times of the visually impaired person in the front mode or the back mode are determined, it is detected whether the head state changes according to a de-bouncing strategy. If the head state changes and reaches the de-bouncing frame number, the head rotation times are counted.

9. The method of claim 1, wherein, In step 3, the construction process of the standardized reference table specifically includes: Collect the head rotation times of a plurality of visually impaired persons in the standardized obstacle avoidance and search task scene to obtain real samples; Collect the head rotation times of simulated visually impaired persons in the standardized obstacle avoidance and search task scene to obtain simulation samples; Integrate the real samples and the simulation samples, and count the head rotation times and fluctuation ranges of different visual field range groups in the standardized obstacle avoidance and search task scene to obtain the standardized reference table.

10. A system for assessing the behavior of a visually impaired person, characterized in that it comprises: It includes: A data acquisition module for acquiring a dynamic behavior video of a visually impaired person; A head rotation counting module for analyzing the dynamic behavior video by using a human body posture recognition library and a deep learning algorithm, and determining the head rotation direction and rotation times of the visually impaired person; An aid effect evaluation module for comparing and analyzing the head rotation direction and rotation times of the visually impaired person with a pre-established standardized reference table, and evaluating the actual use effect of a magnifying aid or a visual field expanding aid worn by the visually impaired person, and generating an individualized rehabilitation plan according to the actual use effect; The head rotation counting module specifically includes: Frame-by-frame acquisition of frame data in the dynamic behavior video, and extraction of human body posture key point position coordinates and confidence scores, face key point position coordinates and confidence scores, and head region position coordinates in each frame by using a human body posture recognition library and a deep learning algorithm; Calculation of the confidence average of the face key points in each frame, and drawing of a confidence waveform graph according to the confidence average of the face key points in each frame; Determination of a first threshold according to the confidence waveform graph, and determination of whether the visually impaired person in the dynamic behavior video is in a front mode or a back mode according to the first threshold; Determination of the head rotation direction and rotation times of the visually impaired person in the front mode or the back mode according to the human body posture key point position coordinates and confidence scores, the face key point position coordinates and confidence scores, and the head region position coordinates.

Images