Intelligent auxiliary rehabilitation exercise multifunctional wheelchair

Abstract

The invention provides an intelligent auxiliary rehabilitation exercise multifunctional wheelchair, which relates to an intelligent control technology, and comprises a wheelchair main body, a control motor for controlling the wheelchair main body to act, and a controller connected with the control motor, the posture recognition unit is used for acquiring height adjustment parameters of a wheelchair assembly in a wheelchair main body by a user side based on a wheelchair model and performing height adjustment on the wheelchair assembly according to the height adjustment parameters, and the element selection unit is used for determining posture recognition modes selected by the user side and comprises a side placement mode and a flat placement mode. The control element selected by the user side is determined according to the posture recognition mode, the direction adjusting unit is used for shooting a posture image, the guiding direction corresponding to the control element in the posture image is recognized, and the moving direction of the wheelchair body is adjusted to be the guiding direction. The speed adjusting unit is used for shooting a dynamic image set. And identifying the adjusting speed corresponding to the control element in the dynamic image set, and adjusting the movement speed of the wheelchair main body to be the adjusting speed.

Description

Technical Field

[0001] The invention relates to intelligent control technology, and in particular to an intelligently assisted multifunctional wheelchair for rehabilitation exercise. Background Art

[0002] In recent years, with the rapid development of artificial intelligence, the Internet of Things and sensor technology, the functions of smart medical wheelchairs have been greatly improved.

[0003] In the prior art, the movement of smart medical wheelchairs is usually controlled by a grip, which requires the user to continuously hold the grip to maintain the movement of the wheelchair or adjust the direction. However, long-term grip operation may cause hand fatigue in the user, especially for users with limited mobility, such continuous operation may be more difficult.

[0004] Therefore, how to more conveniently realize the control of the intelligent medical wheelchair has become an urgent problem to be solved. Summary of the invention

[0005] The present invention provides an intelligent assisted rehabilitation multifunctional wheelchair, comprising a wheelchair body, a control motor for controlling the movement of the wheelchair body, and a controller connected to the control motor, which can more conveniently realize the control of the intelligent medical wheelchair.

[0006] In a first aspect of the present invention, there is provided an intelligently assisted rehabilitation multifunctional wheelchair, comprising a wheelchair body, a control motor for controlling the movement of the wheelchair body, and a controller connected to the control motor, wherein the controller comprises: A height adjustment unit, used for obtaining a height adjustment parameter of a wheelchair component in the wheelchair body based on a wheelchair model by a user end, and adjusting the height of the wheelchair component according to the height adjustment parameter; An element selection unit, used to determine a gesture recognition mode selected by a user terminal, wherein the gesture recognition mode includes a side-laying mode and a horizontal-laying mode, and to determine a control element selected by the user terminal according to the gesture recognition mode; a direction adjustment unit, used for capturing a posture image, identifying a guiding direction corresponding to the control element in the posture image, and adjusting the movement direction of the wheelchair body to the guiding direction; The speed adjustment unit is used to shoot a dynamic image set, identify the adjustment speed corresponding to the control element in the dynamic image set, and adjust the movement speed of the wheelchair body to the adjustment speed.

[0007] Optionally, in a possible implementation manner of the first aspect, the height adjustment unit is used to obtain a height adjustment parameter of a wheelchair component in the wheelchair body based on a wheelchair model by the user end, and adjust the height of the wheelchair component according to the height adjustment parameter, including: The height adjustment unit obtains a component module selected by the user terminal based on the wheelchair model, and determines a height value selected by the user terminal based on a height sliding axis corresponding to the component module; The height value is bound to the component module to obtain a height adjustment parameter, a wheelchair component corresponding to the component module is determined, and the height of the wheelchair component is adjusted according to the height adjustment parameter.

[0008] Optionally, in a possible implementation manner of the first aspect, the element selection unit is used to determine a gesture recognition mode selected by the user terminal, the gesture recognition mode includes a side-laying mode and a horizontal-laying mode, and determining the control element selected by the user terminal according to the gesture recognition mode includes: Determine a gesture reference image corresponding to the gesture recognition mode and send it to the user terminal, and when the gesture recognition mode is the side placement mode, determine a preset hand element corresponding to the side placement mode as a control element; When the posture recognition mode is the flat mode, photographing an element recognition image, and extracting a hand contour in the element recognition image; Acquire multiple salient points in the hand contour, and acquire distance and angle information between each salient point and a center point of the hand contour; Determine the protruding point with the largest distance as the target point, traverse multiple preset angle intervals based on the angle information of the target point, determine the preset angle interval where the angle information is located as the target angle interval, and determine the preset hand element corresponding to the target angle interval as the control element.

[0009] Optionally, in a possible implementation manner of the first aspect, acquiring multiple salient points in the hand contour, and acquiring distance and angle information between each of the salient points and a center point of the hand contour includes: Traversing adjacent pixel points in the outer contour of the hand contour, obtaining vector angles between adjacent pixel points, and determining pixel points whose vector angles are greater than a vector angle threshold as salient points; Connecting each of the protruding points and the center point of the hand contour to obtain a plurality of connecting lines, and determining the distance between each of the protruding points and the center point according to the length of each of the connecting lines; Determine the midline of the two connecting lines with the largest angle, generate a reference line perpendicular to the midline, and determine the pointing direction of the reference line according to the azimuth side of the hand contour, the azimuth side includes a left side and a right side, and the left side and the right side correspond to opposite pointing directions; The direction from the center point to each of the salient points is obtained as the pointing direction of each connecting line, and the angle information of each of the salient points and the center point of the hand contour is determined according to the vector angle corresponding to the baseline and the pointing direction of each of the connecting lines.

[0010] Optionally, in a possible implementation manner of the first aspect, the direction adjustment unit is used to capture a posture image, identify a guiding direction corresponding to the control element in the posture image, and adjust the movement direction of the wheelchair body to the guiding direction, including: Based on the voice interaction unit broadcasting the calibration prompt information, the direction adjustment unit is controlled to shoot the calibration image, and the hand contour in the calibration image is extracted; Acquire multiple salient points in the hand contour, and determine the salient point with the largest distance as the calibration point according to the distance and angle information between each salient point and the center point of the hand contour; Determine a preset hand element corresponding to the angle information of the calibration point as a calibration element, and when the calibration element is consistent with the control element, respond with calibration success information, and broadcast selected prompt information based on a voice interaction unit; The control direction adjustment unit captures a posture image, extracts a hand contour in the posture image and a finger contour in the hand contour, determines the pointing direction of the finger contour as a guiding direction, and adjusts the movement direction of the wheelchair body to the guiding direction.

[0011] Optionally, in a possible implementation manner of the first aspect, extracting a hand contour in the gesture image and a finger contour in the hand contour, and determining a pointing direction of the finger contour as a guiding direction includes: When the number of contours of the finger contour is greater than the reference number, an error prompt message is broadcasted according to the voice interaction unit; When the number of contours of the finger contour is equal to the reference number, obtaining a pixel point with the largest curvature in the finger contour as a fingertip point; Determine the palm back region of the hand contour in the gesture image, and obtain a plurality of pixel points where the palm back region and the finger contour intersect as finger root points; The center point of multiple finger base points is obtained, and the direction from the center point to the finger tip point is determined as the guiding direction.

[0012] Optionally, in a possible implementation manner of the first aspect, the speed adjustment unit is used to capture a dynamic image set, identify an adjustment speed corresponding to the control element in the dynamic image set, and adjust the movement speed of the wheelchair body to the adjustment speed, including: Acquire voice interaction data of the voice interaction unit, convert the voice interaction data into text data, and when there is a speed keyword in the text data, send a hand placement reference map to the user terminal; Controlling the speed adjustment unit to shoot a first dynamic image, extracting a first finger contour in the first dynamic image, and connecting a fingertip point of the first finger contour and a center point of a plurality of finger base points to obtain a grade division line; Determine multiple level division points on the level division line and the division level of each level division point according to the preset number of levels, determine multiple control speeds corresponding to the user terminal, match the division levels and the control speeds one by one from small to large, generate a speed display diagram and send it to the user terminal; Controlling the speed adjustment unit to shoot a second dynamic image, extracting a second finger contour in the second dynamic image, the dynamic image set including the first dynamic image and the second dynamic image; Determine a reference length corresponding to each of the division levels, obtain a reference length closest to the finger length of the second finger contour as a target length, determine a control speed corresponding to the target length as an adjustment speed, and adjust the movement speed of the wheelchair body to the adjustment speed.

[0013] Optionally, in a possible implementation manner of the first aspect, determining a plurality of level division points on the level division line and a division level of each of the level division points according to a preset number of levels, determining a plurality of control speeds corresponding to the user terminal, matching the division levels and the control speeds one by one from small to large, generating a speed display graph and sending it to the user terminal, including: The level division line is equally divided based on the preset level number to obtain a plurality of level division points, and the length of the line segment from each level division point to the center point of the plurality of root points is determined as a reference length; Arrange the level division points from small to large according to the reference length to obtain a point sequence, arrange the preset division levels from small to large to obtain a level sequence, and match the level division points and division levels with the same arrangement position in the point sequence and the level sequence one by one; Retrieving a preset maximum speed threshold, offsetting the maximum speed threshold according to the height adjustment parameter to obtain a maximum adjustment threshold, and dividing the maximum adjustment threshold based on the preset number of levels to obtain a plurality of control speeds; A level slot corresponding to each level division point is generated in the first dynamic image, and each control speed is filled into the corresponding level slot to obtain a speed display diagram. The larger the division level of the level slot, the greater the control speed.

[0014] Optionally, in a possible implementation manner of the first aspect, the maximum speed threshold is offset according to the height adjustment parameter to obtain a maximum adjustment threshold, and the maximum adjustment threshold is divided based on the preset number of levels to obtain a plurality of control speeds, including: Obtaining an overall height value corresponding to the height adjustment parameter, and determining a standard height value corresponding to the maximum speed threshold; A speed offset coefficient is obtained according to a ratio of the standard height value to the overall height value, and a maximum adjustment threshold is obtained by multiplying the speed offset coefficient by the maximum adjustment threshold; A unit speed is obtained based on the ratio of the maximum adjustment threshold and the preset number of levels, the number of levels corresponding to each of the divided levels is determined, and a plurality of control speeds are obtained by multiplying the unit speed and each of the number of levels.

[0015] Optionally, in a possible implementation manner of the first aspect, acquiring a reference length closest to the finger length of the second finger contour as a target length, and determining a control speed corresponding to the target length as an adjustment speed includes: Fitting the second finger contour to obtain a fitting line, and determining the length between the endpoints of both ends of the fitting line as the finger length; Determine the length difference between each reference length and the target length, obtain the reference length with the smallest length difference as the target length, and determine the control speed corresponding to the target length as the adjustment speed.

[0016] The beneficial effects of the present invention are as follows: 1. The present invention can automatically adjust the motion state of the intelligent medical wheelchair according to the user's hand posture, so that the user can adjust the direction and speed of the wheelchair through simple gestures, thereby improving the convenience of the user when using the wheelchair.

[0017] 2. The present invention can customize the height of each component of the wheelchair by combining the actual height requirements of the user through the height adjustment unit in the controller, so that the intelligent medical wheelchair can better meet the height requirements of the user.

[0018] 3. When the present invention controls the movement process of the wheelchair, a variety of modes for recognizing hand postures can be provided for the user to choose from, and hand posture reference diagrams corresponding to different modes can be sent to the user end. The user can select the posture recognition mode he wants to use by clicking on the corresponding schematic diagram, so that the user can more intuitively understand the hand postures corresponding to different modes, so that the user can correctly execute the corresponding gestures to achieve control of the wheelchair movement process.

[0019] 4. The present invention can capture the user's hand posture image through the control direction adjustment unit, identify the guiding direction of the control element in the hand posture image, and automatically adjust the movement direction of the wheelchair according to the guiding direction, so that the user can flexibly control the movement direction of the wheelchair through simple gestures.

[0020] 5. The present invention can broadcast calibration prompt information through the voice interaction unit to guide the user to calibrate the hand posture, ensuring that the finger extended by the user is consistent with the control element, thereby improving the accuracy of direction control.

[0021] 6. When adjusting the movement speed of the wheelchair, the present invention can adjust the speed by identifying the length of the control element selected by the user. Specifically, the speed adjustment unit can be controlled to continuously shoot the user's hand posture to form a dynamic image set. The current length of the control element can be obtained from the dynamic image set, and the corresponding adjustment speed can be determined according to the current length. Therefore, the movement speed of the wheelchair can be adjusted accordingly according to the adjustment speed, thereby realizing precise adjustment of the movement speed of the wheelchair. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of the motion control process of the intelligent assisted rehabilitation multifunctional wheelchair provided in an embodiment of the present invention; Figure 2 A schematic diagram of determining the pointing direction corresponding to a connecting line provided by an embodiment of the present invention; Figure 3 A schematic diagram of determining a guiding direction provided by an embodiment of the present invention; Figure 4 A schematic diagram of a speed display diagram provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0025] See also Figure 1 , is a schematic diagram of a motion control flow of an intelligently assisted rehabilitation multifunctional wheelchair provided in an embodiment of the present invention, Figure 1The execution subject of the method shown may be a software and / or hardware device. The execution subject of the present application may include but is not limited to at least one of the following: user equipment, network equipment, etc. Among them, the user equipment may include but is not limited to computers, smart phones, personal digital assistants (Personal Digital Assistant, referred to as: PDA) and the electronic devices mentioned above. The network equipment may include but is not limited to a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of computers or network servers, wherein cloud computing is a type of distributed computing, a super virtual computer composed of a group of loosely coupled computers. This embodiment does not limit this.

[0026] The intelligent assisted rehabilitation multifunctional wheelchair in this solution includes a wheelchair body, a control motor for controlling the movement of the wheelchair body, and a controller connected to the control motor, wherein the wheelchair body refers to the intelligent assisted rehabilitation multifunctional wheelchair body, the control motor is the power source of the intelligent assisted rehabilitation multifunctional wheelchair, and is responsible for driving the movement and steering of the wheelchair body. The motor speed and steering are adjusted by receiving instructions from the controller, thereby realizing the wheelchair's forward, backward, acceleration, deceleration and steering functions. The controller is mainly responsible for receiving user input instructions, such as voice and gestures, and controlling the movement of the control motor according to the instructions, thereby realizing precise control of the wheelchair body. The controller mainly includes a height adjustment unit, an element selection unit, a direction adjustment unit and a speed adjustment unit, and the multimodal interaction technology between the multiple units brings a more natural, convenient and efficient operation mode to the intelligent medical wheelchair, so as to realize intelligent control of the wheelchair body height, posture recognition mode, movement direction and movement speed. When the controller performs intelligent control on the movement process through the above multiple units, it mainly includes steps S1 to S4, as follows: S1, a height adjustment unit, is used to obtain a height adjustment parameter of a wheelchair component in the wheelchair body based on a wheelchair model by a user end, and adjust the height of the wheelchair component according to the height adjustment parameter.

[0027] Among them, the height adjustment unit refers to a unit that can adjust the height of each wheelchair component in the wheelchair body, the user terminal refers to a terminal held by a user who uses an intelligent-assisted rehabilitation sports multifunctional wheelchair, such as a mobile phone, the wheelchair model refers to a virtual wheelchair model corresponding to the intelligent-assisted rehabilitation sports multifunctional wheelchair, and the user can preview and adjust the height of each wheelchair component through the wheelchair model, the wheelchair component refers to the various components that constitute the intelligent-assisted rehabilitation sports multifunctional wheelchair, including but not limited to seats, armrests, foot pedals, etc., the height adjustment unit can adjust the height of one or more of these components to meet the user's personalized needs, the height adjustment parameter refers to a parameter that can be used to adjust the height of the wheelchair component, including the virtual module corresponding to the wheelchair component and the corresponding height value, the height adjustment unit will drive the relevant control motor according to the height adjustment parameter, thereby realizing the height adjustment of the wheelchair component.

[0028] This solution proposes a multifunctional intelligent medical wheelchair, which can recognize the user's gestures by introducing advanced gesture recognition technology, and can automatically adjust the movement state of the intelligent medical wheelchair according to the user's gestures, so that the user can adjust the direction and speed of the wheelchair through simple gestures, thereby improving the convenience of the user when using the wheelchair.

[0029] In actual applications, the heights and proportions of different users may be different. Therefore, when using a wheelchair, the height requirements for wheelchair components such as seat height, armrest height, and footrest height may also be different. This solution can be combined with the user's actual height requirements. The height of each component of the wheelchair can be customized through the height adjustment unit in the controller, so that the smart medical wheelchair can better meet the user's height requirements.

[0030] Specifically, when the user needs to adjust the height, the user can interact with the controller of the wheelchair through the user terminal. The controller can control the height adjustment unit to generate a virtual model corresponding to the wheelchair, that is, the wheelchair model, and send it to the user terminal. The user can preview the wheelchair model on his or her smart terminal and select the wheelchair components that need to be adjusted according to his or her height and sitting habits, such as seats, armrests, foot pedals, etc. After selecting the wheelchair components to be adjusted in height, the user can configure the height of these wheelchair components through the user terminal. The user terminal can collect the height information input by the user and generate corresponding height adjustment parameters. The height adjustment parameters can provide precise guidance for subsequent adjustment work.

[0031] After generating the corresponding height adjustment parameters, the user end can send the height adjustment parameters to the wheelchair controller. After receiving the height adjustment parameters, the controller can pass them to the height adjustment unit. The height adjustment unit can drive the control motor to accurately adjust the height of the wheelchair components according to the height adjustment parameters, so that the wheelchair seat, armrests, footrests and other components can be adjusted to a height that matches the user's height.

[0032] In some embodiments, step S1 includes S11 to S12, which are specifically as follows: S11, the height adjustment unit obtains a component module selected by the user terminal based on the wheelchair model, and determines a height value selected by the user terminal based on a height sliding axis corresponding to the component module.

[0033] Among them, component modules refer to the modules corresponding to each wheelchair component in the wheelchair model, the height sliding axis refers to a pre-configured numerical axis that can determine the height value corresponding to the component module, each scale on the height sliding axis has a corresponding height value, and the height value refers to the height value corresponding to the wheelchair component determined by the user through the height sliding axis.

[0034] After generating a wheelchair model corresponding to the intelligent-assisted rehabilitation sports multifunctional wheelchair, the wheelchair model can be sent to the user end, and the user can select the component module corresponding to the wheelchair component to be height-adjusted from the wheelchair model through the user end. For example, if the user wants to adjust the height of the armrest of the wheelchair, the user can select the component module corresponding to the wheelchair armrest from the wheelchair model for corresponding height adjustment. After selecting the corresponding component module, when determining the height corresponding to the component module, in order to enable the user to intuitively adjust and determine the required height, the present solution can configure a corresponding height sliding axis for each component module on the wheelchair model, allowing the user to select and determine the height of the component module through sliding operations on the user end, and can intuitively and quickly obtain the height value configured by the user for the corresponding wheelchair component.

[0035] Specifically, the height sliding axis corresponding to the component module can be retrieved and displayed on the user end. The user can slide the sliding point on the height sliding axis through the user end to adjust the position of the sliding point on the height sliding axis. When the user stops sliding the sliding point, the current position of the sliding point on the height sliding axis can be recorded, and the height value corresponding to the current position can be determined, and the height value is determined as the height value corresponding to the component module.

[0036] S12, binding the height value with the component module to obtain a height adjustment parameter, determining a wheelchair component corresponding to the component module, and adjusting the height of the wheelchair component according to the height adjustment parameter.

[0037] In actual applications, there may be multiple component modules that are height-adjustable, and the height values ​​corresponding to each component module may be different. In order to determine the height values ​​corresponding to each component module, the determined height values ​​can be bound to the corresponding component modules, and the height adjustment parameters of the wheelchair components corresponding to the component modules can be obtained. For each component module in the wheelchair model, there is a corresponding wheelchair component in the wheelchair body. The wheelchair components corresponding to each component module are determined respectively, and the height of the wheelchair components can be adjusted according to the height adjustment parameters corresponding to the component modules.

[0038] Through the above implementation, the height value configured by the user can be quickly obtained, the efficiency and accuracy of height adjustment can be improved, and the wheelchair can be made to better meet the height requirements of the user.

[0039] S2, an element selection unit, is used to determine a gesture recognition mode selected by the user terminal, wherein the gesture recognition mode includes a side-laying mode and a horizontal-laying mode, and determine the control element selected by the user terminal according to the gesture recognition mode.

[0040] Among them, the element selection unit refers to a unit that selects specific fingers for motion control, the posture recognition mode refers to a mode for recognizing the user's hand posture, and the side-by-side mode refers to a mode in which the user's hand is clenched into a fist and placed sideways. In this mode, the element selection unit mainly recognizes the posture and direction of the thumb, thereby achieving precise control of the wheelchair's movement direction and speed. For example, when the thumb points forward, the wheelchair moves forward, and when it points to the left or right, the wheelchair turns left or right accordingly. The flat mode refers to a mode in which the user's hand is flat. In this mode, the element selection unit needs to determine the specific fingers used by the user for control, such as the index finger, middle finger, etc. The control element refers to the specific finger for motion control.

[0041] When the present solution controls the movement process of the wheelchair, it controls it through the user's hand posture. Before the movement control, the element selection unit can determine the specific control element according to the user's hand posture. Therefore, the element selection unit can identify the user's hand posture accordingly, so as to determine the specific finger for the movement control. In addition, the present solution provides a variety of modes for identifying hand postures for the user to choose. In order to help the user better understand and select the posture recognition mode, the hand posture reference schematic diagram corresponding to the different modes can be sent to the user end. The user can select the posture recognition mode he wants to use by clicking the corresponding schematic diagram. In order to ensure that the user can correctly perform the gesture and obtain an accurate control effect, the element selection unit can provide the user with real-time feedback and guidance. For example, after the user selects the side-laying mode, the element selection unit can prompt the user to keep the hand clenched and side-laying posture. After selecting the flat mode, the element selection unit can prompt the user to keep the hand flat. According to the posture recognition mode selected by the user, the element selection unit can determine the specific finger selected by the user end to control the movement process of the wheelchair, that is, the control element.

[0042] Based on the above embodiment, the specific implementation of step S2 may be: S21, determining a posture reference image corresponding to the posture recognition mode and sending it to a user terminal, and when the posture recognition mode is a side-by-side mode, determining a preset hand element corresponding to the side-by-side mode as a control element.

[0043] Specifically, according to the posture recognition mode selected by the user, the corresponding hand posture reference schematic diagram, i.e., the posture reference diagram, can be sent to the user end. For example, when the posture recognition mode selected by the user is the side-laying mode, the corresponding posture reference diagram is a schematic diagram of the hand posture being a fist and laid sideways. The posture reference diagram can clearly show the position and direction of the thumb, helping the user understand how to correctly place the hand to control the movement of the wheelchair. When the posture recognition mode selected by the user is the flat mode, the corresponding posture reference diagram is a schematic diagram of the hand being laid flat with the fingers pointing forward. In the posture reference diagram, the user can see the overall posture of the hand when laid flat, as well as the relative position and direction of each finger.

[0044] When the gesture recognition mode selected by the user is the side-positioning mode, the thumb can be pre-determined as a preset hand element corresponding to the side-positioning mode, so in the side-positioning mode, the thumb can be used as a control element in the motion control process.

[0045] Among them, the posture reference refers to the reference schematic diagram of the hand posture when placed in the corresponding recognition mode, and the preset hand element corresponding to the side placement mode refers to the thumb.

[0046] S22, when the gesture recognition mode is the flat mode, photographing an element recognition image, and extracting a hand contour in the element recognition image.

[0047] Specifically, when the posture recognition mode selected by the user is the flat mode, it is necessary to determine the specific fingers selected by the user for motion control. In actual applications, in order to realize posture recognition in the flat mode, a shooting device for overhead shooting can be configured at the corresponding position of the armrest of the multifunctional intelligent wheelchair. In the flat mode, the element selection unit can prompt the user to place the hand under the shooting device according to the hand posture in the corresponding posture reference diagram through voice broadcast, and can prompt the user to extend the finger to perform motion control. After the user places the hand according to the prompt and extends the selected finger, the controller can control the element selection unit to capture the image of the user's hand posture to obtain the corresponding element recognition image, and the corresponding hand contour can be extracted from the element recognition image through contour extraction technology.

[0048] S23, obtaining a plurality of salient points in the hand contour, and obtaining distance and angle information between each of the salient points and the center point of the hand contour.

[0049] It can be understood that the hand contour is composed of multiple fingers and palm edges. Since the fingertips or joints of the fingers are more prominent than the palm and other finger parts, the protruding parts appear as convex points of the local contour on the image. Therefore, when the user extends a finger, an obvious bulge can be formed on the hand contour, and the point with the obvious bulge can be determined as a convex point. In the flat mode, since the hand is in a relatively static state, the specific finger selected by the user for motion control can be more accurately determined by identifying the convex points, and the data processing amount can be reduced. Since the distance from the fingertip to the center of the palm and the corresponding angle of different fingers are different when they are extended, in order to more accurately identify the specific finger selected by the user, the element selection unit can be controlled to obtain the distance and angle information of each convex point relative to the center point of the hand contour. By comparing the distance and angle information of different convex points, the specific finger for motion control can be more accurately determined.

[0050] Among them, the protruding points refer to points on the hand contour that are obviously protruding, such as points corresponding to joints, points corresponding to fingertips, etc., and the angle information refers to information that can be used to represent the tilt angle of the finger.

[0051] Based on the above embodiment, the specific implementation of step S23 may be: S231, traversing adjacent pixel points in the outer contour of the hand contour, obtaining vector angles between adjacent pixel points, and determining pixel points whose vector angles are greater than a vector angle threshold as salient points.

[0052] Specifically, the outer contour corresponding to the hand contour can be traversed to obtain multiple adjacent pixel points. On the hand contour, the connecting line between adjacent pixel points can be regarded as a vector. By connecting multiple adjacent pixel points, multiple vectors can be obtained. The angles of these vectors can reflect the degree of curvature and directional changes of the contour. Therefore, for each pair of adjacent pixel points, the vector angle between them can be calculated. A vector angle threshold can be set in advance. When the vector angle is greater than the vector angle threshold, it can be considered that the vector angle between adjacent pixel points is large, indicating that a large directional change has occurred in this part of the hand contour, which may correspond to protruding parts such as fingertips or joints. The corresponding pixel points can be determined as salient points.

[0053] Among them, adjacent pixel points refer to adjacent pixel points in the hand contour, vector angle refers to the angle between two vectors, which can be used to reflect the directional relationship between the vectors, and vector angle threshold refers to the angle threshold that can be used to determine whether the vector angle between adjacent pixel points is too large. If the vector angle is greater than the vector angle threshold, it can be considered that the vector angle between adjacent pixel points is too large. If the vector angle is less than the vector angle threshold, it can be considered that the vector angle between adjacent pixel points is not too large.

[0054] S232, connecting each of the protruding points and the center point of the hand contour to obtain a plurality of connecting lines, and determining the distance between each of the protruding points and the center point according to the length of each of the connecting lines.

[0055] Specifically, by connecting each salient point with the center point of the hand contour, a plurality of connection lines can be obtained, and by obtaining the length corresponding to each connection line, the distance between each salient point and the center point of the hand contour can be determined. The connection line refers to a line segment obtained by connecting the salient point and the center point of the hand contour.

[0056] S233, determine the midline of the two connecting lines with the largest angle, generate a baseline perpendicular to the midline, and determine the pointing direction of the baseline according to the orientation side of the hand contour, the orientation side includes a left side and a right side, and the corresponding pointing directions of the left side and the right side are opposite.

[0057] After obtaining multiple connection lines, the two connection lines with the largest angles can be found. These two connection lines are usually located at the outermost side of the hand contour and correspond to the two farthest fingers, such as the thumb and the little finger. By bisecting the angles corresponding to the two connection lines, the corresponding midline can be obtained. In order to more accurately identify the specific extended finger, a straight line perpendicular to the midline can be generated and determined as the baseline. The baseline can be used to determine the angle information corresponding to each protruding point, and the specific finger extended by the user for motion control can be further determined based on the angle information.

[0058] In actual applications, corresponding camera devices for overhead shooting can be configured on the armrests on both sides of the multifunctional intelligent medical chair. The user can choose the left hand or the right hand for motion control according to his or her needs. For example, if the user chooses the right hand for motion control, the user can place the right hand under the camera device on the right. However, the left hand and the right hand are mirror-symmetrical in shape, that is, the arrangement order and relative position of the fingers of the left and right hands are the same, but their overall direction of the hand contour is opposite. For example, when the back of the hand is placed upward, if the hand is looked down from above, the thumb of the left hand will be on the right and the little finger will be on the left. On the contrary, the thumb of the right hand will be on the left and the little finger will be on the right. Therefore, when we perform finger recognition during motion control, we need to take this directional difference into account.

[0059] Specifically, after obtaining the baseline, the pointing direction of the baseline can be determined according to the orientation side of the hand contour. If the hand selected by the user for motion control is the left hand, the corresponding orientation side can be determined to be the left side, and the pointing direction of the corresponding baseline can be determined to be from left to right. If the hand selected for motion control is the right hand, the corresponding orientation side can be determined to be the right side, and the pointing direction of the corresponding baseline can be determined to be from right to left. Finger recognition for motion control based on the pointing direction can ensure that whether it is the left hand or the right hand, the angle information corresponding to the thumb will be the smallest, that is, the angle formed with the baseline will be the smallest, thereby ensuring consistency in subsequent finger recognition.

[0060] Among them, the center line refers to the straight line that bisects the angle between the two connecting lines with the largest angle, the baseline refers to the straight line that can be used to determine the angle information of each salient point, the azimuth side refers to the directional feature that can be used to represent the corresponding hand of the user for motion control. If the user uses the left hand, the corresponding azimuth side is the left side, if the user uses the right hand, the corresponding azimuth side is the right side, and the pointing direction of the baseline refers to the direction that can be used to determine the angle of the vector corresponding to each connecting line.

[0061] S234, obtaining the direction from the center point to each of the salient points as the pointing direction of each connecting line, and determining the angle information of each of the salient points and the center point of the hand contour according to the vector angle corresponding to the baseline and the pointing direction of each of the connecting lines.

[0062] Specifically, the direction from the center point of the hand contour to each salient point can be determined as the direction corresponding to each connecting line, and the vector angle corresponding to the reference line and the direction of each connecting line can be obtained, and the angle information between each salient point and the center point of the hand contour can be determined according to the vector angle. The direction of the connecting line refers to the direction from the center point of the hand contour to the salient point.

[0063] See also Figure 2 , which is a schematic diagram of determining the pointing direction corresponding to a connecting line provided by an embodiment of the present invention, such as Figure 2 As shown in , in the element recognition image, the direction from the center point of the hand contour to the salient point 1 can be determined as the pointing direction corresponding to the connecting line 1, and the direction from the center point of the hand contour to the salient point 2 can be determined as the pointing direction corresponding to the connecting line 2. There is a corresponding vector angle between the baseline and the pointing direction corresponding to the connecting line 1, and the angle information between the salient point 1 and the center point of the hand contour can be determined based on the vector angle. There is also a corresponding vector angle between the baseline and the pointing direction corresponding to the connecting line 2, and the angle information between the salient point 2 and the center point of the hand contour can be determined based on the vector angle.

[0064] S24, determining the protruding point with the largest distance as the target point, traversing multiple preset angle intervals based on the angle information of the target point, determining the preset angle interval where the angle information is located as the target angle interval, and determining the preset hand element corresponding to the target angle interval as the control element.

[0065] Specifically, the salient point with the largest distance can be determined as the target point, for example, it may be the point corresponding to the fingertip of an extended finger, and the angle information corresponding to the target point can be obtained. According to the angle information corresponding to the target point, multiple pre-configured preset angle intervals are traversed, and each preset angle interval corresponds to a preset hand element. For example, when the preset angle interval is 0°-30°, the corresponding preset hand element is the thumb, and when the preset angle interval is 30°-60°, the corresponding preset hand element is the index finger. The preset angle interval where the angle information is located is determined as the target angle interval, and the preset hand element corresponding to the target angle interval can be determined as the control element.

[0066] Among them, the target point refers to the protruding point with the largest distance, the preset angle interval refers to multiple pre-configured angle intervals, the target angle interval refers to the preset angle interval where the angle information is located, and the preset hand element corresponding to the target angle interval refers to the pre-configured hand element corresponding to the angle interval.

[0067] Through the above implementation, the user can more intuitively understand how to correctly place the hands to control the wheelchair.

[0068] S3, a direction adjustment unit, used to capture a posture image, identify a guiding direction corresponding to the control element in the posture image, and adjust the movement direction of the wheelchair body to the guiding direction.

[0069] Among them, the direction adjustment unit refers to the unit responsible for adjusting the direction of the wheelchair during movement, the posture image refers to the image obtained by capturing the user's hand posture, the guiding direction refers to the direction of the specific finger selected by the user for motion control in the posture image, and the motion direction refers to the direction of movement of the wheelchair during movement.

[0070] After determining the control element, the controller can control the direction adjustment unit to determine the corresponding movement direction according to the corresponding information of the specific finger selected by the user for motion control. Specifically, in order to obtain the user's current hand posture information for subsequent direction identification, the direction adjustment unit can capture the user's hand posture to obtain the corresponding posture image. After obtaining the posture image, the direction adjustment unit will use image recognition technology to identify the direction of the control element in the image. This direction is the direction in which the user wants the wheelchair to move, that is, the guiding direction. According to the identified guiding direction, the direction adjustment unit will adjust the movement direction of the wheelchair body to make it consistent with the direction specified by the user, so that the user can easily control the movement direction of the wheelchair by changing his hand posture.

[0071] Based on the above embodiment, the specific implementation of step S3 may be: S31, based on the voice interaction unit broadcasting the calibration prompt information, controlling the direction adjustment unit to shoot the calibration image, and extracting the hand contour in the calibration image.

[0072] In actual applications, the finger extended by the user may not be consistent with the selected control element, and the guidance direction may not be accurately determined at this time. In order to determine the guidance direction more accurately, when adjusting the direction, the controller can control the direction adjustment unit to calibrate the user's hand posture accordingly. Specifically, the controller can first control the voice interaction unit to broadcast the calibration prompt information to remind the user that the hand posture calibration is currently required, and then control the direction adjustment unit to capture the user's hand posture to obtain the corresponding calibration image, from which the user's hand contour can be extracted. Among them, the voice interaction unit refers to a unit that can interact with the user by voice, the calibration prompt information refers to the information that prompts the user to calibrate the hand posture, and the calibration image refers to an image that can calibrate the user's hand posture.

[0073] S32, obtaining a plurality of salient points in the hand contour, and determining a salient point with the largest distance as a calibration point according to distance and angle information between each of the salient points and the center point of the hand contour.

[0074] Specifically, multiple salient points corresponding to the hand contour in the calibration image can be obtained, and by connecting the multiple salient points corresponding to the hand contour in the calibration image with the center point of the hand contour, the distance and angle information between each salient point in the calibration image and the center point of the hand contour in the calibration image can be obtained, and by comparing the distance and angle information of different salient points in the calibration image, the salient point with the largest distance can be determined as the calibration point. The hand contour refers to the hand contour in the calibration image, and the calibration point refers to the salient point that can be used for hand posture calibration.

[0075] S33, determining that the preset hand element corresponding to the angle information of the calibration point is the calibration element, and when the calibration element is consistent with the control element, responding to the calibration success information, and broadcasting the selected prompt information based on the voice interaction unit.

[0076] Specifically, in order to determine whether the finger currently extended by the user is consistent with the finger previously selected by the user, the preset hand element corresponding to the calibration point can be determined according to the angle information of the calibration point. For example, when the angle information of the calibration point is 45°, it can be determined that the preset hand element corresponding to the angle information is the index finger, and the index finger can be determined as the corresponding calibration element. By comparing the calibration element with the control element, it can be determined whether it is consistent with the previously selected finger. When the control element is also the index finger, it can be considered that the calibration element is consistent with the control element. At this time, the calibration can be considered successful, and the calibration success information can be responded to. The voice interaction unit can broadcast the selection prompt information, prompting the direction adjustment unit to select the direction.

[0077] Among them, the calibration element refers to the preset hand element corresponding to the angle information of the calibration point, the calibration success information refers to the information that can be used to indicate the success of the calibration, and the selection prompt information refers to the information that can be used to prompt the direction selection.

[0078] S34, controlling the direction adjustment unit to shoot a posture image, extracting a hand contour in the posture image and a finger contour in the hand contour, determining the pointing direction of the finger contour as a guiding direction, and adjusting the movement direction of the wheelchair body to the guiding direction.

[0079] Specifically, after the voice broadcast unit has finished broadcasting the selected prompt information, the controller can control the direction adjustment unit to capture the user's current hand posture to obtain a corresponding posture image. In order to improve the accuracy of direction determination, the hand contour corresponding to the user's hand and the finger contour corresponding to the hand contour in the posture image can be extracted from the posture image. The corresponding pointing direction can be determined based on the finger contour, and the pointing direction of the finger contour can be determined as the guiding direction, and the movement direction of the wheelchair body can be adjusted to the guiding direction.

[0080] In actual applications, the intelligent assisted rehabilitation sports multifunctional wheelchair is not only equipped with a device for overhead shooting, but also with a display screen that can interact with the user, and the display screen can be associated with the shooting device. The screen of the display screen can display the user's finger captured and the current corresponding front direction of the wheelchair body, that is, the positive indication direction, and the positive indication direction can be displayed by an indication arrow. By observing the real-time picture on the display screen, the user can clearly see the direction of his or her finger and obtain the current forward direction of the wheelchair at the same time. The user can adjust the direction of the finger in real time in combination with the positive indication direction, thereby achieving precise control of the movement direction of the wheelchair body.

[0081] The finger contour refers to the contour corresponding to the finger extended by the user for motion control in the gesture image, and the pointing direction of the finger contour refers to the direction in which the finger contour in the gesture image points.

[0082] In some embodiments, the step S34 of “extracting the hand contour in the gesture image and the finger contour in the hand contour, and determining the pointing direction of the finger contour as the guiding direction” includes the following steps: S341, when the number of contours of the finger contour is greater than the reference number, an error prompt message is broadcast according to the voice interaction unit.

[0083] Specifically, when the user extends a finger, he may extend multiple fingers, which may cause misjudgment when determining the direction. In order to improve the accuracy of direction determination, it can be judged whether the user extends multiple fingers according to the number of finger contours. When the number of extracted finger contours is greater than 1, it can be considered that the user extends multiple fingers. In order to ensure the uniqueness of direction determination, the controller can control the voice interaction unit to report error prompt information to guide the user to make a correct hand posture. For example, it can be "The current hand posture is wrong, please adjust the hand posture". Among them, the number of contours refers to the number of finger contours, the base number is 1, and the error prompt information refers to the information that can prompt the user that the hand posture is wrong.

[0084] S342: When the number of contours of the finger contour is equal to the reference number, a pixel point with the largest curvature in the finger contour is obtained as a fingertip point.

[0085] Specifically, when the number of contours of the finger contour is 1, the pixel point with the largest curvature in the finger contour can be obtained, and the pixel point may be the inflection point of the finger contour, and the pixel point can be determined as the fingertip point. The curvature can be used to indicate the speed of the change of the tangent direction of the pixel point on the finger contour.

[0086] S343, determining the palm back region of the hand contour in the gesture image, and obtaining a plurality of pixel points where the palm back region and the finger contour intersect as finger root points.

[0087] Specifically, the area corresponding to the palm or back of the hand contour, i.e., the palm back area, can be identified from the posture image, and multiple intersecting pixel points can be obtained between the palm back area and the finger contour, and the corresponding multiple pixel points can be determined as the finger root points. Among them, the palm back area refers to the area corresponding to the palm or back of the hand contour, and the finger root point refers to the pixel point corresponding to the root of the finger.

[0088] When obtaining the area corresponding to the palm or back of the hand contour, the geometric features of the hand contour, such as the concavity and convexity and curvature of the contour, can be analyzed, and feature points in the hand contour, such as fingertip points, knuckle points, etc., can be extracted. According to the geometric features and feature points of the hand contour, the hand contour can be divided into a back of hand area and a finger area, or divided into a palm area and a finger area. Generally, the back of the palm area usually presents a relatively smooth contour, while the finger area contains multiple protruding knuckles and fingertips. According to the results of the hand contour and area division, the area contour corresponding to the palm can be extracted through an image segmentation algorithm to obtain the corresponding palm area, or the area contour corresponding to the back of the hand can be extracted to obtain the corresponding back of the hand area.

[0089] S344, obtaining the center point of multiple finger base points, and determining the direction from the center point to the finger tip point as the guiding direction.

[0090] Specifically, the center point corresponding to the multiple finger-tip points can be obtained, and the direction from the center point to the fingertip point is determined as the guiding direction, see Figure 3 , which is a schematic diagram of determining a guiding direction provided by an embodiment of the present invention, such as Figure 3 As shown in , in the posture image, multiple finger base points can be obtained based on multiple pixel points where the palm back area and the finger contour intersect, and the direction from the center point corresponding to the multiple finger base points to the fingertip point can be determined as the guiding direction.

[0091] Through the above implementation, the user can flexibly control the movement direction of the wheelchair through simple gestures, and the accuracy of direction control can be improved.

[0092] S4, a speed adjustment unit, is used to shoot a dynamic image set, identify the adjustment speed corresponding to the control element in the dynamic image set, and adjust the movement speed of the wheelchair body to the adjustment speed.

[0093] When determining the speed of the wheelchair during movement, this solution is determined based on the finger length selected by the user for motion control. For example, the longer the finger length, the greater the corresponding speed of the wheelchair may be, and the shorter the finger length, the smaller the corresponding speed of the wheelchair may be. In order to accurately identify changes in finger length and achieve precise adjustment of speed, the controller can control the speed adjustment unit to continuously shoot the user's hand posture to form a dynamic image set. The dynamic image set can provide a series of continuous finger images, which record the length of the fingers at different time points.

[0094] After acquiring the dynamic image set, the speed adjustment unit can identify the control element selected by the user for adjusting the speed from the dynamic image set, and can determine the corresponding adjustment speed according to the current length of the control element, and adjust the movement speed of the wheelchair body accordingly according to the adjustment speed.

[0095] Among them, the speed adjustment unit refers to the unit responsible for adjusting the movement speed of the wheelchair body, the dynamic image set refers to a set of multiple images obtained by the speed adjustment unit through continuous shooting of the user's fingers, the adjustment speed refers to the speed corresponding to the length of the control element, and the movement speed refers to the speed of the wheelchair body during the movement.

[0096] Based on the above embodiment, the specific implementation of step S4 may be: S41, acquiring voice interaction data of the voice interaction unit, converting the voice interaction data into text data, and when there is a speed keyword in the text data, sending a hand placement reference map to the user terminal.

[0097] In actual applications, the intelligent-assisted rehabilitation sports multifunctional wheelchair will move at a pre-configured speed at the beginning. When the user needs to adjust the current speed, the user can interact with the voice interaction unit. For example, the user can say "adjust the speed", and what the user says is the voice interaction data. After the voice interaction unit obtains the user's voice interaction data, it can use voice recognition technology to convert the voice interaction data into text data, and the voice interaction unit can determine whether the converted text data contains specific speed keywords, such as "adjust speed", "accelerate", "slow down", etc. When these keywords are recognized, the controller will think that the user wants to adjust the speed. At this time, the corresponding hand placement reference diagram can be sent to the user end. The hand placement reference diagram is to guide the user on how to place his hands correctly, so that the speed the user wants to adjust can be further determined by the finger length later.

[0098] Among them, voice interaction data refers to the data when the user conducts voice interaction with the voice interaction unit, text data refers to the text data corresponding to the voice interaction data, speed keywords refer to keywords related to speed, and hand reference reference refers to the reference when the fingers are placed straight.

[0099] S42, controlling the speed adjustment unit to shoot a first dynamic image, extracting a first finger contour in the first dynamic image, and connecting the fingertip point of the first finger contour and the center point of multiple finger base points to obtain a grade division line.

[0100] Specifically, after the hand placement reference reference is sent to the user, the user can refer to the finger placement posture in the hand placement reference reference to place the fingers for speed control, and then the controller can control the speed adjustment unit to capture the image of the user's placed fingers to obtain the corresponding first dynamic image. Through the contour extraction technology, the contour corresponding to the user's finger for motion control, that is, the first finger contour, can be extracted from the first dynamic image.

[0101] When controlling the speed of the wheelchair, this solution adjusts the speed of the wheelchair in combination with the length of the fingers performing motion control. The speeds corresponding to different finger lengths are different. The user can adjust the speed of the wheelchair by adjusting the length of his or her fingers. In order to accurately determine the speeds corresponding to different finger lengths, a grade line that can display the speed grades corresponding to different lengths can be generated on the first finger contour, that is, a grade dividing line. The grade dividing line can be used to accurately measure the finger length and determine the corresponding speed grade. Subsequently, the speed grade can be further determined in combination with the speed grade. Specifically, the corresponding grade dividing line can be obtained by connecting the fingertip point of the first finger contour and the center point of multiple finger root points. Subsequently, the grade dividing line can be divided to obtain multiple speed grades.

[0102] Among them, the first dynamic image refers to an image that can be used to determine the specific finger performing motion control, the first finger contour refers to the contour corresponding to the finger performing motion control, and the level division line refers to a line segment that can be used to determine the speed level corresponding to the finger length.

[0103] S43, determining multiple level division points on the level division line and the division level of each level division point according to the preset level number, determining multiple control speeds corresponding to the user terminal, matching the division levels and the control speeds one by one from small to large, generating a speed display diagram and sending it to the user terminal.

[0104] In actual application, the user can change the speed of the wheelchair by adjusting the length of his or her fingers. In order to enable the user to intuitively understand the correspondence between different finger lengths and speed levels, a speed display graph corresponding to the user's finger length can be customized and generated for display to the user. The speed display graph can clearly show the different finger lengths corresponding to the speed levels and the specific speeds corresponding to each speed level. By observing the speed display graph, the user can intuitively understand the relationship between finger length and speed, and thus adjust the finger length according to their needs to achieve precise control of the wheelchair speed.

[0105] Specifically, after obtaining the level division line, the number of multiple different speed levels can be pre-configured, that is, the preset number of levels. According to the preset number of levels, multiple level division points can be determined on the level division line. For example, if the preset number of levels is three, then there will be three division points on the level division line. Each level division point has a corresponding division level, and the finger length corresponding to each division level can be determined according to the level division point. The longer the finger length, the larger the corresponding division level may be.

[0106] After obtaining multiple division levels, multiple control speeds corresponding to the user end can be determined according to the number of division levels, and the multiple division levels and the multiple control speeds are matched one by one in ascending order. The lower the division level, the smaller the corresponding control speed. Therefore, the control speed corresponding to the division level of each level division point can be determined, and the corresponding speed display graph can be obtained and sent to the user end for reference. In the speed display graph, the finger length and speed corresponding to each division level are personalized according to the actual finger length of the user. Therefore, although different users have the same number of division levels, the finger lengths and speeds corresponding to the division levels are different.

[0107] Among them, the number of preset levels refers to the number of pre-configured different speed levels, the level division points refer to the points corresponding to different speed levels determined on the level division lines, the division levels refer to the speed levels corresponding to different finger lengths, and the control speed refers to the multiple speeds for controlling the wheelchair movement corresponding to the user end.

[0108] In some embodiments, the specific implementation of step S43 may be: S431, equally dividing the level division line based on the preset number of levels to obtain a plurality of level division points, and determining the length of a line segment from each of the level division points to the center point of a plurality of root points as a reference length.

[0109] Specifically, according to the preset number of levels, the level division line is equally divided to obtain the level division points. For example, when the preset number of levels is 3, 3 level division points can be obtained by equally dividing the level division line. The intervals between adjacent level division points are the same. Each level division point is connected with the center point of multiple finger root points corresponding to the first finger contour to obtain the line segment corresponding to each level division point. The line segment length corresponding to each line segment is the reference length corresponding to each level division point. The reference length refers to the line segment length between each level division point and the center point of multiple finger root points.

[0110] S432, arranging the level division points from small to large according to the reference length to obtain a point sequence, arranging the preset division levels from small to large to obtain a level sequence, and matching the level division points and division levels with the same arrangement position in the point sequence and the level sequence one by one.

[0111] Specifically, by arranging the various level division points in the order of reference length from small to large, a corresponding point sequence can be obtained. In the point sequence, the smaller the reference length of the level division point, the higher the arrangement position, and the various division levels can be arranged in the order of small to large to obtain a corresponding level sequence. In the level sequence, each division top has a corresponding arrangement position. If the arrangement position of the level division point in the point sequence is the same as the arrangement position of the division level in the level sequence, then the corresponding level division line can be matched with the division level. For example, if the arrangement position of the level division point 1 in the point sequence is the first, and the arrangement position of the division level 1 in the level sequence is also the first, then the level division point 1 can be matched with the division level 1, that is, the division level corresponding to the level division point 1 is the division level 1. The point sequence refers to a sequence obtained by arranging the points of each level division, and the level sequence refers to a sequence obtained by arranging the levels of division.

[0112] S433, calling a preset maximum speed threshold, offsetting the maximum speed threshold according to the height adjustment parameter to obtain a maximum adjustment threshold, and dividing the maximum adjustment threshold based on the preset number of levels to obtain a plurality of control speeds.

[0113] Specifically, the corresponding maximum speed threshold pre-configured for the wheelchair can be retrieved. In actual applications, different users may have different height adjustment parameters for wheelchairs. Since the height of the wheelchair may affect the center of gravity of the user, the higher the height, the higher the center of gravity moves, and the worse the stability of the wheelchair during driving. If the speed is too fast, it may roll over. For the safety of the user, the preset maximum speed threshold can be offset in combination with the height adjustment parameter configured by the user for the wheelchair to obtain the maximum adjustment threshold corresponding to the height adjustment parameter, and the maximum adjustment threshold can be divided according to the preset number of levels to obtain multiple control speeds. Among them, the maximum speed threshold refers to the speed pre-configured for the wheelchair, and the maximum adjustment speed refers to the speed after the maximum speed threshold is offset in combination with the height adjustment parameter.

[0114] In some embodiments, the step S433 of "obtaining a maximum adjustment threshold value by offsetting the maximum speed threshold value according to the height adjustment parameter, and obtaining a plurality of control speeds by dividing the maximum adjustment threshold value based on the preset number of levels" includes the following steps: S4331, obtaining an overall height value corresponding to the height adjustment parameter, and determining a standard height value corresponding to the maximum speed threshold.

[0115] Specifically, the height values ​​corresponding to each wheelchair component of the user can be obtained according to the height adjustment parameter, and the average of multiple height values ​​can be calculated to obtain the overall height value corresponding to the height adjustment parameter, and the maximum speed threshold and the corresponding height value, that is, the standard height value, can be obtained. Among them, the overall height value refers to the overall height corresponding to the intelligent assisted rehabilitation multifunctional wheelchair, for example, it can be the average value of the height values ​​of multiple wheelchair components, and the standard height value refers to the height value corresponding to the maximum speed threshold.

[0116] S4332: Obtain a speed offset coefficient according to the ratio of the standard height value to the overall height value, and obtain a maximum adjustment threshold by multiplying the speed offset coefficient by the maximum adjustment threshold.

[0117] Specifically, by calculating the ratio of the standard height value to the overall height value, a speed offset coefficient for offsetting the maximum speed threshold can be obtained, and by multiplying the speed offset coefficient by the maximum adjustment threshold, the corresponding maximum adjustment threshold can be obtained. The speed offset value refers to a value that can offset the maximum speed threshold.

[0118] S4333, obtaining a unit speed based on the ratio of the maximum adjustment threshold and the preset number of levels, determining the number of levels corresponding to each of the divided levels, and multiplying the unit speed by each of the number of levels to obtain a plurality of control speeds.

[0119] Specifically, by calculating the ratio of the maximum adjustment threshold and the preset number of levels, the unit speed can be obtained, and the level number corresponding to each division level is determined. The unit speed is multiplied by the level number corresponding to the division level to obtain the control speed corresponding to each division level. For example, when the unit speed is 1.5m / s, if there are three division levels, namely division level 1, division level 2, and division level 3, and the corresponding level numbers are 1, 2, and 3 respectively, then the control speed corresponding to division level 1 can be 1.5m / s, the control speed corresponding to division level 2 can be 3m / s, and the control speed corresponding to division level 3 can be 4.5m / s.

[0120] The unit speed refers to a speed unit calculated by the ratio of the maximum adjustment threshold to the number of preset levels, and the number of levels refers to the number corresponding to the divided levels.

[0121] S434, generating a level slot corresponding to each of the level division points in the first dynamic image, filling each of the control speeds into the corresponding level slot to obtain a speed display diagram, the larger the division level of the level slot, the greater the control speed.

[0122] See also Figure 3 , is a schematic diagram of a speed display diagram provided by an embodiment of the present invention, such as Figure 3 As shown in , multiple level division points can be determined on the level division line in the first dynamic image. After obtaining the control speed corresponding to each level, a level slot corresponding to each level division point can be generated in the first dynamic image. The control speed corresponding to each level division point is filled into the corresponding level slot, and the corresponding speed display graph can be obtained. It can be seen from the figure that the larger the division level, the longer the corresponding finger length, and the greater the corresponding control speed.

[0123] S44, controlling the speed adjustment unit to shoot a second dynamic image, extracting a second finger contour in the second dynamic image, the dynamic image set including the first dynamic image and the second dynamic image.

[0124] Specifically, when the length of the user's finger changes, for example, when the user bends the finger, the speed adjustment unit can be controlled to capture the current finger posture of the user to obtain a corresponding second dynamic image, and the contour extraction technology can be used to extract the contour corresponding to the user's bent finger from the second dynamic image, that is, the second finger contour. The second dynamic image refers to the image corresponding to the change in the length of the user's finger, and the second finger contour refers to the finger contour in the second dynamic image.

[0125] S45, determining the reference length corresponding to each of the division levels, obtaining the reference length closest to the finger length of the second finger contour as the target length, determining the control speed corresponding to the target length as the adjustment speed, and adjusting the movement speed of the wheelchair body to the adjustment speed.

[0126] Specifically, each division level has a corresponding reference length, and each reference length has a corresponding control speed. The finger length corresponding to the second finger contour can be obtained, and the reference length closest to the finger length corresponding to the second finger contour is determined as the target length, and the control speed corresponding to the target length can be determined as the adjustment speed. After the adjustment speed is obtained, the movement speed of the wheelchair body can be adjusted to the adjustment speed. Among them, the reference length refers to the finger length corresponding to the division level, the target length refers to the reference length closest to the finger length corresponding to the second finger contour, and the control speed refers to the movement speed corresponding to the target length.

[0127] In some embodiments, the step S45 of "obtaining a reference length closest to the finger length of the second finger contour as a target length, and determining a control speed corresponding to the target length as an adjustment speed" includes the following steps: S451, fitting the second finger contour to obtain a fitting line, and determining the length between the endpoints of both ends of the fitting line as the finger length.

[0128] In some embodiments, when obtaining the finger length corresponding to the second finger contour, the second finger contour can be fitted into a smooth straight line through an appropriate fitting algorithm such as the least squares method, and the straight line is determined as the fitting line. The two endpoints farthest from each other are found on the fitting line, and the distance between the two endpoints is calculated, which is the finger length.

[0129] In some other embodiments, when obtaining the finger length, the second hand contour corresponding to the user can be extracted from the second dynamic image, and the corresponding second finger contour can be obtained from the second hand contour. After obtaining the second hand contour, multiple protruding points on the second hand contour can be obtained, and multiple finger root points corresponding to the second finger contour can be obtained, and the distance between each protruding point and the center point of the multiple finger root points can be calculated, and the protruding point with the largest distance is determined as the target point, the target point and the center point of the multiple finger root points are connected, and the length of the line segment between the target point and the multiple finger root points is obtained, and the length of the line segment is determined as the finger length.

[0130] S452, determining the length difference between each reference length and the finger length, obtaining the reference length with the smallest length difference as the target length, and determining the control speed corresponding to the target length as the adjustment speed.

[0131] Specifically, the difference between each reference length and the finger length, i.e., the length difference, is calculated, the reference length with the smallest difference is determined as the target length, the control length corresponding to the target length is obtained, and the control length is determined as the adjustment length. The length difference refers to the difference between the reference length and the finger length.

[0132] Through the above implementation, accurate adjustment of the movement speed of the wheelchair can be achieved.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent assisted rehabilitation multifunctional wheelchair, characterized in that: The invention comprises a wheelchair body, a control motor for controlling the movement of the wheelchair body, and a controller connected to the control motor, wherein the controller comprises: A height adjustment unit, used for obtaining a height adjustment parameter of a wheelchair component in the wheelchair body based on a wheelchair model by a user end, and adjusting the height of the wheelchair component according to the height adjustment parameter; An element selection unit, used to determine a gesture recognition mode selected by a user terminal, wherein the gesture recognition mode includes a side-laying mode and a horizontal-laying mode, and to determine a control element selected by the user terminal according to the gesture recognition mode; a direction adjustment unit, used for capturing a posture image, identifying a guiding direction corresponding to the control element in the posture image, and adjusting the movement direction of the wheelchair body to the guiding direction; The speed adjustment unit is used to shoot a dynamic image set, identify the adjustment speed corresponding to the control element in the dynamic image set, and adjust the movement speed of the wheelchair body to the adjustment speed.

2. The intelligent assisted rehabilitation multifunctional wheelchair according to claim 1, characterized in that: A height adjustment unit is used to obtain a height adjustment parameter of a wheelchair component in the wheelchair body based on a wheelchair model by a user end, and to adjust the height of the wheelchair component according to the height adjustment parameter, including: The height adjustment unit obtains a component module selected by the user terminal based on the wheelchair model, and determines a height value selected by the user terminal based on a height sliding axis corresponding to the component module; The height value is bound to the component module to obtain a height adjustment parameter, a wheelchair component corresponding to the component module is determined, and the height of the wheelchair component is adjusted according to the height adjustment parameter.

3. The intelligent assisted rehabilitation multifunctional wheelchair according to claim 1, characterized in that: An element selection unit is used to determine a gesture recognition mode selected by a user terminal, wherein the gesture recognition mode includes a side-laying mode and a horizontal-laying mode, and to determine a control element selected by the user terminal according to the gesture recognition mode, including: Determine a gesture reference image corresponding to the gesture recognition mode and send it to the user terminal, and when the gesture recognition mode is the side placement mode, determine a preset hand element corresponding to the side placement mode as a control element; When the posture recognition mode is the flat mode, photographing an element recognition image, and extracting a hand contour in the element recognition image; Acquire multiple salient points in the hand contour, and acquire distance and angle information between each salient point and a center point of the hand contour; Determine the protruding point with the largest distance as the target point, traverse multiple preset angle intervals based on the angle information of the target point, determine the preset angle interval where the angle information is located as the target angle interval, and determine the preset hand element corresponding to the target angle interval as the control element.

4. The intelligent assisted rehabilitation multifunctional wheelchair according to claim 3, characterized in that: Acquiring a plurality of salient points in the hand contour, and acquiring distance and angle information between each of the salient points and a center point of the hand contour, including: Traversing adjacent pixel points in the outer contour of the hand contour, obtaining vector angles between adjacent pixel points, and determining pixel points whose vector angles are greater than a vector angle threshold as salient points; Connecting each of the protruding points and the center point of the hand contour to obtain a plurality of connecting lines, and determining the distance between each of the protruding points and the center point according to the length of each of the connecting lines; Determine the midline of the two connecting lines with the largest angle, generate a reference line perpendicular to the midline, and determine the pointing direction of the reference line according to the azimuth side of the hand contour, the azimuth side includes a left side and a right side, and the left side and the right side correspond to opposite pointing directions; The direction from the center point to each of the salient points is obtained as the pointing direction of each connecting line, and the angle information of each of the salient points and the center point of the hand contour is determined according to the vector angle corresponding to the baseline and the pointing direction of each of the connecting lines.

5. The intelligent assisted rehabilitation multifunctional wheelchair according to claim 1, characterized in that: A direction adjustment unit, used for photographing a posture image, identifying a guiding direction corresponding to the control element in the posture image, and adjusting the movement direction of the wheelchair body to the guiding direction, comprises: Based on the voice interaction unit broadcasting the calibration prompt information, the direction adjustment unit is controlled to shoot the calibration image, and the hand contour in the calibration image is extracted; Acquire multiple salient points in the hand contour, and determine the salient point with the largest distance as the calibration point according to the distance and angle information between each salient point and the center point of the hand contour; Determine a preset hand element corresponding to the angle information of the calibration point as a calibration element, and when the calibration element is consistent with the control element, respond with calibration success information, and broadcast selected prompt information based on a voice interaction unit; The control direction adjustment unit captures a posture image, extracts a hand contour in the posture image and a finger contour in the hand contour, determines the pointing direction of the finger contour as a guiding direction, and adjusts the movement direction of the wheelchair body to the guiding direction.

6. The intelligent assisted rehabilitation multifunctional wheelchair according to claim 5, characterized in that: Extracting a hand contour in the gesture image and a finger contour in the hand contour, and determining a pointing direction of the finger contour as a guiding direction, comprising: When the number of contours of the finger contour is greater than the reference number, an error prompt message is broadcasted according to the voice interaction unit; When the number of contours of the finger contour is equal to the reference number, obtaining a pixel point with the largest curvature in the finger contour as a fingertip point; Determine the palm back region of the hand contour in the gesture image, and obtain a plurality of pixel points where the palm back region and the finger contour intersect as finger root points; The center point of multiple finger base points is obtained, and the direction from the center point to the finger tip point is determined as the guiding direction.

7. The intelligent assisted rehabilitation multifunctional wheelchair according to claim 1, characterized in that: The speed adjustment unit is used to shoot a dynamic image set, identify the adjustment speed corresponding to the control element in the dynamic image set, and adjust the movement speed of the wheelchair body to the adjustment speed, including: Acquire voice interaction data of the voice interaction unit, convert the voice interaction data into text data, and when there is a speed keyword in the text data, send a hand placement reference map to the user terminal; Controlling the speed adjustment unit to shoot a first dynamic image, extracting a first finger contour in the first dynamic image, and connecting a fingertip point of the first finger contour and a center point of a plurality of finger base points to obtain a grade division line; Determine multiple level division points on the level division line and the division level of each level division point according to the preset number of levels, determine multiple control speeds corresponding to the user terminal, match the division levels and the control speeds one by one from small to large, generate a speed display diagram and send it to the user terminal; Controlling the speed adjustment unit to shoot a second dynamic image, extracting a second finger contour in the second dynamic image, the dynamic image set including the first dynamic image and the second dynamic image; Determine a reference length corresponding to each of the division levels, obtain a reference length closest to the finger length of the second finger contour as a target length, determine a control speed corresponding to the target length as an adjustment speed, and adjust the movement speed of the wheelchair body to the adjustment speed.

8. The intelligent assisted rehabilitation multifunctional wheelchair according to claim 7, characterized in that: Determining multiple level division points on the level division line and the division level of each level division point according to the preset level number, determining multiple control speeds corresponding to the user terminal, matching the division levels and the control speeds one by one from small to large, generating a speed display diagram and sending it to the user terminal, including: The level division line is equally divided based on the preset level number to obtain a plurality of level division points, and the length of the line segment from each level division point to the center point of the plurality of root points is determined as a reference length; Arrange the level division points from small to large according to the reference length to obtain a point sequence, arrange the preset division levels from small to large to obtain a level sequence, and match the level division points and division levels with the same arrangement position in the point sequence and the level sequence one by one; Retrieving a preset maximum speed threshold, offsetting the maximum speed threshold according to the height adjustment parameter to obtain a maximum adjustment threshold, and dividing the maximum adjustment threshold based on the preset number of levels to obtain a plurality of control speeds; A level slot corresponding to each level division point is generated in the first dynamic image, and each control speed is filled into the corresponding level slot to obtain a speed display diagram. The larger the division level of the level slot, the greater the control speed.

9. The intelligent assisted rehabilitation multifunctional wheelchair according to claim 8, characterized in that: The maximum speed threshold is offset according to the height adjustment parameter to obtain a maximum adjustment threshold, and the maximum adjustment threshold is divided based on the preset number of levels to obtain a plurality of control speeds, including: Obtaining an overall height value corresponding to the height adjustment parameter, and determining a standard height value corresponding to the maximum speed threshold; A speed offset coefficient is obtained according to a ratio of the standard height value to the overall height value, and a maximum adjustment threshold is obtained by multiplying the speed offset coefficient by the maximum adjustment threshold; A unit speed is obtained based on the ratio of the maximum adjustment threshold and the preset number of levels, the number of levels corresponding to each of the divided levels is determined, and a plurality of control speeds are obtained by multiplying the unit speed and each of the number of levels.

10. The intelligent assisted rehabilitation multifunctional wheelchair according to claim 7, characterized in that: Acquiring a reference length closest to the finger length of the second finger contour as a target length, and determining a control speed corresponding to the target length as an adjustment speed, including: Fitting the second finger contour to obtain a fitting line, and determining the length between the endpoints of both ends of the fitting line as the finger length; Determine the length difference between each reference length and the target length, obtain the reference length with the smallest length difference as the target length, and determine the control speed corresponding to the target length as the adjustment speed.

Images