Wheelchair safety response method and system based on behavior recognition

Abstract

The invention discloses a wheelchair safety response method and system based on behavior recognition, and relates to the technical field of wheelchair safety response, and the method comprises the following steps: S1, recognizing user gesture data; s2, detecting information of a road surface around the wheelchair; s3, extracting and analyzing feature information; s4, executing safety measures; and S5, adjusting the wheelchair in real time. According to the method, the user can use the wheelchair conveniently, the movement track of the wheelchair can be controlled in time according to the psychological reaction of the user in the driving process, response mechanisms of a stopping mechanism, an avoiding mechanism and a centering mechanism are adopted according to obstacle detection information, the user is forcibly required to use the wheelchair according to regulations through a forbidding mechanism, the use safety is improved, and the user experience is improved. Meanwhile, the problems that the wheelchair is single in sensing and poor in road condition sensing effect are solved, and the intelligent effect of the wheelchair is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of wheelchair safety response, and in particular to a wheelchair safety response method and system based on behavior recognition. Background Art

[0002] Traditional wheelchairs require assistance from others during movement to ensure the safety and reliability of the wheelchair when in use. The wheelchair cannot actively ensure the safety of the occupants and lacks intelligence and active safety protection capabilities. The Chinese patent application number 201710509586.8 discloses "an intelligent wheelchair system and control method based on gesture control, wherein the camera video acquisition module acquires gesture images, the gesture detection module receives the gesture images acquired from the camera video acquisition module, detects, analyzes and extracts gesture information from the images, the gesture recognition module receives the gesture information from the gesture detection module and recognizes the gesture information; the gesture information wireless transmission module receives the gesture information recognized by the gesture recognition module and transmits the gesture information back to the controller of the intelligent wheelchair; the controller of the intelligent wheelchair receives the gesture information and controls the intelligent wheelchair according to the gesture information."

[0003] The above document enables people to control wheelchair movements through gestures. However, the existing technology has the problems of single wheelchair perception and poor road condition perception. When using a wheelchair, people with weak arms may not have time to make gestures. When encountering an emergency, they cannot control the wheelchair according to gestures in a timely and accurate manner, which reduces safety. At the same time, for uneven roads and small objects on the road, users may not see them due to negligence, causing the wheelchair to overturn or bump, resulting in a poor riding experience and psychological changes in the user. Therefore, it is also necessary to follow up the user's psychological activities in real time to automatically control the movement of the wheelchair. Summary of the invention

[0004] The purpose of the present invention is to provide a wheelchair safety response method and system based on behavior recognition to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a wheelchair safety response method based on behavior recognition, comprising the following steps:

[0006] S1. Identify user gesture data: The wheelchair is equipped with a PAJ7620 sensor and an embedded DSP processor. The infrared LED inside the PAJ7620 sensor is driven to emit infrared signals. When the sensor array detects an object, the sensor array acquires the original feature data of the detected target, and the acquired data is stored in the register. At the same time, the gesture recognition array recognizes and processes the original data, and finally stores the gesture result in the register. When the embedded DSP processor detects a gesture change, it controls the movement of the wheelchair.

[0007] S2. Detecting the road surface information around the wheelchair: Six laser rangefinder sensors are installed around the wheelchair. The laser rangefinder sensors are used to sense the object information on the left, right and rear of the wheelchair. The sensing range of the laser rangefinder sensor is set to 5 meters. When the laser rangefinder sensor detects an object within the sensing range during the movement of the wheelchair, it sends a signal to the embedded DSP processor. The laser radar senses the object within 120° and 60 meters in front of the wheelchair, obtains the speed of the object and the distance between the object and the wheelchair, and obtains accurate road surface information through the cooperation of the laser radar and the laser rangefinder sensor.

[0008] S3, extract and analyze feature information: During the movement of the wheelchair, ground points and non-ground points are divided from the 3D laser point cloud data image, and then feature extraction is performed to extract edge points and plane points. The laser point cloud at time t and the laser point cloud at time t-1 are then associated to estimate the position of the wheelchair. Finally, a fusion map is constructed. The map is spliced ​​according to each frame of laser point cloud data sent by the radar to obtain a complete global map, thereby changing the direction of the wheelchair.

[0009] S4. Execution of safety measures: The user is forced to use the wheelchair as specified through the disabling mechanism. When the sensor array does not detect the user's gesture operation information and the wheelchair is in an unsafe state, the user is assisted in emergency avoidance of objects through the stop mechanism and the avoidance mechanism, and the autonomous movement of the wheelchair is controlled through the centering mechanism.

[0010] S5. Real-time adjustment of the wheelchair: The wheelchair state is obtained according to the estimated wheelchair posture, and the moving direction and speed of the wheelchair are adjusted according to the user's psychological behavior during driving, the recognized user gesture data and the wheelchair state.

[0011] Preferably, the gesture change in step S1 includes the following steps:

[0012] S101: The hand gesture is forward or backward, and the wheelchair moves forward or backward;

[0013] S102: The hand gesture is left or right, and the wheelchair turns left or right;

[0014] S103: The gesture is down, and the wheelchair stops moving.

[0015] Preferably, step S3 comprises the following steps:

[0016] S301, laser radar will send a frame of data each time it scans. One frame of data represents one frame of laser point cloud, which is expressed as P t ={p1,p2,···,p i ,···,p n}, where p i is the laser point cloud P at time tt A point in the laser point cloud is projected into a two-dimensional image. The pixels of each point in the image form a distance image, which is the distance between the actual object and the laser radar.

[0017] S302. For the ground point, the vertical scanning angle of the laser radar is [-12.5°, +12.5°]. The laser radar is installed horizontally, and the ground point is located on the scanning line [-12.5°, 0].

[0018] S303, for the remaining non-ground points, classify the distance image, mark the points of the same class with unique labels, and treat the class with less than 30 points as noise points;

[0019] S304. Each frame of data saved has a label of each category and row and column indexes and pixel values ​​in the range image.

[0020] Preferably, in step S303, edge features and plane features of the laser point cloud are extracted from the classified range image to obtain the posture transformation relationship between time t and time t-1. First, the range image is divided into multiple sub-images, a point is selected in each sub-image, and 5 points are found on both sides of the point in the vertical direction to construct a point set. Then, the curvature of each row of points in the sub-image is calculated, and the edge points and plane points are extracted according to the curvature.

[0021] Preferably, the step S3 further comprises the following steps:

[0022] S305, estimating the posture information of the wheelchair according to the change between two adjacent frames of data, including feature point association and posture solution;

[0023] S306, feature point association is to associate the laser point cloud at time t with the laser point cloud at time t-1, and to construct a constraint relationship between two adjacent scans based on the extracted edge points and plane points to obtain a posture transformation matrix;

[0024] S307, posture solution matches the feature point cloud according to the posture transformation matrix, and estimates the posture of the laser radar according to the matched feature point cloud, that is, estimates the posture of the wheelchair;

[0025] S308, completing point cloud recognition through convolutional neural network, outputting obstacle information and road boundary information, matching the laser point cloud at time t with the global map feature point cloud at time t-1, and then adding them to the global map.

[0026] Preferably, in step S306, the constraint relationship construction process of two adjacent scans is as follows: a DETA100R 4G sensor and an inertial sensor are installed on the wheelchair, the DETA100R 4G sensor receives the differential positioning signal broadcast by the RTK base station based on the network RTK positioning technology, and uses the inertial sensor to collect information, and uses the matching algorithm to obtain the attitude transformation matrix;

[0027] The specific matching algorithm is:

[0028] x=(q,v x ,v y ,p x ,p y ) T

[0029] u=(a x ,a y ,ω) T

[0030] z=(s,θ,v)

[0031] Among them, x, u, z represent the state vector, control vector and observation vector respectively, q represents the rotation amount of the wheelchair movement, and v x 、p x 、a x Represent the velocity, displacement and acceleration on the x-axis respectively, v y 、p y 、a y They represent the velocity, displacement and acceleration on the y-axis respectively, ω represents the angular velocity of the z-axis. The acceleration and angular velocity are provided by the inertial sensor. s, θ and v are provided by the RTK data, representing the displacement, rotation speed and moving speed respectively.

[0032]

[0033]

[0034]

[0035]

[0036] Among them, x t represents the state vector at time t, u t represents the control vector at time t, z t represents the measurement vector at time t, f(·) represents the measurement value created using the inertial sensor, represents the intermediate variable, P t represents the covariance estimator at time t, A t represents the state transfer matrix, represents the intermediate variable, At use The derivative of is calculated from Indicates A t The transposed matrix, Q t represents the process noise covariance, H t represents the state transfer matrix, H t use The derivative of is calculated from Indicates H t The transposed matrix, R t Represents the measurement error, K t Represents the posture transformation matrix.

[0037] Preferably, in step S4, the disabling mechanism, the stopping mechanism, the avoiding mechanism and the centering mechanism include the following steps:

[0038] S401, disabling mechanism: The wheelchair is equipped with a seat belt and a heart rate bracelet. When the user uses the wheelchair, if the user does not fasten the seat belt or wear the heart rate bracelet, the wheelchair is prohibited from starting and waits for the user to handle it;

[0039] S402, stop mechanism: When the laser radar detects a fixed object in front of the wheelchair and the distance from the wheelchair is less than 2 meters, the wheelchair stops automatically and waits for the user to handle it;

[0040] S403, avoidance mechanism: When the wheelchair is moving, if the laser ranging sensors on the left, right, or rear side detect an object, and the distance between the object and the wheelchair is less than 30 cm, the wheelchair moves away from the object to avoid the object;

[0041] S404, centering mechanism: When objects are detected on both the left and right sides of the wheelchair, the wheelchair moves to the center position according to the distances to the objects on both sides of the wheelchair, so that the distances between the wheelchair and the objects on both sides are equal.

[0042] Preferably, in step S5, the wheelchair state includes posture information and speed information;

[0043] The posture information includes the pitch angle, roll angle and yaw angle of the wheelchair. After analysis, it outputs whether the wheelchair has tipped over or is in a dangerous state. If so, the wheelchair is automatically controlled, otherwise, it remains as it is.

[0044] Speed ​​information is used for wheelchair speed limit and speed information prompts.

[0045] The wheelchair safety response system based on behavior recognition includes:

[0046] A gesture recognition unit, which acquires the original feature data of the detected target by driving the sensor array inside the PAJ7620 sensor. The acquired data is stored in a register. Meanwhile, the gesture recognition array recognizes and processes the original data, and finally stores the gesture result in the register. When the embedded DSP processor detects the gesture change, it controls the movement of the wheelchair.

[0047] An information detection unit, wherein the information detection unit obtains accurate road surface information through the cooperation of a laser radar and a laser ranging sensor;

[0048] A feature analysis unit, which extracts features from the three-dimensional laser point cloud data image, then associates the laser point cloud at time t with the laser point cloud at time t-1, estimates the position of the wheelchair, and finally performs fusion mapping. The map is spliced ​​according to each frame of laser point cloud data sent by the radar to obtain a complete global map, thereby changing the direction of the wheelchair;

[0049] A safety execution unit, which forces the user to use the wheelchair as specified through a disabling mechanism, assists the user in completing emergency avoidance of objects through a stop mechanism and an avoidance mechanism, and controls the autonomous movement of the wheelchair through a centering mechanism;

[0050] A wheelchair response unit adjusts the moving direction and moving speed of the wheelchair according to the user's psychological behavior during driving, the recognized user gesture data and the wheelchair status.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] 1. In terms of user perception, the present invention facilitates the use of the user. The movement trajectory of the wheelchair can be controlled in time according to the psychological reaction of the user during driving. A human-computer interaction module based on gesture recognition is installed on the wheelchair to detect changes in user gestures, thereby controlling the movement or stop of the wheelchair. At the same time, in terms of wheelchair state perception, network RTK differential positioning technology is used to correct satellite positioning data, achieve centimeter-level positioning of the wheelchair, and collect the wheelchair posture angle in real time. Laser radar is used to sense the movement speed of the wheelchair. The active safety protection strategy of the wheelchair is designed through the cooperation of laser radar and laser ranging sensor. During the movement process, according to the obstacle detection information, the response mechanism of the stopping mechanism, the avoidance mechanism and the centering mechanism is adopted, and the user is forced to use the wheelchair according to the regulations through the disabling mechanism, thereby improving the safety of use;

[0053] 2. In terms of road surface perception, the present invention divides the scanning angle into ground points and non-ground points for scanning to obtain frame data. For ground points, slopes and planes are identified by extracting edge points and plane points. For non-ground points, distance images are classified and identified, so as to accurately detect obstacles. In addition, point cloud recognition is completed through a convolutional neural network, and obstacle information and road boundary information are output. The map is spliced ​​according to each frame of laser point cloud data sent by the radar to obtain a complete global map, thereby providing support for changes in the direction of the wheelchair, solving the problems of single wheelchair perception and poor road condition perception, and realizing the intelligent effect of the wheelchair. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A schematic diagram of the overall method flow provided by an embodiment of the present invention;

[0055] Figure 2 A schematic diagram of the overall system flow provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] See also Figure 1 to Figure 2 The present invention provides a technical solution: a wheelchair safety response method based on behavior recognition, comprising the following steps:

[0058] S1. Identify user gesture data: The wheelchair is equipped with a PAJ7620 sensor and an embedded DSP processor. The infrared LED inside the PAJ7620 sensor is driven to emit infrared signals. When the sensor array detects an object, the sensor array acquires the original feature data of the detected target, and the acquired data is stored in the register. At the same time, the gesture recognition array recognizes and processes the original data, and finally stores the gesture result in the register. When the embedded DSP processor detects a gesture change, it controls the movement of the wheelchair.

[0059] S2. Detecting the road surface information around the wheelchair: Six laser rangefinder sensors are installed around the wheelchair. The laser rangefinder sensors are used to sense the object information on the left, right and rear of the wheelchair. The sensing range of the laser rangefinder sensor is set to 5 meters. When the laser rangefinder sensor detects an object within the sensing range during the movement of the wheelchair, it sends a signal to the embedded DSP processor. The laser radar senses the object within 120° and 60 meters in front of the wheelchair, obtains the speed of the object and the distance between the object and the wheelchair, and obtains accurate road surface information through the cooperation of the laser radar and the laser rangefinder sensor.

[0060] S3, extract and analyze feature information: During the movement of the wheelchair, ground points and non-ground points are divided from the 3D laser point cloud data image, and then feature extraction is performed to extract edge points and plane points. The laser point cloud at time t and the laser point cloud at time t-1 are then associated to estimate the position of the wheelchair. Finally, a fusion map is constructed. The map is spliced ​​according to each frame of laser point cloud data sent by the radar to obtain a complete global map, thereby changing the direction of the wheelchair.

[0061] S4. Execution of safety measures: The user is forced to use the wheelchair as specified through the disabling mechanism. When the sensor array does not detect the user's gesture operation information and the wheelchair is in an unsafe state, the user is assisted in emergency avoidance of objects through the stop mechanism and the avoidance mechanism, and the autonomous movement of the wheelchair is controlled through the centering mechanism.

[0062] S5. Real-time adjustment of the wheelchair: The wheelchair state is obtained according to the estimated wheelchair posture, and the moving direction and speed of the wheelchair are adjusted according to the user's psychological behavior during driving, the recognized user gesture data and the wheelchair state.

[0063] The gesture change in step S1 includes the following steps:

[0064] S101: The hand gesture is forward or backward, and the wheelchair moves forward or backward;

[0065] S102: The hand gesture is left or right, and the wheelchair turns left or right;

[0066] S103, the gesture is down, and the wheelchair stops moving;

[0067] Step S3 includes the following steps:

[0068] S301, laser radar will send a frame of data each time it scans. One frame of data represents one frame of laser point cloud, which is expressed as P t ={p1,p2,···,p i ,···,p n}, where p i is the laser point cloud P at time t tA point in the laser point cloud is projected into a two-dimensional image. The pixels of each point in the image form a distance image, which is the distance between the actual object and the laser radar.

[0069] S302. For the ground point, the vertical scanning angle of the laser radar is [-12.5°, +12.5°]. The laser radar is installed horizontally, and the ground point is located on the scanning line [-12.5°, 0].

[0070] S303, for the remaining non-ground points, classify the range image, mark the points of the same class with unique labels, and treat the class with less than 30 points as noise points to reduce the interference caused by the non-repetition of tiny objects between adjacent frames on the inter-frame matching;

[0071] S304, each frame of data saved has a label of each category and a row and column index and a pixel value in the range image;

[0072] In step S303, edge features and plane features of the laser point cloud are extracted from the classified range image, so as to obtain the posture transformation relationship between time t and time t-1. First, the range image is divided into multiple sub-images, a point is selected in each sub-image, and 5 points are found on both sides of the point in the vertical direction to construct a point set. Then, the curvature of each row of points in the sub-image is calculated, and the edge points and plane points are extracted according to the curvature;

[0073] The curvature calculation formula is:

[0074] Among them, c represents the curvature, S represents the point set, r i Represents the row index, r j represents the column index, i and j represent the row and column of the point in the image respectively, · represents the value of the determinant of the matrix, · represents the determinant of the matrix;

[0075] Step S3 also includes the following steps:

[0076] S305, estimating the posture information of the wheelchair according to the change between two adjacent frames of data, including feature point association and posture solution;

[0077] S306, feature point association is to associate the laser point cloud at time t with the laser point cloud at time t-1, and to construct a constraint relationship between two adjacent scans based on the extracted edge points and plane points to obtain a posture transformation matrix;

[0078] S307, posture solution matches the feature point cloud according to the posture transformation matrix, and estimates the posture of the laser radar according to the matched feature point cloud, that is, estimates the posture of the wheelchair;

[0079] S308, completing point cloud recognition through a convolutional neural network, outputting obstacle information and road boundary information, and matching the laser point cloud at time t with the global map feature point cloud at time t-1, and then adding them to the global map;

[0080] In step S306, the constraint relationship construction process of two adjacent scans is as follows: a DETA100R 4G sensor and an inertial sensor are installed on the wheelchair. The DETA100R 4G sensor receives the differential positioning signal broadcast by the RTK base station based on the network RTK positioning technology, collects information using the inertial sensor, and obtains the attitude transformation matrix using the matching algorithm;

[0081] The specific matching algorithm is:

[0082] x=(q,v x ,v y ,p x ,p y ) T

[0083] u=(a x ,a y ,ω) T

[0084] z=(s,θ,v)

[0085] Among them, x, u, z represent the state vector, control vector and observation vector respectively, q represents the rotation amount of the wheelchair movement, and v x 、p x 、a x Represent the velocity, displacement and acceleration on the x-axis respectively, v y 、p y 、a y They represent the velocity, displacement and acceleration on the y-axis respectively, ω represents the angular velocity of the z-axis. The acceleration and angular velocity are provided by the inertial sensor. s, θ and v are provided by the RTK data, representing the displacement, rotation speed and moving speed respectively.

[0086]

[0087]

[0088]

[0089]

[0090] Among them, x t represents the state vector at time t, u t represents the control vector at time t, z t represents the measurement vector at time t, f(·) represents the measurement value created using the inertial sensor, represents the intermediate variable, P t represents the covariance estimator at time t, A t represents the state transfer matrix, represents the intermediate variable, A t use The derivative of is calculated from Indicates A t The transposed matrix, Q t represents the process noise covariance, H t represents the state transfer matrix, H t use The derivative of is calculated from Indicates H t The transposed matrix, R t Represents the measurement error, K t represents the posture transformation matrix;

[0091] In step S4, the disabling mechanism, the stopping mechanism, the avoiding mechanism and the centering mechanism include the following steps:

[0092] S401, disabling mechanism: The wheelchair is equipped with a seat belt and a heart rate bracelet. When the user uses the wheelchair, if the user does not fasten the seat belt or wear the heart rate bracelet, the wheelchair is prohibited from starting and waits for the user to handle it;

[0093] S402, stop mechanism: When the laser radar detects a fixed object in front of the wheelchair and the distance from the wheelchair is less than 2 meters, the wheelchair stops automatically and waits for the user to handle it;

[0094] S403, avoidance mechanism: When the wheelchair is moving, if the laser ranging sensors on the left, right, or rear side detect an object, and the distance between the object and the wheelchair is less than 30 cm, the wheelchair moves away from the object to avoid the object;

[0095] S404, centering mechanism: when objects are detected on both the left and right sides of the wheelchair, the wheelchair moves to the center position according to the distances to the objects on both sides of the wheelchair, so that the distances between the wheelchair and the objects on both sides are equal;

[0096] In step S5, the wheelchair state includes posture information and speed information;

[0097] The posture information includes the pitch angle, roll angle and yaw angle of the wheelchair. After analysis, it outputs whether the wheelchair has tipped over or is in a dangerous state. If so, the wheelchair is automatically controlled, otherwise, it remains as it is.

[0098] Speed ​​information is used for wheelchair speed limit and speed information prompts;

[0099] The wheelchair safety response system based on behavior recognition includes:

[0100] Gesture recognition unit: The gesture recognition unit acquires the original feature data of the detected target by driving the sensor array inside the PAJ7620 sensor. The acquired data is stored in the register. At the same time, the gesture recognition array recognizes and processes the original data, and finally stores the gesture result in the register. When the embedded DSP processor detects the gesture change, it controls the movement of the wheelchair.

[0101] Information detection unit, which obtains accurate road surface information through the cooperation of laser radar and laser ranging sensor;

[0102] Feature analysis unit: The feature analysis unit extracts features from the 3D laser point cloud data image, then associates the laser point cloud at time t with the laser point cloud at time t-1, estimates the position of the wheelchair, and finally performs fusion mapping. The map is spliced ​​according to each frame of laser point cloud data sent by the radar to obtain a complete global map, thereby changing the direction of the wheelchair;

[0103] Safety execution unit, which forces the user to use the wheelchair as specified through a disabling mechanism, assists the user in emergency avoidance of objects through a stop mechanism and an avoidance mechanism, and controls the autonomous movement of the wheelchair through a centering mechanism;

[0104] The wheelchair response unit adjusts the movement direction and speed of the wheelchair according to the user's psychological behavior during driving, the recognized user gesture data and the wheelchair status.

[0105] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0106] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A wheelchair safety response method based on behavior recognition, characterized in that: The method comprises the following steps: S1. Identify user gesture data: The wheelchair is equipped with a PAJ7620 sensor and an embedded DSP processor. The infrared LED inside the PAJ7620 sensor is driven to emit infrared signals. When the sensor array detects an object, the sensor array acquires the original feature data of the detected target, and the acquired data is stored in the register. At the same time, the gesture recognition array recognizes and processes the original data, and finally stores the gesture result in the register. When the embedded DSP processor detects a gesture change, it controls the movement of the wheelchair. S2. Detecting the road surface information around the wheelchair: Six laser rangefinder sensors are installed around the wheelchair. The laser rangefinder sensors are used to sense the object information on the left, right and rear of the wheelchair. The sensing range of the laser rangefinder sensor is set to 5 meters. When the laser rangefinder sensor detects an object within the sensing range during the movement of the wheelchair, it sends a signal to the embedded DSP processor. The laser radar senses the object within 120° and 60 meters in front of the wheelchair, obtains the speed of the object and the distance between the object and the wheelchair, and obtains accurate road surface information through the cooperation of the laser radar and the laser rangefinder sensor. S3, extract and analyze feature information: During the movement of the wheelchair, ground points and non-ground points are divided from the 3D laser point cloud data image, and then feature extraction is performed to extract edge points and plane points. The laser point cloud at time t and the laser point cloud at time t-1 are then associated to estimate the position of the wheelchair. Finally, a fusion map is constructed. The map is spliced ​​according to each frame of laser point cloud data sent by the radar to obtain a complete global map, thereby changing the direction of the wheelchair. S4. Execution of safety measures: The user is forced to use the wheelchair as specified through the disabling mechanism. When the sensor array does not detect the user's gesture operation information and the wheelchair is in an unsafe state, the user is assisted in emergency avoidance of objects through the stop mechanism and the avoidance mechanism, and the autonomous movement of the wheelchair is controlled through the centering mechanism. S5. Real-time adjustment of the wheelchair: The wheelchair state is obtained according to the estimated wheelchair posture, and the moving direction and speed of the wheelchair are adjusted according to the user's psychological behavior during driving, the recognized user gesture data and the wheelchair state.

2. The wheelchair safety response method based on behavior recognition according to claim 1, characterized in that: The gesture change in step S1 includes the following steps: S101, the hand gesture is forward or backward, and the wheelchair moves forward or backward; S102: The hand gesture is left or right, and the wheelchair turns left or right; S103: The gesture is down, and the wheelchair stops moving.

3. The wheelchair safety response method based on behavior recognition according to claim 1, characterized in that: The step S3 comprises the following steps: S301, laser radar will send a frame of data each time it scans. One frame of data represents one frame of laser point cloud, expressed as P t ={p1,p2,···,p i ,···,p n }, where p i is the laser point cloud P at time t t A point in the laser point cloud is projected into a two-dimensional image. The pixels of each point in the image form a distance image, which is the distance between the actual object and the laser radar. S302. For the ground point, the vertical scanning angle of the laser radar is [-12.5°, +12.5°]. The laser radar is installed horizontally, and the ground point is located on the scanning line [-12.5°, 0]. S303, for the remaining non-ground points, classify the distance image, mark the points of the same class with unique labels, and treat the class with less than 30 points as noise points; S304. Each frame of data saved has a label of each category and row and column indexes and pixel values ​​in the range image.

4. The wheelchair safety response method based on behavior recognition according to claim 3 is characterized in that: In step S303, edge features and plane features of the laser point cloud are extracted from the classified range image to obtain the posture transformation relationship between time t and time t-1. First, the range image is divided into multiple sub-images, a point is selected in each sub-image, and 5 points are found on both sides of the point in the vertical direction to construct a point set. Then, the curvature of each row of points in the sub-image is calculated, and the edge points and plane points are extracted according to the curvature.

5. The wheelchair safety response method based on behavior recognition according to claim 4, characterized in that: The step S3 further comprises the following steps: S305, estimating the posture information of the wheelchair according to the change between two adjacent frames of data, including feature point association and posture solution; S306, feature point association is to associate the laser point cloud at time t with the laser point cloud at time t-1, and to construct a constraint relationship between two adjacent scans based on the extracted edge points and plane points to obtain a posture transformation matrix; S307, posture solution matches the feature point cloud according to the posture transformation matrix, and estimates the posture of the laser radar according to the matched feature point cloud, that is, estimates the posture of the wheelchair; S308, completing point cloud recognition through convolutional neural network, outputting obstacle information and road boundary information, matching the laser point cloud at time t with the global map feature point cloud at time t-1, and then adding them to the global map.

6. The wheelchair safety response method based on behavior recognition according to claim 5, characterized in that: In step S306, the constraint relationship construction process of two adjacent scans is as follows: a DETA100R 4G sensor and an inertial sensor are installed on the wheelchair, the DETA100R 4G sensor receives the differential positioning signal broadcast by the RTK base station based on the network RTK positioning technology, and uses the inertial sensor to collect information, and uses the matching algorithm to obtain the attitude transformation matrix; The specific matching algorithm is: x=(q,v x ,v y ,p x ,p y ) T u=(a x ,a y ,ω) T z=(s,θ,v) Among them, x, u, z represent the state vector, control vector and observation vector respectively, q represents the rotation amount of the wheelchair movement, and v x 、p x 、a x Represent the velocity, displacement and acceleration on the x-axis respectively, v y 、p y 、a y They represent the velocity, displacement and acceleration on the y-axis respectively, ω represents the angular velocity of the z-axis. The acceleration and angular velocity are provided by the inertial sensor. s, θ and v are provided by the RTK data, representing the displacement, rotation speed and moving speed respectively. Among them, x t represents the state vector at time t, u t represents the control vector at time t, z t represents the measurement vector at time t, f(·) represents the measurement value created using the inertial sensor, represents the intermediate variable, P t represents the covariance estimator at time t, A t represents the state transfer matrix, represents the intermediate variable, A t use The derivative of is calculated from Indicates A t The transposed matrix, Q t represents the process noise covariance, H t represents the state transfer matrix, H t use The derivative of is calculated from Indicates H t The transposed matrix, R t Represents the measurement error, K t Represents the posture transformation matrix.

7. The wheelchair safety response method based on behavior recognition according to claim 1, characterized in that: In step S4, the disabling mechanism, stopping mechanism, avoiding mechanism and centering mechanism include the following steps: S401, disabling mechanism: The wheelchair is equipped with a seat belt and a heart rate bracelet. When the user uses the wheelchair, if the user does not fasten the seat belt or wear the heart rate bracelet, the wheelchair is prohibited from starting and waits for the user to handle it; S402, stop mechanism: When the laser radar detects a fixed object in front of the wheelchair and the distance from the wheelchair is less than 2 meters, the wheelchair stops automatically and waits for the user to handle it; S403, avoidance mechanism: When the wheelchair is moving, if the laser ranging sensors on the left, right, or rear side detect an object, and the distance between the object and the wheelchair is less than 30 cm, the wheelchair moves away from the object to avoid the object; S404, centering mechanism: When objects are detected on both the left and right sides of the wheelchair, the wheelchair moves to the center position according to the distances to the objects on both sides of the wheelchair, so that the distances between the wheelchair and the objects on both sides are equal.

8. The wheelchair safety response method based on behavior recognition according to claim 1, characterized in that: In step S5, the wheelchair state includes posture information and speed information; The posture information includes the pitch angle, roll angle and yaw angle of the wheelchair. After analysis, it outputs whether the wheelchair has tipped over or is in a dangerous state. If so, the wheelchair is automatically controlled, otherwise, it remains as it is. Speed ​​information is used for wheelchair speed limit and speed information prompts.

9. The wheelchair safety response system based on behavior recognition is characterized by: The wheelchair safety response system is applicable to the wheelchair safety response method based on behavior recognition described in claims 1-8, including: A gesture recognition unit, which acquires the original feature data of the detected target by driving the sensor array inside the PAJ7620 sensor. The acquired data is stored in a register. Meanwhile, the gesture recognition array recognizes and processes the original data, and finally stores the gesture result in the register. When the embedded DSP processor detects the gesture change, it controls the movement of the wheelchair. An information detection unit, wherein the information detection unit obtains accurate road surface information through the cooperation of a laser radar and a laser ranging sensor; A feature analysis unit, which extracts features from the three-dimensional laser point cloud data image, then associates the laser point cloud at time t with the laser point cloud at time t-1, estimates the position of the wheelchair, and finally performs fusion mapping. The map is spliced ​​according to each frame of laser point cloud data sent by the radar to obtain a complete global map, thereby changing the direction of the wheelchair; A safety execution unit, which forces the user to use the wheelchair as specified through a disabling mechanism, assists the user in completing emergency avoidance of objects through a stop mechanism and an avoidance mechanism, and controls the autonomous movement of the wheelchair through a centering mechanism; A wheelchair response unit adjusts the moving direction and moving speed of the wheelchair according to the user's psychological behavior during driving, the recognized user gesture data and the wheelchair status.

Images