Control and alarm method applied to walking assisting device and computer equipment

Abstract

The embodiment of the invention relates to the field of walking assistance, in particular to a control and alarm method applied to a walking assistance device and computer equipment. A specific embodiment of the method comprises the following steps: opening a frame assembly; closing the frame assembly, and obtaining a scene sensing information sequence from the infrared sensor; inputting the scene sensing information sequence into a pre-trained first scene feature recognition model to obtain first scene feature information; acquiring scene video information from a video acquisition device; preprocessing the scene video information to generate a scene picture information set; inputting the scene picture information set into a pre-trained second scene feature recognition model to obtain second scene feature information; performing path planning processing on the walking target information, the first scene feature information and the second scene feature information to generate walking auxiliary information; and sending the walking assistance information to the user terminal. According to the embodiment, the safety of the patient during rehabilitation walking can be improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of assisted walking, and specifically to control and alarm methods and computer equipment applied to assisted walking devices. Background Art

[0002] Patients who undergo cerebrovascular intervention usually need to stay in bed for 24 hours, and then need to walk for rehabilitation. However, due to the long lying time, it is easy to cause transient ischemic attack, leading to falls, especially for elderly patients. This risk is particularly high. Once a fall occurs, it will cause irreversible serious consequences. Therefore, when walking for rehabilitation, they often use crutches, wheelchairs and other auxiliary tools to assist walking. However, the auxiliary tools currently used lack protective measures. When patients use auxiliary tools for rehabilitation walking, they are prone to falling, resulting in low safety for patients during rehabilitation walking. Summary of the invention

[0003] The content of this application is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this application is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0004] Some embodiments of the present application propose control and alarm methods and computer equipment applied to auxiliary walking devices to solve one or more of the technical problems mentioned in the above background technology section.

[0005] In a first aspect, some embodiments of the present application provide a control and alarm method for an auxiliary walking device, wherein the auxiliary walking device includes: a frame component, a safety protection device, an infrared sensor, and a video acquisition device. The method includes: in response to receiving an opening instruction sent by a user terminal, opening the frame component so that the user terminal starts the safety protection device and sends a completion instruction and walking target information; in response to receiving the completion instruction and walking target information sent by the user terminal, closing the frame component and acquiring a scene sensing information sequence from the infrared sensor; inputting the scene sensing information sequence into a pre-trained first scene feature recognition model to obtain first scene feature information; acquiring scene video information from the video acquisition device; pre-processing the scene video information to generate a scene picture information set; inputting the scene picture information set into a pre-trained second scene feature recognition model to obtain second scene feature information; performing path planning processing on the walking target information, the first scene feature information, and the second scene feature information to generate walking assistance information; and sending the walking assistance information to the user terminal.

[0006] In a second aspect, the present application also provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the method described in any implementation of the first aspect is implemented.

[0007] In a third aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the method described in any implementation of the first aspect is implemented.

[0008] The above-mentioned embodiments of the present application have the following beneficial effects: through the control and alarm method applied to the auxiliary walking device of some embodiments of the present application, the safety of the patient in rehabilitation walking can be improved. Specifically, the reason for the low safety of the patient in rehabilitation walking is that the auxiliary tools currently used lack protective measures, and the patient is prone to fall when using the auxiliary tools for rehabilitation walking. Based on this, the control and alarm method applied to the auxiliary walking device of some embodiments of the present application, first, in response to receiving the opening instruction sent by the user terminal, the above-mentioned framework component is turned on so that the user terminal starts the safety protection device and sends the completion instruction and walking target information. Thus, the safety protection device can be used to protect the patient, which can improve the safety of the patient in rehabilitation walking. Secondly, in response to receiving the completion instruction and walking target information sent by the above-mentioned user terminal, the above-mentioned framework component is turned off, and the scene sensing information sequence is obtained from the above-mentioned infrared sensor. Thus, the framework component can be turned off, so that the patient can always be inside the auxiliary walking device when walking, further improving the safety of the patient in rehabilitation walking. Then, the above-mentioned scene sensing information sequence is input into the pre-trained first scene feature recognition model to obtain the first scene feature information. Thus, the first scene feature information representing the obstacle information of the current scene identified by the infrared sensor can be obtained. Then. The scene video information is obtained from the above-mentioned video acquisition device. Then, the above-mentioned scene video information is preprocessed to generate a scene picture information set. Thus, scene picture information of different angles can be collected by the video acquisition device so as to subsequently identify the second scene feature information. After that, the above-mentioned scene picture information set is input into the pre-trained second scene feature recognition model to obtain the second scene feature information. Thus, the first scene feature information representing the obstacle information of the current scene identified by the video acquisition device can be obtained. Thereafter, the above-mentioned walking target information, the above-mentioned first scene feature information and the above-mentioned second scene feature information are processed by path planning to generate walking assistance information. Thus, more accurate walking assistance information considering the first scene feature information and the second scene feature information can be obtained. Finally, the above-mentioned walking assistance information is sent to the above-mentioned user terminal. Thus, the patient can perform rehabilitation walking according to the more accurate walking assistance information, which can reduce the situation where the patient hits obstacles. Thus, the safety of the patient in rehabilitation walking can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present application will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0010] Figure 1 is a flow chart of some embodiments of the control and alarm method applied to the walking assist device according to the present application; Figure 2 is a schematic structural diagram of some embodiments of the walking assistance device according to the present application; Figure 3 It is a schematic diagram of the structure of a computer device suitable for implementing some embodiments of the present application. DETAILED DESCRIPTION

[0011] Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0012] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0013] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0014] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0015] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0016] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0017] Figure 1 The process 100 of some embodiments of the control and alarm method applied to the auxiliary walking device according to the present application is shown. The auxiliary walking device includes: a frame component, a safety protection device, an infrared sensor, and a video acquisition device. The control and alarm method applied to the auxiliary walking device includes the following steps: Step 101, in response to receiving an opening instruction sent by a user terminal, opening the framework component so that the user terminal starts the safety protection device and sends a completion instruction and walking target information.

[0018] In some embodiments, the execution body of the control and alarm method applied to the auxiliary walking device responds to receiving the opening instruction sent by the user terminal, and opens the above-mentioned framework component so that the user terminal starts the safety protection device and sends the completion instruction and walking target information. Among them, the opening instruction can represent: the user terminal wants to enter the framework component included in the auxiliary walking device. The completion instruction can represent: the user terminal has successfully started the safety protection device. The walking target information may include but is not limited to at least one of the following: user target coordinates, user walking preference information. The user target coordinates can represent: the geographical coordinates of the location that the user terminal wants to reach. The user walking preference information can represent: the geographical coordinates of the location that the user terminal wants to pass. Here, the user terminal can be the terminal used by the target user. The target user can be a user who needs to use an auxiliary walking device to walk (for example, a cerebral hemorrhage patient, an elderly patient).

[0019] Here, the auxiliary walking device can refer to Figure 2 Schematic diagrams of the structures of some embodiments of the walking assistance device according to the present application are shown. Figure 2 As shown, the walking assistance device may include: a frame assembly 1, a safety protection device 2, an infrared sensor 3, and a video acquisition device 4. The frame assembly 1 may include: a guard plate 1001, a baffle 1002, an armrest 1003, a telescopic rod assembly 1004, and a base plate 1005. The guard plate 1001 and the baffle 1002 are connected by a lock. The armrest 1003 is installed above the guard plate 1001. The telescopic rod assembly 1004 is installed between the guard plate 1001 and the base plate 1005. The safety protection device 2 is installed on the guard plate 1001 for fixing the target user. The telescopic rod assembly 1004 may include five telescopic rods.

[0020] Here, the handrail 1003 is made of a soft non-slip material. This makes it convenient for the patient to support the body while walking. The telescopic rod assembly 1004 included in the frame assembly 1 can adjust the height of the auxiliary walking device to accommodate patients of different heights. Here, the frame is designed as an opening and closing structure that can be easily opened, similar to a foldable fence. When opened, the patient can easily enter the inside of the frame and close the frame to ensure safety during walking. The connection part of the frame adopts a strong lock design to ensure that it will not be accidentally opened during use.

[0021] For example, the safety protection device 2 may include a cloth safety seat 2001, on the upper part of which is installed an adjustable length safety belt 2002. Thus, the auxiliary walking device is equipped with a cloth safety seat that can be sat on in an emergency, and when the target user uses the auxiliary walking device to walk, the safety belt on the cloth safety seat can be fastened around the waist and crotch, which can prevent the target user from falling down due to transient ischemic attack, and can prevent the patient from being further injured.

[0022] For example, the video acquisition device 4 may be a rotatable camera.

[0023] In practice, opening the frame assembly may be: opening the lock between the guard plate 1001 and the baffle 1002 included in the frame assembly, and lifting the baffle 1002. The user terminal starts the safety protection device may be: the user terminal fastens the safety belt installed on the safety protection device 2 around the waist and crotch of the target user.

[0024] Step 102, in response to receiving the completion instruction and walking target information sent by the user terminal, closing the framework component and acquiring a scene sensing information sequence from the infrared sensor.

[0025] In some embodiments, the execution subject may close the framework component and obtain a scene sensing information sequence from the infrared sensor in response to receiving a completion instruction and walking target information sent by the user terminal. The scene sensing information in the scene sensing information sequence may represent an infrared image of the current scene collected from the infrared sensor. The scene sensing information sequence may be each scene sensing information obtained from the infrared sensor within a preset time period. Here, the scene sensing information in the scene sensing information sequence may correspond to a time granularity within a preset time period. For example, the preset time period may be: a time period from one hour before the current time to the current time. The time granularity may be one minute.

[0026] In practice, closing the frame assembly may be as follows: lowering the raised baffle 1002 and fastening the lock between the guard plate 1001 and the baffle 1002 .

[0027] Step 103: input the scene sensing information sequence into a pre-trained first scene feature recognition model to obtain first scene feature information.

[0028] In some embodiments, the execution subject may input the scene sensing information sequence into a pre-trained first scene feature recognition model to obtain first scene feature information. The first scene feature recognition model may be a neural network model that takes a pre-trained scene sensing information sequence as input and takes the first scene feature information as output. Here, the first scene feature information may represent information (including area, geographic coordinates) of obstacles (e.g., living things such as humans and animals) included in the current scene.

[0029] Optionally, before inputting the scene sensing information sequence into a pre-trained first scene feature recognition model to obtain the first scene feature information, the method further includes: The first step is to obtain the first training sample set.

[0030] In some embodiments, the execution subject may obtain a first training sample set, wherein the first training sample in the first training sample set includes: a sample scene sensing information sequence and first sample scene feature information.

[0031] The second step is to determine the first initial scene feature recognition model.

[0032] In some embodiments, the execution subject may determine a first initial scene feature recognition model, wherein the first initial scene feature recognition model includes: an initial prediction model and an initial feature extraction model.

[0033] The initial prediction model may be a neural network model that takes a sample scene sensing information sequence as input and takes initial scene prediction information as output. For example, the initial prediction model may be, but is not limited to, an ARMA (auto regression moving average) model or an LSTM (Long Short-Term Memory) model.

[0034] The initial feature extraction model may be a neural network model that takes the initial scene prediction information as input and takes the first initial scene feature information as output. For example, the initial feature extraction model may be, but is not limited to, a SIFT (Scale-Invariant Feature Transform) model, an ORB (Oriented FAST and Rotated BRIEF) model, or a CNN (Convolutional Neural Networks) model.

[0035] The third step is to select the first target training sample from the first training sample set.

[0036] In some embodiments, the execution entity may select the first target training sample from the first training sample set. In practice, the execution entity may randomly select the first training sample from the first training sample set as the first target training sample.

[0037] In the fourth step, the sample scene sensing information sequence included in the selected first target training sample is input into the initial prediction model to obtain initial scene prediction information.

[0038] In some embodiments, the execution entity may input the sample scene sensing information sequence included in the selected first target training sample into the initial prediction model to obtain initial scene prediction information.

[0039] The fifth step is to input the initial scene prediction information into the initial feature extraction model to obtain the first initial scene feature information.

[0040] In some embodiments, the above-mentioned execution entity can input the initial scene prediction information into the initial feature extraction model to obtain the first initial scene feature information.

[0041] In the sixth step, based on the first preset loss function, a first difference value between the first initial scene feature information and the first sample scene feature information is determined.

[0042] In some embodiments, the execution subject may determine the first difference value between the first initial scene feature information and the first sample scene feature information based on a first preset loss function. For example, the first preset loss function may be, but is not limited to: mean square error loss function (MSE), hinge loss function, cross entropy loss function (CrossEntropy), 0-1 loss function, absolute value loss function, log loss function, square loss function, exponential loss function, etc.

[0043] In a seventh step, in response to determining that the first difference value is greater than or equal to the first preset difference value, adjusting the network parameters of the first initial scene feature recognition model.

[0044] In some embodiments, the execution subject may adjust the network parameters of the first initial scene feature recognition model in response to determining that the first difference value is greater than or equal to the first preset difference value. For example, the difference between the first difference value and the first preset difference value may be calculated. On this basis, the parameters of the initial code scheduling information prediction model are adjusted using methods such as back propagation and gradient descent. For example, the first preset difference value may be 0.1.

[0045] Optionally, in response to determining that the first difference value is smaller than the first preset difference value, the first initial scene feature recognition model is determined as the first scene feature recognition model.

[0046] In some embodiments, the execution entity may determine the first initial scene feature recognition model as the first scene feature recognition model in response to determining that the first difference value is less than the first preset difference value.

[0047] Thus, firstly, the initial prediction model can be used to predict relatively accurate initial scene prediction information representing the infrared image at the next time granularity based on the scene sensing information sequence representing the infrared image in a preset time period. Then, the initial feature extraction model can be used to relatively accurately predict the first initial scene feature information representing the obstacle information from the initial scene prediction information. Therefore, a relatively accurate first scene feature recognition model can be trained by training a first initial scene feature recognition model including the initial prediction model and the initial feature extraction model. Thus, relatively accurate first scene feature information representing the obstacle information at the next time granularity can be identified based on the relatively accurate first scene feature recognition model. Furthermore, the target user can perform rehabilitation walking based on the relatively accurate first scene feature information, which can reduce the situation where the target user hits an obstacle. Thus, the safety of the target user during rehabilitation walking can be improved.

[0048] Step 104, obtaining scene video information from a video acquisition device.

[0049] In some embodiments, the execution entity may obtain scene video information from the video acquisition device.

[0050] Optionally, the video acquisition device may include: a video light sensor, a video lighting system, and a camera. The video light sensor may be a sensor for acquiring the luminous intensity of the scene captured by the camera. The video lighting system may be a light bulb for illuminating the scene captured by the camera. The camera may be a rotatable camera.

[0051] Optionally, before acquiring the scene video information from the video acquisition device, the method further includes: The first step is to obtain video optical information from the video optical sensor, wherein the video optical information can represent the luminous intensity of the current scene collected by the video optical sensor.

[0052] The second step is to, in response to determining that the video optical information meets a preset video low-light condition, turn on the video lighting system and control the camera to capture scene video information. The preset video low-light condition may be: the luminous intensity represented by the video optical information is less than 1.

[0053] The third step is to control the camera to capture scene video information in response to determining that the video optical information does not satisfy the preset video low-light condition.

[0054] Thus, the video light sensor can be used to identify whether the environment captured by the camera is in a low-light environment. When in a low-light environment, the video lighting system can be turned on so that the camera can capture a clearer picture.

[0055] Step 105: pre-process the scene video information to generate a scene picture information set.

[0056] In some embodiments, the execution entity may pre-process the scene video information to generate a scene picture information set.

[0057] In practice, the execution subject may pre-process the scene video information through the following steps to generate a scene picture information set: The first step is to perform frame processing on the scene video information to generate an initial scene picture information set. In practice, the execution subject can determine each frame of the scene video information as the initial scene picture information to obtain the initial scene picture information set. The initial scene picture information in the initial scene picture information set can be a picture taken by a video acquisition device with the auxiliary walking device as the center and at a preset shooting angle. For example, the preset shooting angle can be: front, back, left, right, left front, right front, left back, right back.

[0058] The second step is to select a target scene picture information set from the above-mentioned initial scene picture information set. In practice, first, the above-mentioned execution subject may use each initial scene picture information that meets the preset angle condition in the initial scene picture information set as a scene angle picture information group to obtain a scene angle picture information group set. Then, for each scene angle picture information group in the above-mentioned scene angle picture information group set, the above-mentioned execution subject may randomly select scene angle picture information from the above-mentioned scene angle picture information group as the target scene picture information. Afterwards, the above-mentioned execution subject may determine the selected target scene picture information as the target scene picture information set. Among them, the preset angle condition may be: the preset shooting angles corresponding to each initial scene picture in the initial scene picture information set are the same.

[0059] In the third step, for each target scene image information in the target scene image information set, perform the following image cleaning sub-steps: In the first sub-step, grayscale processing is performed on the target scene image information to generate scene grayscale image information. In practice, the execution subject may perform grayscale processing on the target scene image information by a preset grayscale algorithm to generate scene grayscale image information. For example, the preset grayscale algorithm may be, but is not limited to, a maximum value method, an average value method, or a weighted average method.

[0060] The second sub-step is to perform noise reduction processing on the above-mentioned scene grayscale image information to generate scene noise reduction image information. In practice, the above-mentioned execution subject can perform noise reduction processing on the above-mentioned scene grayscale image information by using a preset noise reduction algorithm to generate scene noise reduction image information. For example, the preset noise reduction algorithm can be but is not limited to: a median filter algorithm, a mean filter algorithm, and a wavelet denoising algorithm.

[0061] The third sub-step is to perform equalization processing on the above-mentioned scene noise reduction image information to generate scene equalized image information. In practice, the above-mentioned execution subject can perform equalization processing on the above-mentioned scene noise reduction image information by using a preset equalization algorithm to generate scene equalized image information. For example, the preset equalization algorithm can be: a histogram equalization algorithm.

[0062] The fourth sub-step is to perform edge detection processing on the scene equalization image information to generate scene image information. In practice, the execution subject may perform edge detection processing on the scene equalization image information by using a preset edge detection algorithm to generate scene image information. For example, the preset edge detection algorithm may be, but is not limited to, a Sobel algorithm or a Roberts algorithm.

[0063] The fourth step is to determine the obtained scene picture information as a scene picture information set.

[0064] Therefore, first, the scene video information can be processed by frame division so as to process each frame of the picture taken by the camera. Then, the target scene picture information set can be selected from the initial scene picture information set so as to select pictures taken by the camera at different angles. After that, the target scene picture information can be processed by grayscale processing, noise reduction processing, equalization processing, and edge detection processing so as to clean the target scene picture information and obtain scene picture information with higher accuracy.

[0065] Step 106: input the scene image information set into a pre-trained second scene feature recognition model to obtain second scene feature information.

[0066] In some embodiments, the execution subject may input the scene image information set into a pre-trained second scene feature recognition model to obtain second scene feature information. The second scene feature recognition model may be a pre-trained neural network model that takes scene image information as input and takes second scene feature information as output. The second scene feature information may represent information (including: area, geographic coordinates) of obstacles (e.g., tables, walls, etc.) included in the current scene.

[0067] Optionally, before inputting the scene picture information set into a pre-trained second scene feature recognition model to obtain the second scene feature information, the method further includes: The first step is to obtain the second training sample set.

[0068] In some embodiments, the execution subject may obtain a second training sample set, wherein the second training samples in the second training sample set include: a sample scene picture information set and second sample scene feature information.

[0069] The second step is to determine the second initial scene feature recognition model.

[0070] In some embodiments, the execution subject may determine a second initial scene feature recognition model, wherein the second initial scene feature recognition model includes: an initial fusion model, an initial noise reduction model, and an initial feature extraction model.

[0071] The initial fusion model may be a model that takes the sample scene image information set as input and takes the initial fusion image information as output. For example, the initial fusion model may be: the image represented by each sample scene image information in the sample scene image information set is spliced ​​with the walking assist device as the center to obtain the initial fusion image information.

[0072] The initial denoising model may be a model that takes the initial fused image information as input and takes the initial scene denoised image information as output. For example, the initial denoising model may be a total variation denoising model.

[0073] The initial feature extraction model may be a neural network model that takes the initial scene denoised image information as input and takes the second initial scene feature information as output. For example, the initial feature extraction model may be, but is not limited to, a SIFT (Scale-Invariant Feature Transform) model, an ORB (Oriented FAST and Rotated BRIEF) model, or a CNN (Convolutional Neural Networks) model.

[0074] The third step is to select a second target training sample from the second training sample set.

[0075] In some embodiments, the execution entity may select the second target training sample from the second training sample set. In practice, the execution entity may randomly select the second training sample from the second training sample set as the second target training sample.

[0076] In the fourth step, the sample scene image information set included in the selected second target training sample is input into the initial fusion model to obtain the initial scene fusion image information.

[0077] In some embodiments, the execution entity may input the sample scene image information set included in the selected second target training sample into the initial fusion model to obtain initial scene fusion image information.

[0078] The fifth step is to input the initial scene fusion image information into the initial denoising model to obtain the initial scene denoised image information.

[0079] In some embodiments, the execution entity may input the initial scene fusion picture information into the initial denoising model to obtain the initial scene denoising picture information.

[0080] In the sixth step, the initial scene denoised image information is input into the initial feature extraction model to obtain the second initial scene feature information.

[0081] In some embodiments, the execution entity may input the initial scene denoised image information into an initial feature extraction model to obtain second initial scene feature information.

[0082] In the seventh step, based on the second preset loss function, a second difference value between the second initial scene feature information and the second sample scene feature information is determined.

[0083] In some embodiments, the execution subject may determine the second difference value between the second initial scene feature information and the second sample scene feature information based on a second preset loss function. For example, the second preset loss function may be, but is not limited to, a mean square error loss function (MSE), a hinge loss function, a cross entropy loss function (CrossEntropy), a 0-1 loss function, an absolute value loss function, a logarithmic loss function, a square loss function, an exponential loss function, etc.

[0084] In an eighth step, in response to determining that the second difference value is greater than or equal to the second preset difference value, adjusting the network parameters of the second initial scene feature recognition model.

[0085] In some embodiments, the execution subject may adjust the network parameters of the second initial scene feature recognition model in response to determining that the second difference value is greater than or equal to the second preset difference value. For example, the second difference value and the second preset difference value may be differentiated. On this basis, the parameters of the initial code scheduling information prediction model are adjusted using methods such as back propagation and gradient descent. For example, the second preset difference value may be 0.1.

[0086] Optionally, in response to determining that the second difference value is smaller than the second preset difference value, the second initial scene feature recognition model is determined as the second scene feature recognition model.

[0087] In some embodiments, the execution entity may determine the second initial scene feature recognition model as the second scene feature recognition model in response to determining that the second difference value is less than the second preset difference value.

[0088] Step 103 is combined with the optional technical content in step 106 as an inventive point of an embodiment of the present application, which solves the second technical problem "the safety of patients during rehabilitation walking is low". Among them, the factors that lead to the low safety of patients during rehabilitation walking are often as follows: obstacles in the current scene are not taken into account. When patients are performing rehabilitation walking, it is difficult for patients to accurately control the walking direction and distance, which makes it easy for patients to bump into obstacles. If the above factors are solved, the effect of improving the safety of patients during rehabilitation walking can be achieved. In order to achieve this effect, the present application can first train a first initial scene feature recognition model including an initial prediction model and an initial feature extraction model to obtain a relatively accurate first scene feature recognition model. Therefore, through the first scene feature recognition model, the first scene feature information characterizing the obstacle information at the next time granularity of the current scene can be more accurately identified. Then, a second initial scene feature recognition model including an initial fusion model, an initial denoising model, and an initial feature extraction model can be trained to obtain a relatively accurate second scene feature recognition model. Thus, through the second scene feature recognition model, the image information of each scene taken from different shooting angles by the camera can be fused, denoised, and feature extracted, so as to obtain the second scene feature information that more accurately identifies the obstacle information representing the current scene. Therefore, through the first scene feature recognition model, the scene sensing information sequence representing the infrared image obtained from the infrared sensor can predict the first scene feature information that more accurately represents the obstacle information of living bodies such as people and animals. And, through the second scene feature recognition model, the image information of each scene taken from different shooting angles obtained from the camera can be used to identify the second scene feature information that more accurately represents the obstacle information of tables, walls, etc. Thus, considering the first scene feature information and the second scene feature information, the obstacle information considered is more comprehensive, so that more accurate walking assistance information can be obtained according to the first scene feature information and the second scene feature information. Thus, the patient can perform rehabilitation walking according to the more accurate walking assistance information. Furthermore, the situation where the patient hits the obstacle can be reduced, and the safety of the patient in rehabilitation walking can be improved.

[0089] Step 107, performing path planning processing on the walking target information, the first scene feature information and the second scene feature information to generate walking assistance information.

[0090] In some embodiments, the execution subject may perform path planning processing on the walking target information, the first scene feature information, and the second scene feature information to generate walking assistance information. In practice, the execution subject may perform path planning processing using a preset path planning algorithm, with the walking target information as the destination, and the first scene feature information and the second scene feature information as obstacles to generate walking assistance information. For example, the preset path planning algorithm may be, but is not limited to: a BUG obstacle avoidance algorithm, an A* (A-star, A star) search algorithm.

[0091] Step 108: Send the walking assistance information to the user terminal.

[0092] In some embodiments, the execution entity may send the walking assistance information to the user terminal.

[0093] Alternatively, if Figure 2 As shown, the above-mentioned auxiliary walking device also includes: a pressure sensor 5, an emergency brake device 6, an alarm button 7, a sound alarm device 8, and a pulley assembly 9. The above-mentioned pressure sensor 5 is installed on the above-mentioned handrail 1003. The above-mentioned alarm button 7 is installed on the above-mentioned handrail. The above-mentioned sound alarm device 8 is installed on the above-mentioned guard plate 1001. The above-mentioned emergency brake device 6 is installed on the above-mentioned pulley assembly 9. The above-mentioned pulley assembly 9 includes four pulleys.

[0094] The pressure sensor 5 is used to collect the pressure value of the target user on the handrail 1003. The emergency brake device 6 is used to stop the pulley in the pulley assembly 9 in a sliding state. The sound alarm device 8 is used to send an alarm sound and flashing light to the target user.

[0095] Optionally, the above method further includes: The first step is to obtain a pressure value sequence from the pressure sensor. The pressure values ​​in the pressure value sequence may correspond to a pressure time granularity in a preset pressure time period. For example, the preset pressure time period may be a time period from ten seconds before the current time to the current time. The pressure time granularity may be one second.

[0096] In the second step, in response to determining that the pressure value sequence satisfies a preset abnormal pressure condition, the emergency braking device is activated. The preset abnormal pressure condition may be: the ratio of two adjacent pressure values ​​in the pressure value sequence is greater than a first target threshold or less than a second target threshold. For example, the first target threshold may be 4. The second target threshold may be 0.25.

[0097] In the third step, in response to determining that the pressure value sequence meets the preset abnormal pressure condition or in response to receiving an alarm instruction sent by the user terminal, the sound alarm device is activated to perform an alarm process on the user terminal. The alarm instruction may be: the target user clicks the alarm button 7.

[0098] Therefore, when the pressure sensor detects that the target user's pressure on the handrail suddenly increases or decreases, it indicates that the target user may suddenly grasp or release the handrail due to physical discomfort, and the emergency brake device can be activated to stop the auxiliary walking device, so that the target user stops in place, so as to reduce the situation where the target user continues to walk when the target user is in physical discomfort and causes the target user to hit an obstacle. Thereby, the safety of the target user during rehabilitation walking can be improved.

[0099] Optionally, in response to determining that the first scene feature information and the second scene feature information meet a preset abnormal condition, the emergency braking device is activated. The preset abnormal condition may be: the distance between the geographical coordinates of the obstacle represented by the first scene feature information and the geographical coordinates of the target user is less than a preset distance, or the distance between the geographical coordinates of the obstacle represented by the second scene feature information and the geographical coordinates of the target user is less than a preset distance. For example, the preset distance may be 30 centimeters.

[0100] Thus, when the target user is close to an obstacle, the emergency brake device can be activated to stop the target user in place, thereby reducing the chance of the target user hitting the obstacle. Thus, the safety of the target user during rehabilitation walking can be improved.

[0101] Alternatively, if Figure 2 As shown, the walking assisting device further includes: a light sensor 10 and a lighting system 11. The lighting system 11 is installed at the bottom of the frame assembly 1. The lighting system 11 can be a light strip. Thus, adding reflective strips on the edge of the frame and the wheels can improve visibility at night or in low light environments, further enhancing walking safety.

[0102] Optionally, the above method further includes: The first step is to obtain optical information from the above optical sensor, wherein the optical information can represent the luminous intensity of the current scene collected by the optical sensor.

[0103] The second step is to start the lighting system in response to determining that the optical information satisfies a preset low light condition, wherein the preset low light condition may be: the luminous intensity represented by the optical information is less than 1.

[0104] Optionally, the above method further includes: The first step is to obtain user status information. In practice, the above-mentioned execution subject can obtain user status information from the user terminal through a wired connection or a wireless connection. The user status information can represent the current physical state of the target user. For example, the user status information can represent that the target user needs to perform rehabilitation walking. The user status information can also represent that the target user needs to perform rehabilitation walking and that the target user has an eye injury.

[0105] In the second step, in response to determining that the user status information satisfies the preset status condition, the walking assistance device is controlled to start the first preset mode. The preset status condition may be: the user status information indicates that the target user needs to perform rehabilitation walking and the target user has an eye injury. Here, the first preset mode may be characterized as: fully automatic, that is, the target user does not need to walk, and only relies on the walking assistance device to walk to the location the target user wants to reach according to the route of the walking assistance information.

[0106] In the third step, in response to determining that the first scene feature information satisfies the first preset scene condition or the second scene feature information satisfies the second preset scene condition, the auxiliary walking device is controlled to start the first preset mode. The first preset scene condition may be: the ratio of the area of ​​the obstacle represented by the first scene feature information to the area of ​​the current scene is greater than the first preset ratio. The second preset scene condition may be: the ratio of the area of ​​the obstacle represented by the second scene feature information to the area of ​​the current scene is greater than the second preset ratio. For example, the first preset ratio may be 0.5. The second preset ratio may be 0.6.

[0107] In step 4, in response to determining that the first scene feature information does not satisfy the first preset scene condition and the second scene feature information does not satisfy the second preset scene condition, the walking assistance device is controlled to start the second preset mode. Here, the second preset mode can be characterized as: semi-automatic, that is, the target user can walk with the aid of the walking assistance device.

[0108] Thus, different modes of the assistive walking device can be activated for different situations. When the target user has an eye injury, the fully automatic mode can be activated, thereby reducing the situation where the target user bumps into obstacles due to blindness during rehabilitation walking. When the obstacles in the current scene account for a large proportion, the fully automatic mode can be activated, thereby reducing the situation where the target user bumps into obstacles due to a large number of obstacles during rehabilitation walking. When the obstacles in the current scene account for a small proportion, the semi-automatic mode can be activated, thereby allowing the target user to perform rehabilitation walking in a relatively open environment. Therefore, the safety of patients during rehabilitation walking can be improved.

[0109] The above-mentioned embodiments of the present application have the following beneficial effects: through the control and alarm method applied to the auxiliary walking device of some embodiments of the present application, the safety of the patient in rehabilitation walking can be improved. Specifically, the reason for the low safety of the patient in rehabilitation walking is that the auxiliary tools currently used lack protective measures, and the patient is prone to fall when using the auxiliary tools for rehabilitation walking. Based on this, the control and alarm method applied to the auxiliary walking device of some embodiments of the present application, first, in response to receiving the opening instruction sent by the user terminal, the above-mentioned framework component is turned on so that the user terminal starts the safety protection device and sends the completion instruction and walking target information. Thus, the safety protection device can be used to protect the patient, which can improve the safety of the patient in rehabilitation walking. Secondly, in response to receiving the completion instruction and walking target information sent by the above-mentioned user terminal, the above-mentioned framework component is turned off, and the scene sensing information sequence is obtained from the above-mentioned infrared sensor. Thus, the framework component can be turned off, so that the patient can always be inside the auxiliary walking device when walking, further improving the safety of the patient in rehabilitation walking. Then, the above-mentioned scene sensing information sequence is input into the pre-trained first scene feature recognition model to obtain the first scene feature information. Thus, the first scene feature information representing the obstacle information of the current scene identified by the infrared sensor can be obtained. Then. The scene video information is obtained from the above-mentioned video acquisition device. Then, the above-mentioned scene video information is preprocessed to generate a scene picture information set. Thus, scene picture information of different angles can be collected by the video acquisition device so as to subsequently identify the second scene feature information. After that, the above-mentioned scene picture information set is input into the pre-trained second scene feature recognition model to obtain the second scene feature information. Thus, the first scene feature information representing the obstacle information of the current scene identified by the video acquisition device can be obtained. Thereafter, the above-mentioned walking target information, the above-mentioned first scene feature information and the above-mentioned second scene feature information are processed by path planning to generate walking assistance information. Thus, more accurate walking assistance information considering the first scene feature information and the second scene feature information can be obtained. Finally, the above-mentioned walking assistance information is sent to the above-mentioned user terminal. Thus, the patient can perform rehabilitation walking according to the more accurate walking assistance information, which can reduce the situation where the patient hits obstacles. Thus, the safety of the patient in rehabilitation walking can be improved. The present application also provides a computer device 300. Figure 3 As shown, the computer device 300 includes: a bus 301, a processor 302, a memory 303 and a communication interface 304. The processor 302, the memory 303 and the communication interface 304 communicate with each other through the bus 301. The computer device 300 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computer device 300.

[0110] The bus 301 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 The bus 301 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 301 may include a path for transmitting information between various components of the computer device 300 (for example, the memory 303, the processor 302, and the communication interface 304).

[0111] The processor 302 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0112] The memory 303 may include a volatile memory, such as a random access memory (RAM). The memory 303 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0113] The memory 303 stores executable program codes, and the processor 302 executes the executable program codes to respectively implement the functions of the aforementioned acquisition module, sampling module, determination module, and mixing module, thereby implementing the aforementioned control and alarm method for the assistive walking device. That is, the memory 303 stores instructions for executing the aforementioned control and alarm method for the assistive walking device.

[0114] The communication interface 304 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computer device 300 and other devices or a communication network.

[0115] An embodiment of the present application also provides a chip, which includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface to execute the above-mentioned control and alarm method applied to the auxiliary walking device.

[0116] The embodiment of the present application also provides a computer-readable storage medium. The above-mentioned computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The above-mentioned available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state hard disk), etc. The computer-readable storage medium includes instructions, and the above-mentioned instructions instruct the computing device to execute the above-mentioned control and alarm method for the auxiliary walking device.

[0117] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0118] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A control and alarm method for a walking assistance device, the walking assistance device comprising: Frame assembly, safety protection device, infrared sensor, video acquisition device, the method includes: In response to receiving an opening instruction sent by a user terminal, opening the framework component so that the user terminal starts the safety protection device and sends a completion instruction and walking target information; In response to receiving the completion instruction and walking target information sent by the user terminal, closing the framework component and acquiring a scene sensing information sequence from the infrared sensor; Inputting the scene sensing information sequence into a pre-trained first scene feature recognition model to obtain first scene feature information; Acquiring scene video information from the video acquisition device; Preprocessing the scene video information to generate a scene picture information set; Inputting the scene picture information set into a pre-trained second scene feature recognition model to obtain second scene feature information; Performing path planning processing on the walking target information, the first scene feature information, and the second scene feature information to generate walking assistance information; The walking assistance information is sent to the user terminal.

2. The control and alarm method for a walking assist device according to claim 1, wherein: The auxiliary walking device further comprises: a pressure sensor, an emergency brake device, an alarm button, and a sound alarm device; the frame assembly comprises: a guard plate, a baffle, an armrest, a telescopic rod assembly, and a base plate; the guard plate and the baffle are connected by a lock; the armrest is installed above the guard plate; the pressure sensor is installed on the armrest; the alarm button is installed on the armrest; and the method further comprises: Acquire a sequence of pressure values ​​from the pressure sensor; In response to determining that the pressure value sequence satisfies a preset abnormal pressure condition, activating the emergency braking device; In response to determining that the pressure value sequence meets the preset abnormal pressure condition or in response to receiving an alarm instruction sent by a user terminal, the sound alarm device is activated to perform an alarm process on the user terminal.

3. The control and alarm method for a walking assist device according to claim 1, wherein: The walking assist device comprises: a light sensor and a lighting system; the lighting system is mounted at the bottom of the frame assembly; and the method further comprises: acquiring optical information from the optical sensor; In response to determining that the optical information satisfies a preset low-light condition, the lighting system is activated.

4. The control and alarm method for a walking assist device according to claim 1, further comprising: Get user status information; In response to determining that the user status information meets a preset status condition, controlling the walking assistance device to start a first preset mode; In response to determining that the first scene feature information satisfies a first preset scene condition or the second scene feature information satisfies a second preset scene condition, controlling the walking assistance device to start a first preset mode; In response to determining that the first scene feature information does not satisfy the first preset scene condition and the second scene feature information does not satisfy the second preset scene condition, the walking assistance device is controlled to start a second preset mode.

5. The control and alarm method for a walking assist device according to claim 1, wherein: The video acquisition device includes: a video light sensor, a video lighting system, and a camera; and before acquiring scene video information from the video acquisition device, the method further includes: Acquiring video optical information from the video light sensor; In response to determining that the video optical information meets a preset video low-light condition, turning on the video lighting system, and controlling the camera to capture scene video information; In response to determining that the video optical information does not satisfy the preset video low-light condition, the camera is controlled to capture scene video information.

6. The control and alarm method for a walking assist device according to claim 1, wherein: The preprocessing of the scene video information to generate a scene picture information set includes: Performing frame processing on the scene video information to generate an initial scene picture information set; Selecting a target scene picture information set from the initial scene picture information set; For each target scene image information in the target scene image information set, the following image cleaning steps are performed: Performing grayscale processing on the target scene image information to generate scene grayscale image information; Performing noise reduction processing on the scene grayscale image information to generate scene noise reduction image information; Performing equalization processing on the scene noise reduction picture information to generate scene equalized picture information; Performing edge detection processing on the scene equalization picture information to generate scene picture information; The obtained individual scene picture information is determined as a scene picture information set.

7. A computer device, wherein: The computer device comprises a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium, wherein: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images