An all-round remote safety monitoring system for intelligent wheelchairs

Abstract

The present invention discloses an all-round remote safety monitoring system for an intelligent wheelchair. A Raspberry Pi, a monitoring PTZ and a speaker are installed on the armrests of the wheelchair. The system is small in size, low in cost and easy to install. The system takes the Raspberry Pi as a central controller and automatically initializes after startup. The monitoring information of the wheelchair side can be remotely obtained without the operation of the wheelchair occupant. The system can monitor the surrounding environment of the wheelchair as well as the wheelchair and the occupant themselves. The system also has an original personalized system authority authentication method, which maximizes the data security of wheelchair monitoring.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent wheelchair monitoring, and in particular relates to an all-round remote safety monitoring system for an intelligent wheelchair. Background Art

[0002] The problem of aging population in my country is becoming increasingly serious. The elderly and disabled people have difficulty in moving, and the safety issues in their daily life and travel need to be solved urgently. At present, the travel needs of the elderly and others are diversified, and their activity areas include but are not limited to home, hospital, nursing home, etc., and they mainly rely on auxiliary mobility tools such as wheelchairs. However, existing wheelchairs have insufficient perception capabilities, the security monitoring system has a limited monitoring angle, and is not flexible enough to use. In addition, the system has no permission authentication function, and anyone can obtain monitoring videos. Data security cannot be guaranteed, and personalized wheelchair monitoring cannot be achieved. In order to achieve safe monitoring of smart wheelchairs, it is necessary to combine sensors such as cameras and monitoring pan-tilt heads to achieve remote safety monitoring of smart wheelchairs, etc.

[0003] Currently developed wheelchair monitoring methods include: 1) installing a camera on the wheelchair to capture image information via a local area network; 2) relying on indoor and outdoor monitoring equipment to monitor the wheelchair's status while the wheelchair is in motion; and 3) installing a camera and microprocessor on the wheelchair to process images and obtain information about the environment in front of the wheelchair. However, dedicated monitoring systems for wheelchairs and their occupants still have certain shortcomings. These are due to the high cost, bulk, limited mobility, and limited viewing angles of traditional monitoring systems. Furthermore, traditional wheelchair monitoring systems can only capture images within a local area or in an area with installed monitoring equipment, making remote monitoring impossible. The patent "A Smart Internet of Things Electric Wheelchair Control System (202022422812.9)" proposes a remote monitoring method for smart wheelchairs, which mainly realizes remote monitoring through communication between the wheelchair user's smart device and the remote smart device, but its method has inherent disadvantages: First, the system requires the user to connect to the Internet, communicate, and perform other operations when in use, which is not friendly to the daily use of the elderly; second, it is impossible to implement personalized monitoring authority authentication for wheelchair occupants, and there is a privacy leakage problem to a certain extent; third, the wheelchair monitoring perspective is limited, and it is impossible to achieve all-round monitoring of the wheelchair and occupants. Summary of the Invention

[0004] To address the above-mentioned issues, the present invention discloses an all-round remote safety monitoring system for intelligent wheelchairs. The system is compact, low-cost, and easy to install. It uses a Raspberry Pi as the central controller, automatically initializes after startup, and can remotely obtain monitoring information on the wheelchair side without the need for wheelchair occupant operation. It can monitor the wheelchair's surrounding environment as well as the wheelchair and the occupant themselves. An original personalized system authority authentication method maximizes the data security of wheelchair monitoring.

[0005] To achieve the above object, the technical solution of the present invention is as follows:

[0006] An all-round remote safety monitoring system for an intelligent wheelchair includes a wheelchair body, which includes a wheelchair base, wheelchair armrests on both sides of the wheelchair base, and three parts of the wheelchair armrests, wherein a CSI camera is installed on the monitoring pan-tilt platform, a 5G communication module is installed under the wheelchair base, and is connected to the Raspberry Pi via a network cable, the Raspberry Pi is connected to the monitoring pan-tilt platform via a ribbon cable and a DuPont cable, and the Raspberry Pi is connected to the speaker via an audio cable. Support devices are provided on both sides of the wheelchair armrests.

[0007] As an improvement of the present invention, the holding device includes a clamping device mainboard, a clamping device side plate, a clamping device bottom plate, a limit plate, spring 1, and spring 2. The clamping device mainboard is provided with a positioning hole for fixing the device on the wheelchair; the clamping device mainboard is provided with a fixing hole for fixing the Raspberry Pi, the monitoring pan-tilt head and the speaker. A limit plate is provided on one side of the clamping device mainboard, and the other side of the clamping device mainboard is connected to the clamping device side plate through spring 1. The clamping device side plate is perpendicular to the space of the clamping device mainboard, and the bottom of the clamping device side plate is connected to the clamping device bottom plate through spring 2.

[0008] As an improvement of the present invention, the monitoring platform is composed of a two-degree-of-freedom steering gear.

[0009] As an improvement of the present invention, the CSI camera has 5 million pixels, a sensor pixel of 1080p, a focal length of 2.8mm, an OV5647 photosensitive chip, supports 1080p 30 frames and 720p 60 frames, and has a red status indicator light.

[0010] As an improvement to the present invention, the Raspberry Pi serves as the main controller, combined with a cloud server and independently designed app software to implement the functions of a wheelchair monitoring system. The Raspberry Pi is installed on the wheelchair side to process the surveillance footage captured by the camera and control the movement of the pan / tilt head (PTZ). The cloud server is deployed on the internet for remote transmission of image data. A wheelchair monitoring app is designed based on Android technology. The wheelchair user's guardian can then install the app to access the remote surveillance footage and issue PTZ control commands. The surveillance footage is displayed in the video receiving area of ​​the app's main interface. Any user action on the main interface, including clicking the remote connection button or the connect camera button, will prompt a toast dialog box to alert the user of the ongoing action, creating a visual human-computer interaction interface with feedback. (Software Implementation Method)

[0011] The beneficial effects of the present invention are:

[0012] (1) The intelligent wheelchair all-round remote safety monitoring system described in the present invention is small in size, low in cost, and easy to install. It uses Raspberry Pi as the central controller and automatically initializes after power-on. It can remotely obtain monitoring information on the wheelchair side without the need for wheelchair occupant operation; it can monitor the environment around the wheelchair, as well as the wheelchair and the occupant themselves; and the original personalized system authority authentication method maximizes the data security of wheelchair monitoring.

[0013] (2) The central controller uses a Raspberry Pi instead of a smartphone. After the Raspberry Pi is turned on, it automatically performs initialization according to the settings: initializing the posture parameters of the monitoring pan / tilt, opening Socket communication to wait for client connections, and uploading the video surveillance images to the web. Wheelchair users only need to turn on the Raspberry Pi and do not need to perform any other operations. This is very user-friendly for wheelchair users who do not know how to use or are not proficient in using smart devices.

[0014] (3) Operation information feedback: When the user clicks the connect camera button in the remote connection area of ​​the main interface, the APP pops up a toast dialog box to remind the user that the connection is in progress. At the same time, the system performs a second authorization authentication based on the original video security monitoring method. After successful authentication, the remote monitoring video screen can be obtained. This not only ensures the data security of the wheelchair occupant video monitoring and expands the user's travel area, but also makes it convenient for the monitor to remotely know the location and personal health of the wheelchair occupant. A better human-computer interaction experience, with an operation interface with connection information feedback, reduces the user's operation difficulty. Even users who are not familiar with smartphones can operate conveniently and quickly according to the prompts. The monitoring screen can be freely controlled. The user can pinch to zoom in and out of the video receiving area, zoom in on any part of the video monitoring area, and fully understand the situation of the wheelchair occupant.

[0015] (4) Two-step authentication of system permissions: After installing the wheelchair monitoring app and registering, users can log in to the monitoring system with their account and password. The registrant's information will be stored in the database. This is the first step in the authentication of the video security monitoring system. The original video security monitoring method guarantees the data security of video monitoring to the greatest extent possible. It includes a family information database and a voice broadcast system. Each mobile phone has a globally unique mobile equipment identification code (IMEI). Through the getIMEI() method developed in Android, the identification code of the mobile phone remotely connected to the Raspberry Pi is obtained. The mobile phone identification code of the family member is matched with the family member's name and stored in the family information database in advance; the wheelchair user can understand the family member who initiated the remote monitoring through voice broadcast, which improves the user experience; the mobile device identification code entered into the family information database is globally unique. Even for mobile phones of the same brand and model, the mobile device identification code is different. Visitors who are not entered into the family information database cannot obtain remote video images or send pan-tilt control commands, which greatly improves the privacy and data security of the wheelchair monitoring system; different users can set up personalized family information databases according to actual conditions, and apply the wheelchair monitoring system to more actual scenarios, without limiting the usage scenarios.

[0016] (5) The entire hardware structure is fixed by a clamping device. The clamping device has a stable structure, is easy to install and disassemble, and is ready for use. This greatly facilitates users to replace wheelchairs and disassemble and install the monitoring system. The device can be fixed by spring force or screws according to actual conditions, or both can be used to fix the device, ensuring the personal safety of wheelchair occupants to the greatest extent. Moreover, the structure is novel and lightweight, the design is ingenious, and the adaptability is strong. It has the characteristics of low cost, high stability and high flexibility, and can clamp the majority of wheelchair armrests. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart for comprehensive safety monitoring of wheelchairs;

[0018] Figure 2 Schematic diagram of the monitoring system installation location;

[0019] Figure 3 Schematic diagram of the clamping device structure;

[0020] Figure 4 The main interface of the Android application.

[0021] List of Figure Symbols:

[0022] F1 and F2 are wheelchair armrests, U3 is Raspberry Pi, U2 is the monitoring pan / tilt head and CSI camera, U1 is the speaker, D1 is the wheelchair base, J1 is the clamping device, S1 is the clamping device mainboard, S2 is the positioning hole, S3 is the fixing hole, S4 is spring 1, S5 is the clamping device side plate, S6 is spring 2, S7 is the clamping device bottom plate, and S8 is the limit plate. DETAILED DESCRIPTION

[0023] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0024] As shown in the figure, the intelligent wheelchair omnidirectional remote safety monitoring system of the present invention includes the following steps:

[0025] Step 1: Device connection and installation: Connect the CSI camera to the Raspberry Pi via a ribbon cable, connect the industrial-grade 5G communication module to the Raspberry Pi via a network cable, connect the monitoring pan / tilt head to the Raspberry Pi via a DuPont cable, and connect the speaker to the Raspberry Pi via an audio cable. Install the Raspberry Pi, speaker, and monitoring pan / tilt head on the clamping device, and clamp the device to the wheelchair armrest.

[0026] Step 2: Module self-test and system initialization: The camera and industrial-grade 5G communication module self-test, and the gimbal initializes posture parameters; the cloud server is opened, the FRP tool is run, the Raspberry Pi identifies and initializes the camera parameters, and uploads the video to the web; the Raspberry Pi establishes a socket connection and waits for client access;

[0027] Step 3: The user downloads the wheelchair remote monitoring APP and registers. The user registration information is saved in the database;

[0028] Step 4: The user logs in to the wheelchair monitoring APP with the account and password. The APP performs the first system permission verification on the logged-in user. After the verification is passed, the user can enter the main interface;

[0029] Step 5: The user clicks the Connect Camera button on the APP main interface. The APP then performs a second system permission authentication based on the customized video security monitoring method for the user who wants to obtain the monitoring image.

[0030] Step 6: After successful authentication, the video surveillance image is displayed in the video receiving area of ​​the main interface, and a dialog box pops up to prompt the user that the connection is successful; if the authentication is unsuccessful, the surveillance image cannot be obtained, the PTZ cannot be controlled, and the APP pops up a dialog box to prompt the user that the connection failed;

[0031] Step 7: After the user is successfully authenticated, the Raspberry Pi will announce the family member's nickname corresponding to the mobile phone identification code stored in the family member information database through the speaker.

[0032] Step 8: The user clicks the remote connection button on the main interface of the APP. Users who have passed the second system permission authentication can obtain remote control rights for the monitoring PTZ. The APP pops up a toast dialog box to prompt the user that the connection is successful. Users who have not passed the permission authentication cannot remotely control the PTZ. The APP pops up a toast dialog box to prompt the user that the connection failed.

[0033] Step 9: After the remote connection is successful, the user clicks the arrow in the PTZ control area on the main interface of the APP to trigger a click event and send the set hexadecimal command to the Raspberry Pi;

[0034] Step 10: After receiving the hexadecimal command, the Raspberry Pi parses it according to the agreed communication protocol and outputs PWM at a specific IO port according to the command to control the posture of the monitoring gimbal.

[0035] Step 11: If multiple users are successfully authenticated, they can obtain the monitoring screen at the same time, but the monitoring PTZ is locked and cannot be controlled by remote commands.

[0036] It should be noted that the above content merely illustrates the technical idea of ​​the present invention and cannot be used to limit the scope of protection of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications all fall within the scope of protection of the claims of the present invention.

Claims

1. An intelligent wheelchair omnidirectional remote safety monitoring system, comprising a wheelchair body, wherein the wheelchair body comprises a wheelchair base, and wheelchair armrests are arranged on both sides of the wheelchair base, characterized in that: The wheelchair armrest is provided with three parts: Raspberry Pi, monitoring pan-tilt and speaker, wherein a CSI camera is installed on the monitoring pan-tilt, an industrial-grade 5G communication module is installed under the wheelchair base, and is connected to the Raspberry Pi through a network cable, the Raspberry Pi is connected to the monitoring pan-tilt DuPont cable, and the Raspberry Pi is connected to the speaker through an audio cable, and clamping devices are provided on both sides of the wheelchair armrest; The clamping device comprises a clamping device mainboard, a clamping device side plate, a clamping device bottom plate, a limit plate, a spring 1, and a spring 2. A positioning hole is provided on the clamping device mainboard for fixing the clamping device on the wheelchair; a fixing hole is provided on the clamping device mainboard for fixing the Raspberry Pi, the monitoring pan / tilt and the speaker; a limit plate is provided on one side of the clamping device mainboard, and the other side of the clamping device mainboard is connected to the clamping device side plate through a spring 1; the clamping device side plate is perpendicular to the clamping device mainboard space, and the lower side of the clamping device side plate is connected to the clamping device bottom plate through a spring 2; The spring 1 and the spring 2 each have three; The method for using the intelligent wheelchair omnidirectional remote safety monitoring system comprises the following steps: Step 1: Equipment connection and installation: Connect the CSI camera to the Raspberry Pi via a ribbon cable, connect the industrial-grade 5G communication module to the Raspberry Pi network cable, connect the monitoring PTZ to the Raspberry Pi via a DuPont cable, and connect the speaker to the Raspberry Pi via an audio cable; install the Raspberry Pi, speaker, and monitoring PTZ on the clamping device, and clamp the clamping device on the wheelchair armrest; Step 2: Module self-check and system initialization: CSI camera and industrial-grade 5G communication module self-check, monitor the initialization of gimbal posture parameters; open the cloud server, run the FRP tool, Raspberry Pi recognizes and initializes the CSI camera parameters, and uploads the video to the web; Raspberry Pi creates a Socket connection and waits for client access; Step 3: The user downloads the wheelchair remote monitoring APP and registers. The user registration information is saved in the database; Step 4: The user logs in to the wheelchair remote monitoring APP with the account and password. The APP performs the first system permission verification on the logged-in user, and enters the main interface after the verification is passed; Step 5: The user clicks the CSI camera connection button on the main interface of the APP. The APP will perform a second system permission authentication for the user who wants to obtain the monitoring screen according to the customized video security monitoring method: the system will obtain and verify the IMEI identification code of the mobile phone, compare it with the independent family information database associated with the smart wheelchair, and perform a second system permission authentication; Step 6: After successful authentication, the video surveillance screen is displayed in the video receiving area of ​​the main interface, and a dialog box pops up to prompt the user that the connection is successful; if the authentication is unsuccessful, the surveillance screen cannot be obtained, the surveillance PTZ cannot be controlled, and the APP pops up a dialog box to prompt the user that the connection failed; Step 7: After the user is successfully authenticated, the Raspberry Pi will announce the family member’s nickname corresponding to the mobile phone identification code stored in the family member information database through the speaker; Step 8: The user clicks the remote connection button on the main interface of the APP. For users who have passed the second system permission authentication, they will obtain remote control rights for the monitoring PTZ. The APP will pop up a Toast dialog box to prompt the user that the connection is successful. Users who have not passed the second system permission authentication cannot remotely control the PTZ. The APP will pop up a Toast dialog box to prompt the user that the connection has failed. Step 9: After the remote connection is successful, the user clicks the arrow in the PTZ control area of ​​the APP main interface to trigger a click event and send the set hexadecimal command to the Raspberry Pi; Step 10: After receiving the hexadecimal command, Raspberry Pi parses it according to the agreed communication protocol and outputs PWM at a specific IO port according to the command to control the posture of the monitoring gimbal; Step 11: If multiple users are successfully authenticated, they can obtain the monitoring screen at the same time, but the monitoring PTZ is locked and cannot be controlled by remote commands.

2. The intelligent wheelchair omnidirectional remote safety monitoring system according to claim 1, characterized in that: The monitoring platform is composed of a two-degree-of-freedom steering gear.

3. The intelligent wheelchair omnidirectional remote safety monitoring system according to claim 1, characterized in that: The CSI camera has 5 megapixels and a sensor pixel of 1080p; the focal length is 2.8mm; the photosensitive chip is OV5647; it supports 1080p30 frames and 720p 60 frames; and it has a red status indicator light.

4. The intelligent wheelchair omnidirectional remote safety monitoring system according to claim 1, characterized in that: The Raspberry Pi is used as the main controller, combined with the cloud server and APP software to realize the functions of the wheelchair remote monitoring system; the Raspberry Pi is installed at the wheelchair end to process the monitoring picture obtained by the camera and control the movement of the pan-tilt head; the cloud server is arranged on the Internet to remotely transmit image data; the wheelchair remote monitoring APP is designed based on Android technology, and then the guardian of the wheelchair user installs the APP to obtain the remote monitoring picture and issue the pan-tilt head control command at the same time; the monitoring picture is displayed in the video receiving area of ​​the APP main interface, and any operation of the user on the main interface, including clicking the remote connection button or clicking the connect camera button, the APP will pop up a Toast dialog box to remind the user of the ongoing operation, thereby realizing a visual human-computer interaction interface with feedback.

Images