A walking assistance navigation method

Abstract

The application discloses a walking auxiliary navigation method and relates to the technical field of navigation for the visually impaired. The method comprises the following steps: acquiring image information by a camera according to first coordinate information; acquiring character recognition information and character gesture information according to the image information; acquiring an individual auxiliary navigation mode and destination information according to the task recognition information, the task gesture information and an auxiliary knowledge graph; generating a first auxiliary individual auxiliary navigation method according to the individual auxiliary navigation mode and the destination information; and sending the first auxiliary individual auxiliary navigation method to a walking auxiliary navigation system for the visually impaired in a nursing home, so that the walking auxiliary navigation system for the visually impaired in the nursing home performs auxiliary navigation according to the individual auxiliary navigation method. The application can enable the visually impaired to make individualized settings according to their own needs during walking to a destination in a corridor and select the most suitable guiding mode for use.

Description

Technical Field

[0001] This application relates to the field of navigation technology for visually impaired individuals, specifically to a walking assistance navigation method. Background Technology

[0002] Many nursing homes are unable to provide each elderly person with an independent room that is fully equipped with a kitchen, bathroom, and other facilities. Such nursing homes usually only provide one public bathroom per room, and they generally only have one medical station on a certain floor.

[0003] In this situation, the elderly residents of the nursing home have no choice but to leave their rooms and walk to the toilet or medical station when they need to go to the toilet or get medical supplies.

[0004] Most nursing homes are understaffed (especially at night), which means that not every time an elderly person needs to go to the restroom or medical station, there is a caregiver to accompany them. This means that if an elderly person in a nursing home has visual impairments (such as congenital blindness, glaucoma, or eye diseases caused by diabetes), and cannot see clearly, they may not be able to walk to the restroom or medical station on their own.

[0005] Existing technologies for assisting visually impaired people with walking generally rely on intelligent guidance devices (such as intelligent canes or head-mounted intelligent guidance devices) to guide blind people. The disadvantages of this method are that, on the one hand, there needs to be a certain amount of light in the surroundings, that is, it is generally not possible to carry out guidance work well in dark environments; on the other hand, the user must hold it himself, and if the user does not hold it, it cannot serve as a navigation tool. Summary of the Invention

[0006] The purpose of this invention is to provide a walking-assisted navigation method to at least solve one of the above-mentioned technical problems.

[0007] One aspect of the present invention provides a walking-assisted navigation method, the walking-assisted navigation method comprising:

[0008] Get room door opening information;

[0009] The first coordinate information is obtained based on the room door opening information, and the first coordinate information is sent to the camera devices in the nursing home that are capable of capturing the first coordinate information.

[0010] Acquire image information transmitted by a camera device capable of capturing the scene at the doorway of the room based on the first coordinate information;

[0011] First person recognition information and first person gesture information are obtained from one or more image information from each image information;

[0012] Obtain an auxiliary knowledge graph, which includes multiple character nodes, gesture nodes corresponding to each character node, personalized auxiliary navigation methods corresponding to each gesture node, and destination information corresponding to each gesture node.

[0013] Obtain each gesture node of the person node corresponding to the first person recognition information;

[0014] The similarity between the first person's gesture information and each of the gesture nodes is measured to obtain the personalized auxiliary navigation method and destination information corresponding to the gesture node with the highest similarity to the first person's gesture information among the various gesture nodes of the corresponding person node.

[0015] A first personalized auxiliary navigation method is generated based on the personalized auxiliary navigation method and destination information;

[0016] The first personalized auxiliary navigation method is sent to the walking auxiliary navigation system for visually impaired people in nursing homes, so that the walking auxiliary navigation system for visually impaired people in nursing homes can perform assisted navigation according to the first personalized auxiliary navigation method.

[0017] Optionally, obtaining room door opening information includes:

[0018] Acquire the door opening detection signal transmitted by the door opening detector and the identification information of the door opening detector.

[0019] Optionally, the step of obtaining the first coordinate information based on the room door opening information and sending the first coordinate information to the various camera devices in the nursing home, including the camera devices capable of capturing the first coordinate information, includes:

[0020] Obtain a coordinate database, which includes at least one preset door opening detector identifier and first coordinate information corresponding to each preset door opening detector identifier;

[0021] Obtain the first coordinate information corresponding to the preset door opening detector identifier that is the same as the identifier information of the door opening detector;

[0022] Obtain a preset camera device coordinate database, which includes at least one preset camera device identifier and a captureable coordinate table corresponding to each preset camera device identifier, and each captureable coordinate table includes at least one preset first coordinate information;

[0023] Obtain the preset camera device identifier corresponding to each camera coordinate table that has the same preset first coordinate information as the first coordinate information;

[0024] The first coordinate information is sent to each camera device corresponding to each preset camera device identifier based on the obtained preset camera device identifier.

[0025] Optionally, obtaining the first person recognition information and the first person gesture information based on one or more image information from the various image information includes:

[0026] Obtain a trained face neural network model;

[0027] Extract facial features from each image separately;

[0028] The facial features are input into the trained facial neural network model to obtain first person recognition information;

[0029] Obtain a preset gesture navigation database, which includes preset person recognition information and gesture navigation reminder information corresponding to each preset person recognition information;

[0030] Obtain gesture navigation reminder information corresponding to preset person recognition information that is identical to the first person recognition information;

[0031] The gesture navigation reminder information is sent to the corresponding reminder device, so that the reminder device performs a reminder operation based on the gesture navigation reminder information;

[0032] Obtain the trained gesture neural network model;

[0033] After the reminder operation is performed, the gesture features in each image transmitted by the camera device capable of capturing the first coordinate information are extracted respectively.

[0034] The gesture features are input into the trained gesture neural network model to obtain the gesture information of the first person.

[0035] Optionally, the personalized auxiliary navigation methods include light-guided navigation, voice-guided navigation, and hybrid navigation.

[0036] Optionally, the method for generating the first personalized auxiliary navigation based on the personalized auxiliary navigation method and destination information includes:

[0037] A first planned walking trajectory is generated based on the destination information and the first coordinate information. The first planned walking trajectory includes multiple necessary coordinate information between the destination information and the first coordinate information.

[0038] Obtain a pre-stored database of a walking assistance navigation system for visually impaired people in nursing homes. The pre-stored database includes the identifiers of each assistance device and the preset assistance coordinate information corresponding to each assistance device identifier.

[0039] Obtain the identifier of each auxiliary device corresponding to the preset auxiliary coordinate information that is the same as at least one of the multiple required coordinate information;

[0040] A first personalized auxiliary navigation method is generated based on the auxiliary device identifier and the personalized auxiliary navigation method.

[0041] Optionally, after sending the first personalized assisted navigation method to the walking assistance navigation system for visually impaired people in nursing homes, the walking assistance navigation method further includes:

[0042] Determine whether new room door opening information has been obtained within a preset time period; if so, then...

[0043] Obtain the second coordinate information based on the new room door opening information;

[0044] Determine whether the first coordinate information is the same as the second coordinate information; if not, then...

[0045] The second coordinate information is sent to the cameras in the nursing home that are capable of capturing the second coordinate information.

[0046] Acquire new image information transmitted by a camera device capable of acquiring a new view of the room entrance based on the second coordinate information;

[0047] Obtain second person recognition information and second person gesture information based on one or more of the new image information;

[0048] The auxiliary knowledge graph is obtained, which includes multiple character nodes, gesture nodes corresponding to each character node, personalized auxiliary navigation methods corresponding to each gesture node, and destination information corresponding to each gesture node.

[0049] Obtain each gesture node of the person node corresponding to the second person recognition information;

[0050] The similarity measurement between the second person's gesture information and each gesture node is performed to obtain the personalized auxiliary navigation method and destination information corresponding to the gesture node with the highest similarity to the second person's gesture information among the various gesture nodes of the corresponding person node. The personalized auxiliary navigation method obtained through the second person's gesture information is called the second personalized auxiliary navigation method, and the destination information obtained through the second person's gesture information is called the second destination information.

[0051] A second personalized auxiliary navigation method is generated based on the second personalized auxiliary navigation method and the second destination information;

[0052] A second planned walking trajectory is generated based on the second destination information and the second coordinate information. The second planned walking trajectory includes multiple necessary coordinate information between the destination information and the second coordinate information.

[0053] Determine whether the second planned walking trajectory and the first planned walking trajectory share common necessary coordinate information. If so, then...

[0054] Determine whether the personalized auxiliary navigation method obtained through the first person's gesture information conflicts with the second personalized auxiliary navigation method. If not, then...

[0055] The second personalized auxiliary navigation method is sent to the walking assistance navigation system for visually impaired people in nursing homes, so that the walking assistance navigation system for visually impaired people in nursing homes can perform assisted navigation according to the second personalized auxiliary navigation method.

[0056] Optionally, after sending the personalized assisted navigation method to the walking assisted navigation system for visually impaired people in nursing homes, the walking assisted navigation method further includes:

[0057] Determine whether the personalized auxiliary navigation method conflicts with the second personalized auxiliary navigation method; if so, then...

[0058] The system uses various cameras in the walking assistance navigation system for visually impaired people in nursing homes to obtain the floor information of the pedestrian corresponding to the first person identification information and the floor information of the pedestrian corresponding to the second person identification information.

[0059] If the floor information of the pedestrian corresponding to the first person identification information is not the same floor as the floor information of the pedestrian corresponding to the second person identification information, then

[0060] The system for assisting visually impaired people in nursing homes uses the first personalized navigation method to provide navigation assistance based on the floor information of the pedestrian corresponding to the first person identification information in the walking assistance navigation system, and generates second person identification information navigation information to notify the pedestrian corresponding to the first person identification information through voice broadcast.

[0061] The system for assisting visually impaired people in nursing homes uses the second personalized navigation method to provide assisted navigation based on the floor information of the pedestrian corresponding to the second person identification information in the walking assistance navigation system. The system generates navigation information based on the first person identification information and notifies the pedestrian corresponding to the second person identification information via voice broadcast.

[0062] Optionally, after using the second personalized navigation method to assist navigation based on the floor information of the pedestrian corresponding to the second person recognition information in the walking assistance navigation system for visually impaired people in the nursing home, the walking assistance navigation method further includes:

[0063] The system monitors the location of the pedestrian corresponding to the first person identification information and the location of the pedestrian corresponding to the second person identification information in real time. When the locations of the pedestrians corresponding to the first person identification information and the second person identification information are on the same floor, the second auxiliary personalized navigation method is fixed as voice guidance navigation.

[0064] This application also provides a walking assistance navigation system for visually impaired people in nursing homes, the walking assistance navigation system for visually impaired people in nursing homes comprising:

[0065] The facility includes multiple camera devices, each installed at a specific first location within the nursing home. These first locations include each corridor and elevator. Each camera device comprises multiple rotatable and multiple non-rotatable cameras. The rotatable cameras rotate to capture images based on first and second coordinate information, while the non-rotatable cameras capture images from a fixed position. Each corridor contains at least one rotatable camera and at least one non-rotatable camera, with the non-rotatable camera in each corridor capable of covering the entire shooting area within that corridor.

[0066] A voice broadcasting device, wherein there are multiple voice broadcasting devices, and each camera device is respectively installed in a second location in the nursing home, the first location including each corridor and elevator of the nursing home;

[0067] Night lights, there are multiple night lights, each night light is set in a third location in the nursing home, the third location includes each corridor, elevator, toilet door and function room door of the nursing home. When different night lights are lit at the same time, they can form different light trajectories, and different light trajectories can provide different light trajectory guidance.

[0068] The main controller is connected to each of the night lights, each of the camera devices, and each of the voice broadcasting devices, and is used to execute the walking-assisted navigation method described above. Beneficial effects

[0069] The walking-assisted navigation method of this application treats the entire building as an auxiliary object, distributing all auxiliary devices (such as camera devices, voice broadcasting devices, and indicator lights) in every corner of the building. Through the coordinated work of various auxiliary devices, visually impaired people can choose the most suitable guidance method according to their own personalized settings as they walk from one staircase to their destination. This solves the problems in the prior art where all auxiliary devices can only be used by one person, and where all auxiliary devices in the prior art need to be carried or worn by the user. It also realizes artificial intelligence navigation for visually impaired people throughout the entire building. Attached Figure Description

[0070] Figure 1 This is a schematic flowchart of a walking-assisted navigation method according to an embodiment of this application.

[0071] Figure 2 It is used to implement Figure 1 A schematic diagram of an electronic device for a walking-assisted navigation method is shown.

[0072] Figure 3 This is a schematic diagram of the ground night light arrangement according to an embodiment of this application. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be described in more detail below with reference to the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The described embodiments are some, but not all, embodiments of this application. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0074] Figure 1 This is a schematic flowchart of a walking-assisted navigation method according to an embodiment of this application.

[0075] like Figure 1 The walking-assisted navigation methods shown include:

[0076] Step 1: Obtain room door opening information;

[0077] Step 2: Obtain the first coordinate information based on the room door opening information, and send the first coordinate information to the camera devices in the nursing home that can capture the first coordinate information;

[0078] Step 3: Obtain image information transmitted by a camera device capable of capturing the scene at the room entrance based on the first coordinate information;

[0079] Step 4: Obtain the first person recognition information and the first person gesture information based on one or more image information from each image information;

[0080] Step 5: Obtain the auxiliary knowledge graph, which includes multiple character nodes, gesture nodes corresponding to each character node, personalized auxiliary navigation methods corresponding to each gesture node, and destination information corresponding to each gesture node;

[0081] Step 6: Obtain each gesture node of the person node corresponding to the first person recognition information;

[0082] Step 7: Perform a similarity measurement between the first person's gesture information and each of the gesture nodes to obtain the personalized auxiliary navigation method and destination information corresponding to the gesture node with the highest similarity to the first person's gesture information among the various gesture nodes of the corresponding person node.

[0083] Step 8: Generate a first personalized auxiliary navigation method based on the personalized auxiliary navigation method and destination information;

[0084] Step 9: Send the first personalized auxiliary navigation method to the walking assistance navigation system for visually impaired people in nursing homes, so that the walking assistance navigation system for visually impaired people in nursing homes can perform assisted navigation according to the first personalized auxiliary navigation method.

[0085] The walking-assisted navigation method of this application treats the entire building as an auxiliary object, distributing all auxiliary devices (such as camera devices, voice broadcasting devices, and indicator lights) in every corner of the building. Through the coordinated work of various auxiliary devices, visually impaired people can choose the most suitable guidance method according to their own personalized settings as they walk from one staircase to their destination. This solves the problems in the prior art where all auxiliary devices can only be used by one person, and where all auxiliary devices in the prior art need to be carried or worn by the user. It also realizes artificial intelligence navigation for visually impaired people throughout the entire building.

[0086] In this embodiment, the coordinate information of this application can be generated by SLAM technology (simultaneous localization and mapping), that is, the robot first runs through the entire building to create a virtual map of the building and provide coordinate information for each location.

[0087] In this embodiment, obtaining room door opening information includes:

[0088] Acquire the door opening detection signal transmitted by the door opening detector and the identification information of the door opening detector.

[0089] For example, a door opening detector includes an accelerometer and a door opening controller. After the accelerometer detects that the door has been opened, the door opening controller generates a door opening detection signal and sends the door opening detection signal and the identification information of the door opening detector to the main controller.

[0090] In this embodiment, the step of obtaining the first coordinate information based on the room door opening information and sending the first coordinate information to the various camera devices in the nursing home that are capable of capturing the first coordinate information includes:

[0091] Obtain a coordinate database, which includes at least one preset door opening detector identifier and first coordinate information corresponding to each preset door opening detector identifier;

[0092] Obtain the first coordinate information corresponding to the preset door opening detector identifier that is the same as the identifier information of the door opening detector;

[0093] Obtain a preset camera device coordinate database, which includes at least one preset camera device identifier and a captureable coordinate table corresponding to each preset camera device identifier, and each captureable coordinate table includes at least one preset first coordinate information;

[0094] Obtain the preset camera device identifier corresponding to each camera coordinate table that has the same preset first coordinate information as the first coordinate information;

[0095] The first coordinate information is sent to each camera device corresponding to each preset camera device identifier based on the obtained preset camera device identifier.

[0096] In this embodiment, a coordinate position is set for a portion of the area outside each room. It can be understood that the coordinate position can be a coordinate point or a coordinate area.

[0097] When the camera device obtains the coordinate position, it can be rotated using the built-in gimbal to align the camera's shooting area with the coordinate position.

[0098] It is understandable that each time the camera device aligns with the coordinate position, it can be seen as the process of the camera device finding the target point.

[0099] In this embodiment, obtaining the first person recognition information and the first person gesture information based on one or more image information from various image information includes:

[0100] Obtain a trained face neural network model;

[0101] Extract facial features from each image separately;

[0102] The facial features are input into the trained facial neural network model to obtain first person recognition information;

[0103] Obtain a preset gesture navigation database, which includes preset person recognition information and gesture navigation reminder information corresponding to each preset person recognition information;

[0104] Obtain gesture navigation reminder information corresponding to preset person recognition information that is identical to the first person recognition information;

[0105] The gesture navigation reminder information is sent to the corresponding reminder device, so that the reminder device performs a reminder operation based on the gesture navigation reminder information;

[0106] Obtain the trained gesture neural network model;

[0107] After the reminder operation is performed, the gesture features in each image transmitted by the camera device capable of capturing the first coordinate information are extracted respectively.

[0108] The gesture features are input into the trained gesture neural network model to obtain the gesture information of the first person.

[0109] In this embodiment, the personalized auxiliary navigation methods include light-guided navigation, voice-guided navigation, and hybrid navigation.

[0110] In this embodiment, the method for generating the first personalized auxiliary navigation based on the personalized auxiliary navigation method and destination information includes:

[0111] A first planned walking trajectory is generated based on the destination information and the first coordinate information. The first planned walking trajectory includes multiple necessary coordinate information between the destination information and the first coordinate information.

[0112] Obtain a pre-stored database of a walking assistance navigation system for visually impaired people in nursing homes. The pre-stored database includes the identifiers of each assistance device and the preset assistance coordinate information corresponding to each assistance device identifier.

[0113] Obtain the identifier of each auxiliary device corresponding to the preset auxiliary coordinate information that is the same as at least one of the multiple required coordinate information;

[0114] A first personalized auxiliary navigation method is generated based on the auxiliary device identifier and the personalized auxiliary navigation method.

[0115] In this embodiment, after sending the first personalized assisted navigation method to the walking assisted navigation system for visually impaired people in nursing homes, the walking assisted navigation method further includes:

[0116] Determine whether new room door opening information has been obtained within a preset time period; if so, then...

[0117] Obtain the second coordinate information based on the new room door opening information;

[0118] Determine whether the first coordinate information is the same as the second coordinate information; if not, then...

[0119] The second coordinate information is sent to the cameras in the nursing home that are capable of capturing the second coordinate information.

[0120] Acquire new image information transmitted by a camera device capable of acquiring a new view of the room entrance based on the second coordinate information;

[0121] Obtain second person recognition information and second person gesture information based on one or more of the new image information;

[0122] The auxiliary knowledge graph is obtained, which includes multiple character nodes, gesture nodes corresponding to each character node, personalized auxiliary navigation methods corresponding to each gesture node, and destination information corresponding to each gesture node.

[0123] Obtain each gesture node of the person node corresponding to the second person recognition information;

[0124] The similarity measurement between the second person's gesture information and each gesture node is performed to obtain the personalized auxiliary navigation method and destination information corresponding to the gesture node with the highest similarity to the second person's gesture information among the various gesture nodes of the corresponding person node. The personalized auxiliary navigation method obtained through the second person's gesture information is called the second personalized auxiliary navigation method, and the destination information obtained through the second person's gesture information is called the second destination information.

[0125] A second personalized auxiliary navigation method is generated based on the second personalized auxiliary navigation method and the second destination information;

[0126] A second planned walking trajectory is generated based on the second destination information and the second coordinate information. The second planned walking trajectory includes multiple necessary coordinate information between the destination information and the second coordinate information.

[0127] Determine whether the second planned walking trajectory and the first planned walking trajectory share common necessary coordinate information. If so, then...

[0128] Determine whether the personalized auxiliary navigation method obtained through the first person's gesture information conflicts with the second personalized auxiliary navigation method. If not, then...

[0129] The second personalized auxiliary navigation method is sent to the walking assistance navigation system for visually impaired people in nursing homes, so that the walking assistance navigation system for visually impaired people in nursing homes can perform assisted navigation according to the second personalized auxiliary navigation method.

[0130] In this embodiment, after sending the personalized assisted navigation method to the walking assisted navigation system for visually impaired people in nursing homes, the walking assisted navigation method further includes:

[0131] Determine whether the personalized auxiliary navigation method conflicts with the second personalized auxiliary navigation method; if so, then...

[0132] The system uses various cameras in the walking assistance navigation system for visually impaired people in nursing homes to obtain the floor information of the pedestrian corresponding to the first person identification information and the floor information of the pedestrian corresponding to the second person identification information.

[0133] If the floor information of the pedestrian corresponding to the first person identification information is not the same floor as the floor information of the pedestrian corresponding to the second person identification information, then

[0134] The system for assisting visually impaired people in nursing homes uses the first personalized navigation method to provide navigation assistance based on the floor information of the pedestrian corresponding to the first person identification information in the walking assistance navigation system, and generates second person identification information navigation information to notify the pedestrian corresponding to the first person identification information through voice broadcast.

[0135] The system for assisting visually impaired people in nursing homes uses the second personalized navigation method to provide assisted navigation based on the floor information of the pedestrian corresponding to the second person identification information in the walking assistance navigation system. The system generates navigation information based on the first person identification information and notifies the pedestrian corresponding to the second person identification information via voice broadcast.

[0136] In this embodiment, after using the second personalized auxiliary navigation method to assist navigation based on the floor information of the pedestrian corresponding to the second person recognition information in the walking assistance navigation system for visually impaired people in the nursing home, the walking assistance navigation method further includes:

[0137] The system monitors the location of the pedestrian corresponding to the first person identification information and the location of the pedestrian corresponding to the second person identification information in real time. When the locations of the pedestrians corresponding to the first person identification information and the second person identification information are on the same floor, the second auxiliary personalized navigation method is fixed as voice guidance navigation.

[0138] This application also provides a walking assistance navigation system for visually impaired people in nursing homes, which includes a camera device, a voice broadcasting device, a night light, and a main controller.

[0139] There are multiple camera devices, each installed at a specific location within the nursing home. These locations include each corridor and elevator. Each camera device comprises multiple rotatable and multiple non-rotatable cameras. The rotatable cameras rotate to capture images based on first and second coordinate information, while the non-rotatable cameras capture images from a fixed position. Each corridor contains at least one rotatable and at least one non-rotatable camera, with the non-rotatable cameras covering the entire shooting area within the corridor.

[0140] There are multiple voice broadcasting devices, and each camera device is set up in a second location in the nursing home. The first location includes each corridor and elevator in the nursing home.

[0141] There are multiple night lights, each of which is set in a third location in the nursing home. The third location includes each corridor, elevator, toilet door, and door of functional room in the nursing home. When different night lights are lit at the same time, they can form different light trajectories, and different light trajectories can provide different light trajectory guidance.

[0142] The main controller is connected to each of the aforementioned night lights, each of the respective camera devices, and each of the respective voice broadcasting devices. The main controller is used to execute the walking-assisted navigation method described above.

[0143] The following examples illustrate this application in detail. It is understood that these examples do not constitute any limitation on this application.

[0144] Background information:

[0145] The building for the elderly is a multi-story building, for example, a 6-story building with 10 rooms and 1 bathroom on each floor. The first floor has a medical room (it is understood that there may be other rooms or facilities on each floor, but since other rooms and facilities are not within the scope of this embodiment, they will not be described in detail here).

[0146] In this embodiment, multiple camera devices are installed in each floor corridor. Each camera device is used to capture part of the scene in the corridor, and each camera device can achieve full coverage of the entire corridor without blind spots.

[0147] In this embodiment, the staircases between each corridor will also be monitored by corresponding camera devices.

[0148] In this embodiment, some of the camera devices are rotatable, allowing them to rotate to different directions as needed. The combination of these cameras can achieve coverage of every doorway. In a more extreme approach, one camera device is installed opposite each doorway.

[0149] In this embodiment, the remaining camera devices are non-rotatable camera devices. These camera devices are generally set in positions that can observe the situation in the corridor to the greatest extent, and are mainly used to monitor the situation in the corridor.

[0150] In this embodiment, a voice broadcasting device, such as a loudspeaker, is installed in the corridor. In one embodiment, the voice broadcasting device of this application is installed on the wall next to the door of each room. By installing a voice broadcasting device on the wall next to the door of each room, the volume of the voice broadcasting device can be adjusted to be as low as possible, but it can still allow pedestrians to use the voice broadcasting device next to them for voice navigation.

[0151] In this embodiment, multiple night lights are installed in the corridor of this application. The night lights can be signal lights, flashing signal lights, signal lights that change color at any time, or constantly lit signal lights.

[0152] In this embodiment, nightlights can be distributed throughout the walls of the corridor, with one nightlight installed every few centimeters along the length of the corridor, so that the entire nightlight system can clearly indicate the direction of travel when illuminated. It is understood that the nightlights themselves can also be in the form of light strings.

[0153] In this embodiment, the arrangement of the nightlights allows them to clearly guide the path when they are bright. For example, assuming room A is located at one end of a corridor and room B is located at the other end, at least several nightlights are installed on the wall between room A and room B. When all these nightlights are lit (this lighting can be simultaneous or sequential, starting from one end and moving towards the other, for example, three nightlights: the first nightlight, the second nightlight, and the third nightlight; they can be lit simultaneously or sequentially, with the first nightlight lit first, the second and third nightlights off, after a preset time interval, such as 1 second, the second nightlight lit, the first and third nightlights off, and after another 1 second, the third nightlight lit, while the first and second nightlights are off), this method can create a relatively clear light path, allowing pedestrians to follow the light path.

[0154] In one embodiment, nightlights can be installed not only on the wall but also embedded in the ground, so that when the nightlights on the ground and the nightlights on the wall are lit at the same time, the user who observes the nightlights can have a more three-dimensional sense of space.

[0155] See Figure 3 , Figure 3 The circles in the diagram represent nightlights. In this embodiment, the nightlights set on the ground are located in the center of the ground. For example, the ground can be regarded as a rectangle with two long sides and two short sides. The nightlights set on the rectangle are evenly distributed from one end of the long side of the rectangle to the other end. This distribution can be one every 10 centimeters or other spacing can be used. The position of each nightlight in the short side direction is located at the center point of the short side direction.

[0156] This method can be used to guide the route and also allows users to understand the location of the center line of the corridor in dim or unlit environments.

[0157] Understandably, each of the aforementioned nightlights, cameras, and voice broadcasting devices has its own identification mark.

[0158] In this embodiment, the main controller of this application is connected to each night light, each camera device, and each voice broadcasting device on each floor.

[0159] The main controller can control the on / off state of each night light, control each camera device and acquire the images captured by each camera device, and control each voice broadcast device.

[0160] In this embodiment, when assisted navigation is not required, all voice broadcasting devices, nightlights, and movable cameras in the camera system can be in sleep mode to save power. These devices will only be woken up when the main controller obtains the room door opening information. It can be understood that the devices that are woken up are those that need to be woken up based on the room door opening information. For example, if a user is on the second floor and wants to walk from his room to the bathroom on the second floor, it is not necessary to wake up the various devices on the third floor (such as camera devices, voice broadcasting devices, etc.).

[0161] The reason this application can employ light-guided navigation, voice-guided navigation, and mixed-guided navigation is that not everyone with visual impairments is unable to see light. For example, visual impairment is generally classified into five levels:

[0162] The categories are: low vision level 1 (0.1 ≤ corrected visual acuity of the best eye < 0.3), low vision level 2 (0.05 ≤ corrected visual acuity of the best eye < 0.1), blindness level 1 (0.02 ≤ corrected visual acuity of the best eye < 0.05), blindness level 2 (light perception ≤ corrected visual acuity of the best eye < 0.02), and blindness level 3 (no light perception in both eyes).

[0163] As can be seen from the above description, although some visually impaired people cannot see specific objects clearly, they can have light perception, that is, they can sense light sources. In this case, they can be guided by light.

[0164] In practical use, the method of this application is as follows:

[0165] Scene: Nighttime, the hallway is dimly lit or almost completely dark.

[0166] Step 1: Obtain room opening information. In this embodiment, assuming that one of the rooms on the second floor (referred to as room A) is opened, the room A identifier is obtained. In this application, each room is equipped with a door opening detector, which includes an accelerometer and a door opening controller. The accelerometer can determine whether the door opening action has been performed, and each door opening controller has a room identifier pre-stored in the room.

[0167] The first coordinate information is obtained based on the room door opening information, and then sent to the various camera devices in the nursing home that are capable of capturing the first coordinate information. Specifically, this application sets the coordinate data of the doorway of each room, and each camera device is also set with a camera device identification identifier. Furthermore, this application sets a preset camera device coordinate database, which includes at least one preset camera device identifier and a table of possible camera coordinates corresponding to each preset camera device identifier. Each table of possible camera coordinates includes at least one preset first coordinate information. For example, suppose... Room A is labeled A. A query in the coordinate database reveals that the first coordinate information corresponding to A is (X1, Y1, Z1). A search in the preset camera device coordinate database reveals that the camera device labeled B has (X1, Y1, Z1) as its corresponding image capture coordinate. Therefore, it is assumed that the camera device labeled B can capture an image of the location with the first coordinate information. The first coordinate information is then sent to the camera device in the nursing home that can capture the first coordinate information, i.e., (X1, Y1, Z1) is sent to the camera device labeled B.

[0168] Understandably, each floor's camera system, rooms, nightlights, and voice broadcasting system are set up independently. In other words, when the main controller obtains the room identifier of room A, it will know that room A is located on the second floor. Therefore, the camera that can capture the first coordinate information will not send the first coordinate information to the camera devices on other floors.

[0169] Using this method, each floor can be divided by an identifier. When establishing the coordinate system, the coordinate system of each floor can be the same. For example, if the coordinate information of room A on the second floor is (X1, Y1, Z1), the coordinate information of a room on the third floor can also be (X1, Y1, Z1). The difference between the two is that the room identifier allows the controller to know the floor where the room is located, so that the coordinates of room A on the second floor will not be sent to the corresponding device on the third floor.

[0170] It is understood that in other embodiments, different coordinate systems can be set for each floor to distinguish different locations on different floors.

[0171] After sending the coordinate information to the camera device that can capture the coordinate information, the image information transmitted by the camera device that can capture the scene at the door of the room based on the first coordinate information will be obtained.

[0172] First person recognition information and first person gesture information are obtained from one or more image information from each image information.

[0173] Specifically, obtaining the first person recognition information and the first person gesture information based on one or more image information from various image information includes:

[0174] Obtain a trained face neural network model;

[0175] Extract facial features from each image separately;

[0176] The facial features are input into the trained facial neural network model to obtain first person recognition information;

[0177] Obtain a preset gesture navigation database, which includes preset person recognition information and gesture navigation reminder information corresponding to each preset person recognition information;

[0178] Obtain gesture navigation reminder information corresponding to preset character recognition information that is identical to the character recognition information;

[0179] The gesture navigation reminder information is sent to the corresponding reminder device, so that the reminder device performs a reminder operation based on the gesture navigation reminder information;

[0180] Obtain the trained gesture neural network model;

[0181] After the reminder operation is performed, the gesture features in each image transmitted by the camera device capable of capturing the first coordinate information are extracted respectively.

[0182] The gesture features are input into the trained gesture neural network model to obtain the gesture information of the first person.

[0183] In this embodiment, face recognition can employ ResNet50 convolutional neural network and Transformer model with multi-head self-attention mechanism. The specific face recognition method is a conventional technique in this field and will not be described in detail here.

[0184] In this embodiment, gesture recognition can also use a convolutional neural network. For example, when a user extends their hand and extends one finger, it represents one meaning, and extending two fingers represents another meaning.

[0185] In this embodiment, the present application can determine whether a visually impaired person has come out of the house by means of image recognition, and obtain whether the visually impaired person needs navigation and the specific destination of navigation by the gesture operation of the visually impaired person.

[0186] Since visually impaired individuals cannot clearly distinguish whether a camera is pointed at them, or do not know the appropriate way to perform gesture operations, the gesture navigation reminder information is sent to the corresponding reminder device, so that the reminder device can perform reminder operations based on the gesture navigation reminder information.

[0187] In this embodiment, the reminder device can be a voice broadcast device.

[0188] For example, when a visually impaired person (e.g., Zhang San) comes out of room A, the image captured by the camera device identifies Zhang San as being near the doorway of room A (i.e., near the first coordinate information and visible to the camera device). At this point, a gesture navigation reminder is broadcast via a voice broadcast device (e.g., "Please raise your hand and give a gesture command"). It is understood that the specific language information of the gesture navigation reminder is preset, so everyone using this assisted navigation method knows exactly how to raise their hand to perform the gesture operation.

[0189] Once the gesture and facial image are obtained, an auxiliary knowledge graph is acquired. The auxiliary knowledge graph includes multiple person nodes, gesture nodes corresponding to each person node, personalized auxiliary navigation methods corresponding to each gesture node, and destination information corresponding to each gesture node.

[0190] Obtain each gesture node of the person node corresponding to the first person recognition information. For example, if one of the person nodes represents Zhang San, then the person node corresponds to the first person recognition information. In this embodiment, the first person recognition information is obtained based on a face neural network model. The model can output a classification identifier, and the person node can also be set to a preset classification identifier. When the output identifier is the same as a preset classification identifier, it is considered to correspond.

[0191] The similarity between the first person's gesture information and each of the gesture nodes is measured to obtain the personalized auxiliary navigation method and destination information corresponding to the gesture node with the highest similarity to the first person's gesture information among the various gesture nodes of the corresponding person node.

[0192] In this way, the first person identification information (i.e., Zhang San) and the first person gesture information (e.g., Zhang San extending one finger) can be obtained.

[0193] For example, Zhang San's personalized auxiliary navigation method is light-guided navigation, where extending one finger indicates going to the medical room on the first floor.

[0194] A first auxiliary personalized navigation method is generated based on the personalized auxiliary navigation method and destination information; specifically, a first planned walking trajectory is generated based on destination information and first coordinate information, the first planned walking trajectory including multiple necessary coordinate information between destination information and first coordinate information;

[0195] Obtain a pre-stored database of a walking assistance navigation system for visually impaired people in nursing homes. The pre-stored database includes the identifiers of each assistance device and the preset assistance coordinate information corresponding to each assistance device identifier.

[0196] Obtain the identifier of each auxiliary device corresponding to the preset auxiliary coordinate information that is the same as at least one of the multiple required coordinate information;

[0197] A personalized auxiliary navigation method is generated based on the auxiliary device identifier and the personalized auxiliary navigation mode.

[0198] For example, one implementation method is as follows:

[0199] By comparing the building's BIM model with its point cloud model in two directions, the effective activity range and the path connection network within the effective activity range are determined, and an effective navigation image is generated.

[0200] Based on the task data, the starting point and destination of the pedestrian are determined on the BIM model. Based on the effective navigation image, the first planned walking trajectory of the robot is determined and multiple coordinate points on the first planned walking trajectory are obtained as necessary coordinate information. It can be understood that the selection of coordinate points can be arbitrary. For example, every few coordinate points can be used as necessary coordinate information. In addition, these necessary coordinate points can all be points on the first planned walking trajectory.

[0201] By using the pre-stored database of the walking assistance navigation system for visually impaired people in nursing homes, we can obtain the identification of auxiliary devices that can be used at these essential coordinate points. It is understood that the identification of auxiliary devices obtained at this time includes voice broadcasting devices and night lights.

[0202] At this time, a personalized auxiliary navigation method is generated based on the auxiliary device identifier and the personalized auxiliary navigation method. In this embodiment, since Zhang San's personalized auxiliary navigation method is light guidance navigation, only the auxiliary device identifier belonging to the night light among the above auxiliary device identifiers is obtained.

[0203] By controlling the illumination of these nightlights based on the acquired auxiliary device markings, an effective navigation route can be formed, thus navigating for Zhang San.

[0204] It is understood that when the personalized assisted navigation method is light-guided navigation, the walking assisted navigation method of this application further includes:

[0205] Obtain the light brightness at multiple preset locations on the current floor;

[0206] Determine whether the light brightness at each preset location on the current floor exceeds the preset light brightness threshold. If so, then...

[0207] Generate navigation voice and send it to the voice broadcasting device;

[0208] Acquire image information transmitted by a camera device that can acquire the scene at the door of the room based on the first coordinate information within a preset time after the navigation voice is sent to the voice broadcasting device;

[0209] Image recognition is performed on each acquired image to obtain new first-person gesture information. It is then determined whether this new first-person gesture information differs from previously acquired first-person gesture information. If so, then...

[0210] Similarity measurement is performed between the first person's gesture information and each of the gesture nodes.

[0211] In this embodiment, using light guidance navigation alone is more suitable for relatively dim scenes, such as scenes where the lights are not turned on at night. In such scenes, using light guidance navigation can play a good guiding role on the one hand, and reduce voice broadcasts on the other hand, which can prevent disturbing other people.

[0212] However, in some cases, users may forget that it is nighttime and the lighting is dim, or the hallway lighting may be sufficient, making it unsuitable to use light-guided navigation. In such cases, the above method can be used to guide the user to switch to other navigation methods.

[0213] It is understandable that in another embodiment, the navigation can be automatically switched to voice guidance without navigating the user, provided that the lighting is bright enough.

[0214] It is understood that in this embodiment, when it comes to changing floors, the destination of the starting floor can be planned to the elevator lobby during trajectory planning, and the user can enter other floors through the elevator. It is understood that there are elevators specifically for visually impaired people, which will not be elaborated here.

[0215] In certain special circumstances, it may be possible for two visually impaired individuals to use the method described in this application simultaneously. For example, in one specific case, Zhang San, located on the third floor, leaves his room at 9:50 AM and wishes to go to the medical room on the first floor. At 9:55 AM, Li Si, also located on the third floor, also leaves his room and may want to go to the restroom on the third floor. In this situation, this application can provide navigation using the following method:

[0216] Determine whether new room door opening information has been obtained within a preset time period; if so, then...

[0217] Obtain the second coordinate information based on the new room door opening information;

[0218] Determine whether the first coordinate information is the same as the second coordinate information; if not, then...

[0219] The second coordinate information is sent to the cameras in the nursing home that are capable of capturing the second coordinate information.

[0220] Acquire new image information transmitted by a camera device capable of acquiring a new view of the room entrance based on the second coordinate information;

[0221] Obtain second person recognition information and second person gesture information based on one or more of the new image information;

[0222] The auxiliary knowledge graph is obtained, which includes multiple character nodes, gesture nodes corresponding to each character node, personalized auxiliary navigation methods corresponding to each gesture node, and destination information corresponding to each gesture node.

[0223] Obtain each gesture node of the person node corresponding to the second person recognition information;

[0224] The similarity measurement between the second person's gesture information and each gesture node is performed to obtain the personalized auxiliary navigation method and destination information corresponding to the gesture node with the highest similarity to the second person's gesture information among the various gesture nodes of the corresponding person node. The personalized auxiliary navigation method obtained through the second person's gesture information is called the second personalized auxiliary navigation method, and the destination information obtained through the second person's gesture information is called the second destination information.

[0225] A second personalized auxiliary navigation method is generated based on the second personalized auxiliary navigation method and the second destination information;

[0226] A second planned walking trajectory is generated based on the second destination information and the second coordinate information. The second planned walking trajectory includes multiple necessary coordinate information between the destination information and the second coordinate information.

[0227] Determine whether the second planned walking trajectory and the first planned walking trajectory share common necessary coordinate information. If so, then...

[0228] Determine whether the personalized auxiliary navigation method conflicts with the second personalized auxiliary navigation method; if not, then...

[0229] The second personalized auxiliary navigation method is sent to the walking assistance navigation system for visually impaired people in nursing homes, so that the walking assistance navigation system for visually impaired people in nursing homes can perform assisted navigation according to the second personalized auxiliary navigation method.

[0230] For example, if Zhang San's planned walking route is completely different from Li Si's planned walking route, there is no need to interfere. Regardless of whether they use the same personalized assisted navigation method (for example, even if they both use light-guided navigation), it will not affect each other's walking. This is because although visually impaired people may not be able to see specific objects, they know the approximate location of their destination. For example, to go to the restroom, they need to turn left after exiting the door. At this time, visually impaired people will not consider whether there is light-guided navigation on their right side, because visually impaired people using the method of this application naturally know that other people may be using the method of this application at the same time.

[0231] In one embodiment, Li Si can also be navigated via voice broadcast when he comes out. For example, the navigation voice could be: "There is another user navigating ahead of you, and their route is different from yours. Please don't worry."

[0232] If the planned walking trajectories of the two methods overlap, it could lead to misleading situations if both use light-guided navigation. Therefore, it's necessary to determine whether the first personalized auxiliary navigation method conflicts with the second. If not, then...

[0233] The second personalized auxiliary navigation method is sent to the walking assistance navigation system for visually impaired people in nursing homes, so that the walking assistance navigation system for visually impaired people in nursing homes can perform assisted navigation according to the second personalized auxiliary navigation method.

[0234] Specifically, if Zhang San uses voice navigation and Li Si uses light navigation, then they can walk according to their respective directions.

[0235] In addition, in this situation, real-time monitoring by a camera device can determine whether Zhang San and Li Si are on the same floor. If so, a voice announcement can be made, for example: "Hello, there are other people walking on your floor. You can communicate with each other to prevent collisions."

[0236] Understandably, if we need to determine whether the personalized auxiliary navigation method conflicts with the second personalized auxiliary navigation method, then...

[0237] The system uses various cameras in the walking assistance navigation system for visually impaired people in nursing homes to obtain the floor information of the pedestrian corresponding to the first person identification information and the floor information of the pedestrian corresponding to the second person identification information.

[0238] If the floor information of the pedestrian corresponding to the first person identification information is not the same floor as the floor information of the pedestrian corresponding to the second person identification information, then

[0239] The system for assisting visually impaired people in nursing homes uses the first personalized navigation method to provide navigation assistance based on the floor information of the pedestrian corresponding to the first person identification information in the walking assistance navigation system, and generates second person identification information navigation information to notify the pedestrian corresponding to the first person identification information through voice broadcast.

[0240] The system for assisting visually impaired people in nursing homes uses the second personalized navigation method to provide assisted navigation based on the floor information of the pedestrian corresponding to the second person identification information in the walking assistance navigation system. The system generates navigation information based on the first person identification information and notifies the pedestrian corresponding to the second person identification information via voice broadcast.

[0241] For example, if after determining the conflict, it is found that the two are not actually on the same floor, then each floor can be navigated separately. For instance, if Zhang San is on the second floor and Li Si is on the third floor, then the devices on the second floor will navigate for Zhang San, while the devices on the third floor will navigate for Li Si.

[0242] In this embodiment, after using the second personalized auxiliary navigation method to assist navigation based on the floor information of the pedestrian corresponding to the second person recognition information in the walking assistance navigation system for visually impaired people in the nursing home, the walking assistance navigation method further includes:

[0243] The system monitors the location of the pedestrian corresponding to the first person identification information and the location of the pedestrian corresponding to the second person identification information in real time. When the locations of the pedestrians corresponding to the first person identification information and the second person identification information are on the same floor, the second auxiliary personalized navigation method is fixed as voice guidance navigation.

[0244] Specifically, once the two are on the same floor due to movement, the second auxiliary personalized navigation method is fixed as voice guidance navigation.

[0245] It is understandable that the method of this application can also use the above logic to control when multiple people come out. In addition, the number of people can be restricted. For example, only two people can be navigated at most. After two people, if it is determined that someone else has come out and made a gesture, the third person can be asked to go back to the room and wait by voice. For example, the voice can announce: Hello, there are many people who need assistance now. Please go back to the room and wait for 10 minutes (this time can be set freely) before coming out.

[0246] In addition, all the camera devices in this application are connected to the central control screen and will transmit real-time images to the central control screen. The nursing home will have dedicated personnel to monitor the situation on each floor. If any problems occur during navigation, they can be dealt with in real time.

[0247] In addition, each camera device in this application can also perform fall detection, and once a fall is detected, it can also provide alarm navigation.

[0248] This application has the following advantages over the prior art:

[0249] 1. All facilities in this application are located inside the building, and it is not necessary to prepare a separate navigation device for each visually impaired person.

[0250] 2. All facilities in this application are located inside the building. When visually impaired people need to walk, they do not need to take out additional assistive devices. This reduces the trouble for visually impaired people who need to hold or wear assistive devices when they are unaccompanied. Especially at night, if visually impaired people want to go to the toilet, they may forget to hold or wear assistive devices, or they may find it troublesome to hold or wear assistive devices.

[0251] 3. This application enables the entire building to have intelligent navigation assistance. Once installed, no additional equipment will be needed regardless of changes in the residents.

[0252] 4. The navigation provided in this application can be personalized for each individual. Each person can choose a personalized auxiliary navigation method according to their own preferences. For example, some visually impaired people may not be able to see the light, but they are afraid of the dark. In this case, they can choose a hybrid guidance navigation (including light guidance and voice guidance). In this way, even though they cannot see the light, they will feel psychologically comforted by using the method provided in this application.

[0253] 5. The navigation provided in this application can set multiple navigation destinations for each user, thereby forming multiple navigation routes. Existing head-mounted or handheld devices can only achieve walking obstacle avoidance functions and cannot achieve real-time navigation to multiple destinations.

[0254] It should be noted that the foregoing explanation of the method embodiments also applies to the apparatus of this embodiment, and will not be repeated here.

[0255] This application also provides an electronic device (i.e., the above-mentioned main controller), including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the above-mentioned walking-assisted navigation method.

[0256] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables the implementation of the above-described walking-assisted navigation method.

[0257] Figure 2 This is an exemplary structural diagram of an electronic device capable of implementing the walking-assisted navigation method provided in one embodiment of this application.

[0258] like Figure 2 As shown, the electronic device includes an input device 501, an input interface 502, a central processing unit 503, a memory 504, an output interface 505, and an output device 506. The input interface 502, central processing unit 503, memory 504, and output interface 505 are interconnected via a bus 507. The input device 501 and output device 506 are connected to the bus 507 via the input interface 502 and output interface 505, respectively, and thus connected to other components of the electronic device. Specifically, the input device 504 receives input information from the outside and transmits it to the central processing unit 503 via the input interface 502. The central processing unit 503 processes the input information based on computer-executable instructions stored in the memory 504 to generate output information, temporarily or permanently storing the output information in the memory 504, and then transmitting the output information to the output device 506 via the output interface 505. The output device 506 outputs the output information to the outside of the electronic device for user use.

[0259] In other words, Figure 2 The illustrated electronic device may also be implemented as including: a memory storing computer-executable instructions; and one or more processors, which can be coupled when executing the computer-executable instructions. Figure 1 Describes a walking-assisted navigation method.

[0260] In one embodiment, Figure 2 The electronic device shown can be implemented as including: a memory 504 configured to store executable program code; and one or more processors 503 configured to run the executable program code stored in the memory 504 to perform the walking-assisted navigation method in the above embodiments.

[0261] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0262] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0263] Computer-readable media include both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, DVD or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0264] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0265] Furthermore, it is clear that the word "comprising" does not exclude other units or steps. Multiple units, modules, or devices recited in the apparatus claims may also be implemented by a single unit or overall apparatus via software or hardware.

[0266] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutively marked blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or the overall flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0267] In this embodiment, the processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0268] Memory can be used to store computer programs and / or modules. The processor implements various functions of the device / terminal equipment by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area can store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). In addition, memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0269] In this embodiment, if the modules / units integrated into the device / terminal equipment are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

[0270] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0271] Furthermore, it is clear that the word "comprising" does not exclude other units or steps. Multiple units, modules, or devices recited in the apparatus claims may also be implemented by a single unit or overall apparatus via software or hardware.

[0272] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A walking-assisted navigation method, characterized in that, The walking-assisted navigation method includes: Get room door opening information; The first coordinate information is obtained based on the room door opening information, and the first coordinate information is sent to the camera devices in the nursing home that are capable of capturing the first coordinate information. Acquire image information transmitted by a camera device capable of capturing the scene at the doorway of the room based on the first coordinate information; First person recognition information and first person gesture information are obtained from one or more image information from each image information; Obtain an auxiliary knowledge graph, which includes multiple character nodes, gesture nodes corresponding to each character node, personalized auxiliary navigation methods corresponding to each gesture node, and destination information corresponding to each gesture node. Obtain each gesture node of the person node corresponding to the first person recognition information; The similarity between the first person's gesture information and each of the gesture nodes is measured to obtain the personalized auxiliary navigation method and destination information corresponding to the gesture node with the highest similarity to the first person's gesture information among the various gesture nodes of the corresponding person node. A first personalized auxiliary navigation method is generated based on the personalized auxiliary navigation method and destination information; The first personalized auxiliary navigation method is sent to the walking auxiliary navigation system for visually impaired people in nursing homes, so that the walking auxiliary navigation system for visually impaired people in nursing homes can perform assisted navigation according to the first personalized auxiliary navigation method; The acquisition of room door opening information includes: Acquire the door opening detection signal transmitted by the door opening detector and the identification information of the door opening detector; The step of obtaining the first coordinate information based on the room door opening information and sending the first coordinate information to the various camera devices in the nursing home, including the camera devices capable of capturing the first coordinate information, includes: Obtain a coordinate database, which includes at least one preset door opening detector identifier and first coordinate information corresponding to each preset door opening detector identifier; Obtain the first coordinate information corresponding to the preset door opening detector identifier that is the same as the identifier information of the door opening detector; Obtain a preset camera device coordinate database, which includes at least one preset camera device identifier and a captureable coordinate table corresponding to each preset camera device identifier, and each captureable coordinate table includes at least one preset first coordinate information; Obtain the preset camera device identifier corresponding to each camera coordinate table that has the same preset first coordinate information as the first coordinate information; The first coordinate information is sent to each camera corresponding to each preset camera device identifier based on the obtained preset camera device identifier; The step of obtaining the first person recognition information and the first person gesture information based on one or more image information from various image information includes: Obtain a trained face neural network model; Extract facial features from each image separately; The facial features are input into the trained facial neural network model to obtain first person recognition information; Obtain a preset gesture navigation database, which includes preset person recognition information and gesture navigation reminder information corresponding to each preset person recognition information; Obtain gesture navigation reminder information corresponding to preset person recognition information that is identical to the first person recognition information; The gesture navigation reminder information is sent to the corresponding reminder device, so that the reminder device performs a reminder operation based on the gesture navigation reminder information; Obtain the trained gesture neural network model; After the reminder operation is performed, the gesture features in each image transmitted by the camera device capable of capturing the first coordinate information are extracted respectively. The gesture features are input into the trained gesture neural network model to obtain the gesture information of the first person. The personalized auxiliary navigation methods include light-guided navigation, voice-guided navigation, and hybrid navigation. The method for generating the first personalized auxiliary navigation based on the personalized auxiliary navigation method and destination information includes: A first planned walking trajectory is generated based on the destination information and the first coordinate information. The first planned walking trajectory includes multiple necessary coordinate information between the destination information and the first coordinate information. Obtain a pre-stored database of a walking assistance navigation system for visually impaired people in nursing homes. The pre-stored database includes the identifiers of each assistance device and the preset assistance coordinate information corresponding to each assistance device identifier. Obtain the identifier of each auxiliary device corresponding to the preset auxiliary coordinate information that is the same as at least one of the multiple required coordinate information; A first personalized auxiliary navigation method is generated based on the auxiliary device identifier and the personalized auxiliary navigation method. After sending the first personalized assisted navigation method to the walking assistance navigation system for visually impaired people in nursing homes, the walking assistance navigation method further includes: Determine whether new room door opening information has been obtained within a preset time period; if so, then... Obtain the second coordinate information based on the new room door opening information; Determine whether the first coordinate information is the same as the second coordinate information; if not, then... The second coordinate information is sent to the cameras in the nursing home that are capable of capturing the second coordinate information. Acquire new image information transmitted by a camera device capable of acquiring a new view of the room entrance based on the second coordinate information; Obtain second person recognition information and second person gesture information based on one or more of the new image information; The auxiliary knowledge graph is obtained, which includes multiple character nodes, gesture nodes corresponding to each character node, personalized auxiliary navigation methods corresponding to each gesture node, and destination information corresponding to each gesture node. Obtain each gesture node of the person node corresponding to the second person recognition information; The similarity measurement between the second person's gesture information and each gesture node is performed to obtain the personalized auxiliary navigation method and destination information corresponding to the gesture node with the highest similarity to the second person's gesture information among the various gesture nodes of the corresponding person node. The personalized auxiliary navigation method obtained through the second person's gesture information is called the second personalized auxiliary navigation method, and the destination information obtained through the second person's gesture information is called the second destination information. A second personalized auxiliary navigation method is generated based on the second personalized auxiliary navigation method and the second destination information; A second planned walking trajectory is generated based on the second destination information and the second coordinate information. The second planned walking trajectory includes multiple necessary coordinate information between the destination information and the second coordinate information. Determine whether the second planned walking trajectory and the first planned walking trajectory share common necessary coordinate information. If so, then... Determine whether the personalized auxiliary navigation method obtained through the first person's gesture information conflicts with the second personalized auxiliary navigation method. If not, then... The second personalized auxiliary navigation method is sent to the walking auxiliary navigation system for visually impaired people in nursing homes, so that the walking auxiliary navigation system for visually impaired people in nursing homes can perform assisted navigation according to the second personalized auxiliary navigation method; After sending the personalized assisted navigation method to the walking assisted navigation system for visually impaired people in nursing homes, the walking assisted navigation method further includes: Determine whether the personalized auxiliary navigation method obtained through the first character's gesture information conflicts with the second personalized auxiliary navigation method. If so, then... The system uses various cameras in the walking assistance navigation system for visually impaired people in nursing homes to obtain the floor information of the pedestrian corresponding to the first person identification information and the floor information of the pedestrian corresponding to the second person identification information. If the floor information of the pedestrian corresponding to the first person identification information is not the same floor as the floor information of the pedestrian corresponding to the second person identification information, then Obtain the floor information of the pedestrian corresponding to the first person identification information, and perform auxiliary navigation according to the first auxiliary personalized auxiliary navigation method; the floor information of the pedestrian corresponding to the first person identification information is the floor information of the pedestrian corresponding to the first person identification information in the walking auxiliary navigation system for visually impaired people in nursing homes; Generate navigation information with second person identification information and notify the pedestrian corresponding to the first person identification information via voice broadcast; Obtain the floor information of the pedestrian corresponding to the second person identification information, and perform auxiliary navigation according to the second auxiliary personalized auxiliary navigation method; the floor information of the pedestrian corresponding to the second person identification information is the floor information of the pedestrian corresponding to the second person identification information in the walking auxiliary navigation system for visually impaired people in the nursing home; The second auxiliary personalized navigation method generates navigation information based on first person identification information and notifies the pedestrian corresponding to the second person identification information via voice broadcast.

2. The walking-assisted navigation method as described in claim 1, characterized in that, After using the second personalized navigation method to assist navigation based on the floor information of the pedestrian corresponding to the second person recognition information in the walking assistance navigation system for visually impaired people in the nursing home, the walking assistance navigation method further includes: The system monitors the location of the pedestrian corresponding to the first person identification information and the location of the pedestrian corresponding to the second person identification information in real time. When the locations of the pedestrians corresponding to the first person identification information and the second person identification information are on the same floor, the second auxiliary personalized navigation method is fixed as voice guidance navigation.

Images