Multi-layer space navigation method and system
By generating three-dimensional maps of the mall and using mobile phone cameras, lidar, AR technology and gyroscope data, the problem of inaccurate navigation in multi-story buildings is solved, and accurate navigation and user interface in complex environments are achieved, reducing the intuitiveness of the user interface The risk of getting lost.
Patent Information
- Application Number
- CN202411545204.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-01
- Publication Date
- 2024-11-29
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In multi-story buildings, existing navigation technology is susceptible to signal interference and positioning errors, resulting in inaccurate navigation and unintuitive user interface, making it difficult for customers to find the target location in the mall and have a high risk of getting lost.
By generating a three-dimensional map of the mall, it is divided into floor areas and connection areas, using mobile phone cameras and lidar for environmental scanning, combining building signs for positioning, using AR technology and gyroscope data for navigation path planning and direction detection, setting bias Navigation reminder distance threshold, update navigation paths in real time to ensure accurate navigation.
It realizes accurate tracking of user orientation and location in complex environments, reducing time waste caused by deviating from the path, reducing the risk of getting lost, and providing more intuitive and accurate navigation services.
Smart Images

Figure CN119043341A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of multi-layer space navigation, and in particular to a multi-layer space navigation method and system. Background Art
[0002] Shopping malls can provide customers with detailed navigation guidance to help customers find target stores, service points or facilities in the complex environment of large shopping malls, reduce the time spent looking for stores, improve the shopping experience, and also help shopping mall managers understand the flow paths and preferences of customers, so as to optimize layout and promotion strategies; At present, navigation is provided to users by integrating the floor plan and real-time positioning information of the shopping mall, and providing the path information between the customer's location and the target location. Customers can obtain navigation instructions through mobile phone applications or electronic guide screens in the shopping mall. However, in actual applications, users may be affected by signal interference or positioning errors in multi-story buildings, resulting in inaccurate navigation. Secondly, the navigation user interface and interactive experience are not intuitive enough, which causes customers to feel confused during use and cannot achieve good navigation effects. Therefore, a multi-layer space navigation method and system are proposed. Summary of the invention
[0003] The object of the present invention is to provide a multi-layer space navigation method and system to solve the problems raised in the above background technology.
[0004] In order to solve the above technical problems, one of the objectives of the present invention is to provide a multi-layer space navigation method, comprising the following steps: S1, generating a three-dimensional map of the shopping mall, and dividing the map into floor areas and connection areas according to building signs; S2, extract the image data of the user's mobile phone camera and combine it with the building logo to locate the user, then plan a navigation path for the user, and use the mobile phone AR combined with the navigation path to detect the moving direction. When the direction detection is correct, stop the mobile phone AR detection; S3, extracting the gyroscope data of the mobile phone, taking the direction of the top of the mobile phone as the target direction, and then setting the direction nodes in the navigation path in sections, and then performing direction detection on the extracted direction node closest to the user's position and the target direction, when the offset angle of the target direction combined with the navigation path and the direction node exceeds forty-five degrees, the user is reminded to adjust through the mobile phone; S4. Set a deviation warning distance threshold. In the planar map, the user position and navigation route are combined with the deviation warning distance threshold to perform distance monitoring. When the user position and navigation route exceed the deviation warning distance threshold, repeat step S2 to update the navigation path.
[0005] As a further improvement of the technical solution, the S1 uses a camera combined with a laser radar to scan the environment inside the mall, thereby generating a three-dimensional map of the mall. During the environmental scanning process, the unique building logo of each location in the mall is extracted, and then the three-dimensional map is combined with the building logo to distribute the floor area and the connection area; The connecting areas are stairs and elevators; Floor areas are all locations in the mall except stairs and elevators.
[0006] As a further improvement of the present technical solution, S2 reminds the user to capture surrounding image data when the user needs to navigate on the mobile phone, and identifies the user's floor and position based on the image data captured by the user and the building signs.
[0007] As a further improvement of the technical solution, the steps of S2 are as follows: S2.1. The floor and location of the user are used as the starting point, and then the end point that the user needs to navigate to is collected. The starting point and the end point are combined with the three-dimensional map to plan the navigation path and generate the navigation path; S2.2. When the navigation path is generated and navigation begins, the mobile phone is controlled to turn on the camera to enter the AR interface, and the route of the navigation path is inserted in the AR interface to guide the user. Then, when the mobile phone detects the movement of the user, it analyzes the scene data transformation in the AR interface to obtain the user's movement direction, and then combines the movement direction with the navigation path for direction detection. When the movement direction is consistent with the direction of the navigation path, the mobile phone AR interface is stopped. Conversely, when the movement direction is inconsistent with the direction of the navigation path, the mobile phone sends a signal to remind the user to change direction.
[0008] As a further improvement of the technical solution, in the navigation process, S3 establishes a dynamic arrow in the mobile phone display interface, the direction of the dynamic arrow always follows the direction node, and the spacing between each direction node is the same during the segmentation process of the direction node; Every time the user reaches a direction node, the phone will vibrate to remind the user until the user reaches the destination.
[0009] As a further improvement of the technical solution, the formula of S3 is as follows: in, is the direction of the top of the phone. It is the direction node closest to the user's location; in, is the 45° threshold, when If the threshold is exceeded, it is considered that the user's direction has deviated from the navigation path and needs to be adjusted, and then a reminder signal to adjust the direction is sent through the mobile phone.
[0010] As a further improvement of the present technical solution, the formula of S4 is as follows: Where d is the distance from the user to the line segment, is the user's current location, and For the endpoints of the navigation path, set The distance threshold for the deviation warning; , it is considered that the user has deviated from the navigation path and the navigation path needs to be updated; , then continue to navigate along the current path.
[0011] As a further improvement of the present technical solution, when S2 locates that the user enters the connection area, S3 and S4 are stopped, and the mobile phone is kept in the AR state until the user is located to enter the floor area.
[0012] The second object of the present invention is to provide a multi-layer space navigation system, comprising any one of the multi-layer space navigation methods described above, including a three-dimensional map generation unit, a motion detection unit and a distance monitoring unit; The three-dimensional map generation unit is used to generate a three-dimensional map of the shopping mall, which is divided into floor areas and connection areas; The movement detection unit is used to obtain the image data of the user's mobile phone camera in combination with the building logo to locate the user, then plan a navigation path for the user, and detect the movement direction through the mobile phone AR in combination with the navigation path; The distance monitoring unit is used to extract the direction node and target direction closest to the user position for direction detection, and simultaneously combines the user position and the navigation route with the deviation warning distance threshold for distance monitoring.
[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: through the real-time feedback of the gyroscope and position data, the user's direction and position can be accurately tracked, and the deviation can be adjusted in time. At the same time, by replanning the path, it is ensured that the user always travels along the optimal route, reducing the time waste caused by deviation from the path, and then through continuous deviation detection and path updating, the risk of users getting lost is reduced, especially in complex or unfamiliar environments. Combined with the user's direction and position data, a navigation service that better meets actual needs is provided, especially in multi-story shopping malls or complex environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 It is the overall flow chart of the present invention; Figure 2 A flowchart of the present invention for reminding the user to change direction; Figure 3 This is a structural principle diagram of the three-dimensional map generation unit of the present invention.
[0015] The meaning of each number in the figure is: 10. Three-dimensional map generation unit; 20. Motion detection unit; 30. Distance monitoring unit. DETAILED DESCRIPTION
[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0017] like Figure 1-3 As shown, one of the purposes of the present invention is to provide a multi-layer space navigation method, comprising the following steps: S1, generating a three-dimensional map of the shopping mall, and dividing the map into floor areas and connection areas according to building signs; S1 uses a camera combined with a laser radar to scan the interior of the mall, thereby generating a 3D map of the mall. During the environmental scanning process, it extracts the unique building logos at each location in the mall, and then uses the 3D map combined with the building logos to distribute the floor areas and connection areas. The connecting areas are stairs and elevators; Floor areas are all locations in the mall except stairs and elevators; The camera is used to capture details and image information of the environment, the lidar is used to scan the environment with high precision and generate point cloud data, and the computer and software are used for data processing and map generation. The specific working steps are as follows: Data collection: LiDAR scans are performed at different locations inside the mall to collect point cloud data, ensuring that the scan covers the entire mall, including every floor, and then cameras are used to take pictures or videos of the environment, especially areas that require details, such as the location of stairs and elevators; Data preprocessing: Perform preprocessing steps such as filtering, denoising, and registration on the point cloud data collected by the lidar for subsequent processing, and process the images collected by the camera, such as correction and segmentation, to extract important building signs and features; 3D map generation: convert the processed point cloud data into a 3D mesh model or other suitable 3D representation, and then fuse the image information into the 3D model to improve the accuracy and clarity of details; Building sign extraction: Extract building signs inside the mall, such as stairs, elevators, store signs, etc., from the image, and then match these building signs with the corresponding positions in the 3D model to mark the locations of connecting areas such as stairs and elevators; Area division: By identifying and marking the locations of stairs and elevators, the connection areas are determined. These areas can be determined by the spatial distribution in the point cloud data and the features in the image. After removing the connection areas, the remaining areas are divided into floor areas. This can be determined based on the height information of the point cloud data and the floor identification in the image. Map integration and visualization: Integrate all area division information into the final 3D map to ensure that each area is correctly marked and distinguished. Use visualization software to display the final 3D map to ensure that the distribution of floor areas and connecting areas is clear and visible.
[0018] S2, extract the image data of the user's mobile phone camera and combine it with the building logo to locate the user, then plan a navigation path for the user, and use the mobile phone AR combined with the navigation path to detect the moving direction. When the direction detection is correct, stop the mobile phone AR detection; When the user needs to navigate on the mobile phone, S2 reminds the user to capture surrounding image data, and identifies the user's floor and location based on the image data captured by the user and the building signs.
[0019] The steps of S2 are as follows: S2.1. The floor and location of the user are used as the starting point, and then the end point that the user needs to navigate is collected. The starting point and the end point are combined with the three-dimensional map to plan the navigation path and generate the navigation path. The specific working steps are as follows: Image acquisition and data preprocessing: Users use their mobile phone cameras to capture images of the current environment and preprocess the images, including denoising and contrast enhancement, to improve the effect of subsequent processing.
[0020] Image feature extraction: Extract feature points such as corners and edges from images, and use feature detection algorithm ORB to identify building signs and key points; Image and building logo matching: Match the extracted feature points with the known building logo database, and use the matching algorithm FLANN to identify the current location and floor; Floor and location positioning: Determine the floor and location of the user based on the matching results, and use the three-dimensional model of the building for spatial positioning; Path planning: Collect the user's destination information, perform path planning on a three-dimensional map, and calculate the optimal path from the current location to the destination; Path generation and navigation: Convert the planned path into navigation instructions that the user can understand, display them on the phone, and provide navigation through voice prompts; Real-time updates and feedback: Track the user's location in real time and update the navigation path.
[0021] S2.2, when the navigation path is generated and navigation begins, the mobile phone is controlled to turn on the camera to enter the AR interface, and the route of the navigation path is inserted in the AR interface to guide the user. Then, when the mobile phone detects the user's movement, it analyzes the scene data transformation in the AR interface to obtain the user's movement direction, and then combines the movement direction with the navigation path for direction detection. When the movement direction is consistent with the direction of the navigation path, the mobile phone AR interface is stopped. On the contrary, when the movement direction is inconsistent with the direction of the navigation path, the mobile phone sends a signal to remind the user to change direction. The specific working steps are as follows: Start the navigation function: detect the user's request to start navigation, call the mobile phone camera, enter the AR mode, and draw the navigation path in the AR interface; Real-time motion detection: Use the phone's accelerometer and gyroscope to detect the user's movement direction, thereby shortening or extending the remaining distance of the navigation path; Direction consistency detection: Dynamically detect the remaining distance of the navigation path. When the remaining distance of the navigation path is extended and the extended distance is greater than the set threshold (for example, 5 meters), a prompt signal is triggered to remind the user to adjust the direction; When the remaining distance of the navigation path is shortened and the shortened distance is greater than the set threshold (for example, 5 meters), the directions are considered consistent and the AR interface update is stopped.
[0022] S3. Extract the gyroscope data of the mobile phone, take the direction of the top of the mobile phone as the target direction, and then set the direction nodes in the navigation path in sections. Then, extract the direction node closest to the user's position and the target direction for direction detection. When the offset angle of the target direction combined with the navigation path and the direction node exceeds 45 degrees, the mobile phone reminds the user to adjust. The specific working steps are as follows: Extract gyroscope data: Read data from the phone's gyroscope to determine the orientation of the top of the phone; Set direction nodes: Set direction nodes in segments in the navigation path. Each node has a direction vector, and the spacing between direction nodes is the same during segmentation. Calculate direction offset: Find the direction node closest to the user's position and calculate the angle between the top of the phone and the direction of the node. The formula is as follows: in, is the direction of the top of the phone. It is the direction node closest to the user's location; in, is the 45° threshold, when If the threshold is exceeded, it is considered that the user's direction has deviated from the navigation path and needs to be adjusted, and then a reminder signal to adjust the direction is sent through the mobile phone.
[0023] During navigation, S3 creates a dynamic arrow in the mobile phone display interface. The direction of the dynamic arrow always follows the direction node, and the distance between each direction node is the same during segmentation. Every time the user reaches a direction node, the phone will vibrate to remind the user until the user reaches the destination.
[0024] S4, setting a deviation warning distance threshold, in the plane map, combining the user position and the navigation route with the deviation warning distance threshold for distance monitoring, when the user position and the navigation route exceed the deviation warning distance threshold, repeating step S2 to update the navigation path, the specific steps are as follows; Set the deviation warning distance threshold: Set a deviation warning distance threshold, that is, when the distance between the user's location and the navigation route exceeds this threshold, the warning is triggered; Get the user's current location: Get the user's current location and the line segment of the current navigation path; Calculate the distance from the user to the navigation route: Calculate the distance from the user's current location to the nearest navigation route segment. This can be achieved by calculating the shortest distance from a point to a line segment. The formula is as follows: Where d is the distance from the user to the line segment, is the user's current location, and For the endpoints of the navigation path, set The distance threshold for the deviation warning; , it is considered that the user has deviated from the navigation path and the navigation path needs to be updated; , then continue to navigate along the current path.
[0025] This formula calculates the vertical distance from a point to a line. If the user's location is between the two endpoints, the distance can be used directly; if not, the distance from the point to the endpoint is calculated and the minimum value is taken. This ensures that the user is always near the navigation path, and when they stray too far away, the path is updated in time to maintain accurate navigation guidance.
[0026] When S2 locates the user entering the connection area, it stops S3 and S4 and keeps the phone in the AR state until the user enters the floor area.
[0027] The second object of the present invention is to provide a multi-layer space navigation system, including any one of the multi-layer space navigation methods described above, including a three-dimensional map generation unit 10, a motion detection unit 20 and a distance monitoring unit 30; The three-dimensional map generation unit 10 is used to generate a three-dimensional map of the shopping mall, which is divided into floor areas and connection areas; The motion detection unit 20 is used to obtain the image data of the user's mobile phone camera in combination with the building logo to locate the user, then plan a navigation path for the user, and detect the moving direction through the mobile phone AR combined with the navigation path; The distance monitoring unit 30 is used to extract the direction node and target direction closest to the user's position for direction detection, and at the same time combine the user's position and navigation route with the deviation warning distance threshold for distance monitoring.
[0028] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.
Claims
1. A multi-layer space navigation method, characterized in that: The steps include: S1, generating a three-dimensional map of the shopping mall, and dividing the map into floor areas and connection areas according to building signs; S2, extract the image data of the user's mobile phone camera and combine it with the building logo to locate the user, then plan a navigation path for the user, and use the mobile phone AR combined with the navigation path to detect the moving direction. When the direction detection is correct, stop the mobile phone AR detection; S3, extracting the gyroscope data of the mobile phone, taking the direction of the top of the mobile phone as the target direction, and then setting the direction nodes in the navigation path in sections, and then performing direction detection on the extracted direction node closest to the user's position and the target direction, when the offset angle of the target direction combined with the navigation path and the direction node exceeds forty-five degrees, the user is reminded to adjust through the mobile phone; S4. Set a deviation warning distance threshold. In the planar map, the user position and navigation route are combined with the deviation warning distance threshold to perform distance monitoring. When the user position and navigation route exceed the deviation warning distance threshold, repeat step S2 to update the navigation path.
2. The multi-layer spatial navigation method according to claim 1, characterized in that: The S1 uses a camera combined with a laser radar to scan the environment inside the mall, thereby generating a three-dimensional map of the mall. During the environmental scanning process, the unique building logo of each location in the mall is extracted, and then the three-dimensional map is combined with the building logo to distribute the floor area and the connection area; The connecting areas are stairs and elevators; Floor areas are all locations in the mall except stairs and elevators.
3. The multi-layer spatial navigation method according to claim 1, characterized in that: When the user needs to navigate on the mobile phone, S2 reminds the user to capture surrounding image data, and identifies the user's floor and location based on the image data captured by the user and the building logo.
4. The multi-layer spatial navigation method according to claim 1, characterized in that: The steps of S2 are as follows: S2.
1. The floor and location of the user are used as the starting point, and then the end point that the user needs to navigate to is collected. The starting point and the end point are combined with the three-dimensional map to plan the navigation path and generate the navigation path; S2.
2. When the navigation path is generated and navigation begins, the mobile phone is controlled to turn on the camera to enter the AR interface, and the route of the navigation path is inserted in the AR interface to guide the user. Then, when the mobile phone detects the movement of the user, it analyzes the scene data transformation in the AR interface to obtain the user's movement direction, and then combines the movement direction with the navigation path for direction detection. When the movement direction is consistent with the direction of the navigation path, the mobile phone AR interface is stopped. Conversely, when the movement direction is inconsistent with the direction of the navigation path, the mobile phone sends a signal to remind the user to change direction.
5. The multi-layer spatial navigation method according to claim 1, characterized in that: In the process of navigation, S3 creates a dynamic arrow in the mobile phone display interface, the direction of the dynamic arrow always follows the direction node, and the distance between each direction node is the same during the segmentation process of the direction node; Every time the user reaches a direction node, the phone will vibrate to remind the user until the user reaches the destination.
6. The multi-layer spatial navigation method according to claim 1, characterized in that: The formula of S3 is as follows: in, is the direction of the top of the phone. It is the direction node closest to the user's location; in, is the 45° threshold, when If the threshold is exceeded, it is considered that the user's direction has deviated from the navigation path and needs to be adjusted, and then a reminder signal to adjust the direction is sent through the mobile phone.
7. The multi-layer spatial navigation method according to claim 1, characterized in that: The formula of S4 is as follows: Where d is the distance from the user to the line segment, is the user's current location, and For the endpoints of the navigation path, set The distance threshold for the deviation warning; , it is considered that the user has deviated from the navigation path and the navigation path needs to be updated; , then continue to navigate along the current path.
8. The multi-layer space navigation method according to claim 1, characterized in that: When S2 locates that the user enters the connection area, S3 and S4 are stopped, and the mobile phone is kept in the AR state until the user is located to enter the floor area.
9. A method for implementing a multi-layer space navigation system, comprising the multi-layer space navigation method according to any one of claims 1 to 8, characterized in that: It comprises a three-dimensional map generating unit (10), a movement detecting unit (20) and a distance monitoring unit (30); The three-dimensional map generation unit (10) is used to generate a three-dimensional map of the shopping mall, which is divided into floor areas and connection areas; The movement detection unit (20) is used to obtain image data from a user's mobile phone camera in combination with a building logo to locate the user, then plan a navigation path for the user, and detect the movement direction through the mobile phone AR in combination with the navigation path; The distance monitoring unit (30) is used to extract the direction node and target direction closest to the user's position for direction detection, and simultaneously combines the user's position and navigation route with the deviation warning distance threshold for distance monitoring.
Citation Information
Patent Citations
Voice navigation control method, system and intelligent watch
CN106568457A
Autonomous navigation robot and autonomous navigation method thereof
CN106708053A
Indoor navigation method, device and mobile terminal
CN108801259A
Virtual reality technology-based indoor navigation method and system
CN109520510A
Augmented reality-based navigation method
CN110440815A
