Indoor navigation methods, devices, equipment, software products and storage media
By establishing a navigation map structure in an indoor environment and using image data to identify the location relationships of points of interest, the problem of delivery personnel having difficulty finding merchants in indoor environments is solved, thus improving navigation efficiency.
Patent Information
- Application Number
- CN202310410295.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-14
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2043-04-14
AI Technical Summary
In indoor environments, especially in places like large shopping malls, delivery capacity struggles to efficiently locate merchants, and existing navigation solutions are ineffective.
By identifying the relative positions of points of interest (POIs) in indoor environmental image data, a navigation map structure is established, including multiple nodes and floor connection nodes, providing navigation data from the current merchant to the destination merchant.
It improves the efficiency for users to find merchants in indoor environments, especially for delivery capacity, reducing the time spent searching for merchants.
Smart Images

Figure CN116448116B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of Internet technology, and in particular to indoor navigation methods, devices, equipment, program products and storage media. Background Technology
[0002] Indoor navigation remains a significant technical challenge due to the substantial drift of satellite signals in indoor environments. Users often struggle to efficiently locate merchants within large shopping malls and supermarkets. This is particularly problematic for delivery services, which are widely used in food delivery and errand services. In these scenarios, delivery personnel must pick up items from physical stores and deliver them to the user's designated location. Since many physical stores are located in indoor environments like shopping malls, delivery personnel often struggle to locate them efficiently indoors. Summary of the Invention
[0003] To overcome the problems existing in related technologies, this disclosure provides indoor navigation methods, devices, equipment, program products, and storage media.
[0004] According to a first aspect of the embodiments of this specification, an indoor navigation method is provided, the method comprising:
[0005] For users in indoor environments, determine the current merchant at the user's location and the destination merchant the user is about to visit;
[0006] Obtain the navigation map structure corresponding to the indoor environment; wherein the navigation map structure includes multiple nodes, each node representing a POI within the indoor environment; the navigation map structure is established by identifying the relative positional relationships between POIs using image data captured of the indoor environment;
[0007] Based on the nodes corresponding to the current merchant and the destination merchant in the navigation graph structure, at least one path node between the current merchant and the destination merchant is obtained using the navigation graph structure;
[0008] Based on the at least one transit node, obtain navigation data from the current merchant to the destination merchant.
[0009] Optionally, the navigation map structure includes merchant nodes representing merchants; wherein, in the case that the indoor environment has multiple floors, the navigation map structure further includes floor connection nodes, which include connection nodes representing stairs and / or connection nodes representing elevators.
[0010] Optionally, the information carried by the edges between adjacent nodes in the navigation graph structure includes: the movement time between the adjacent nodes, and the node carrying node information;
[0011] The step of obtaining at least one path node from the current merchant to the destination merchant using the navigation map structure includes:
[0012] Using the information carried by the edges between nodes in the navigation graph structure, the optimal path with the shortest travel time from the current merchant to the destination merchant is obtained;
[0013] Based on the node information of each node traversed by the optimal path, at least one path node is determined.
[0014] Optionally, determining at least one transit node includes:
[0015] At least one path node is selected based on preset guidance conditions, which include any of the following:
[0016] The travel time between the two transit nodes is within the set time range;
[0017] When the current merchant and the destination merchant are located on different floors, the nodes along the route include floor connection nodes between floors and merchant nodes adjacent to the floor connection nodes; or,
[0018] The popularity of the merchant node is higher than the set popularity condition.
[0019] Optionally, the node information includes any of the following: node type, node name, node identifier, creation time, floor or navigation information.
[0020] Optionally, the navigation graph structure is established in the following manner:
[0021] Acquire one or more frames of images of the indoor environment and identify points of interest (POIs) in the images;
[0022] Based on the identified positional relationships of each POI in one or more image frames, determine the relative positional relationships of each POI in the indoor environment; and / or,
[0023] Obtain a video file of the indoor environment, and extract multiple frames and the shooting time of each frame from the video file;
[0024] After identifying one or more POIs in the image, the movement duration between each POI in each frame is determined based on the capture time of each frame.
[0025] Optionally, identifying one or more POIs in the image includes:
[0026] Identify characters in the image, determine one or more store names based on the identified characters, and use the store names to determine the corresponding merchant POI; and / or,
[0027] Identify staircase POIs or elevator POIs in the image.
[0028] Optionally, the image data of the indoor environment captured is obtained in the following manner:
[0029] Acquire image data of all or part of the indoor environment taken by any one of the following clients: delivery capacity client, user client, or merchant client.
[0030] Optionally, the user includes delivery capacity, and the destination merchant to which the user is about to go is determined in the following way:
[0031] Obtain the current delivery order of the delivery capacity, and determine the destination merchant that the user is about to visit based on the merchant address in the current delivery order.
[0032] Optionally, the navigation graph structure is updated in the following way:
[0033] If the relocation information of the target merchant is obtained, the merchant node corresponding to the target merchant is found in the navigation map structure, and one or more node information carried by the merchant node corresponding to the target merchant is cleared.
[0034] Optionally, the navigation data includes any of the following:
[0035] The node information of the nodes passed through, the edges between adjacent nodes passed through, or the movement time of adjacent nodes passed through.
[0036] Optionally, the method further includes:
[0037] The navigation data is sent to the user client so that the user client can use the navigation data for navigation.
[0038] According to a second aspect of the embodiments of this specification, an indoor navigation method is provided, the method comprising:
[0039] Determine that the user is currently in an indoor environment;
[0040] The system acquires navigation data sent by the server, from the current merchant at the user's current location to the destination merchant the user is about to visit. The navigation data is determined by the server after identifying the nodes between the current merchant and the destination merchant from the navigation map structure corresponding to the indoor environment, and obtaining at least one path node. The navigation map structure is established by identifying the relative positional relationships between POIs using image data taken of the indoor environment.
[0041] Navigation is performed using the navigation data.
[0042] In some examples, the method further includes:
[0043] Determine the current merchant at the user's current location, and / or determine the destination merchant the user is about to visit;
[0044] The current merchant at the user's current location is determined by any of the following methods:
[0045] Acquire wireless signal information and use the wireless signal information to determine the current merchant at the user's current location; or, receive current merchant information input by the user and determine the current merchant at the user's current location based on the received current merchant information.
[0046] The destination merchant that the user is about to visit is determined by any of the following methods:
[0047] When the user is a delivery driver, the system utilizes the user's current order and determines the destination merchant based on the merchant address in the current order; or, it receives the destination merchant information input by the user and determines the destination merchant based on the received destination merchant information.
[0048] In some examples, the use of navigation data for navigation includes:
[0049] Display any of the following information from the navigation data: node information of the passed-through nodes, edges between adjacent passed-through nodes, or travel time of adjacent passed-through nodes.
[0050] In some examples, the method further includes:
[0051] During the navigation process using the navigation data, the current merchant at the user's current location is determined. If it is determined that the user has deviated from the navigation data, the process returns to the step of obtaining the navigation data sent by the server from the current merchant at the user's current location to the destination merchant the user is about to visit.
[0052] In some examples, the navigation map structure includes merchant nodes representing businesses; wherein, in cases where the indoor environment has multiple floors, the navigation map structure also includes floor connection nodes, which include connection nodes representing stairs and / or connection nodes representing elevators.
[0053] In some examples, the information carried by the edges between adjacent nodes in the navigation graph structure includes: the movement time between the adjacent nodes, and the node carrying node information;
[0054] The step of obtaining at least one path node from the current merchant to the destination merchant using the navigation map structure includes:
[0055] Using the information carried by the edges between nodes in the navigation graph structure, the optimal path with the shortest travel time from the current merchant to the destination merchant is obtained;
[0056] Based on the node information of each node traversed by the optimal path, at least one path node is determined.
[0057] In some examples, determining at least one transit node includes:
[0058] At least one path node is selected based on preset guidance conditions, which include any of the following:
[0059] The travel time between the two transit nodes is within the set time range;
[0060] When the current merchant and the destination merchant are located on different floors, the nodes along the route include floor connection nodes between floors and merchant nodes adjacent to the floor connection nodes; or,
[0061] The popularity of the merchant node is higher than the set popularity condition.
[0062] In some examples, the node information includes any of the following: node type, node name, node identifier, creation time, floor or navigation information.
[0063] In some examples, the image data includes multiple frames of images; the navigation graph structure is established as follows:
[0064] Acquire multiple frames of images of the indoor environment and identify one or more points of interest (POIs) in the images;
[0065] For multiple Points of Interest (POIs) in a single image frame, determine the relative positional relationship of each POI within the indoor environment based on their positional relationships within the image; and / or,
[0066] Obtain a video file of the indoor environment, and extract multiple frames and the shooting time of each frame from the video file;
[0067] After identifying one or more POIs in the image, the movement duration between each POI in each frame is determined based on the capture time of each frame.
[0068] In some examples, identifying one or more POIs in the image includes:
[0069] Identify characters in the image, determine one or more store names based on the identified characters, and use the store names to determine the corresponding merchant POI; and / or,
[0070] Identify staircase POIs or elevator POIs in the image.
[0071] In some examples, the navigation graph structure is updated in the following way:
[0072] If the relocation information of the target merchant is obtained, the merchant node corresponding to the target merchant is found in the navigation map structure, and the node information carried by the merchant node corresponding to the target merchant is cleared.
[0073] In some examples, the method further includes:
[0074] An image data upload function is provided, which acquires image data of all or part of the indoor environment taken by the user and sends it to the server so that the server can build the navigation map structure.
[0075] According to a third aspect of the embodiments of this specification, an indoor navigation device is provided, the device comprising:
[0076] The merchant determination module is used to: determine the current merchant at the user's location and the destination merchant the user is about to visit for users in an indoor environment;
[0077] The navigation map structure acquisition module is used to: acquire the navigation map structure corresponding to the indoor environment; wherein, the navigation map structure includes multiple nodes, each node representing a POI within the indoor environment; the navigation map structure is established by identifying the relative positional relationships between POIs using image data captured of the indoor environment;
[0078] The node acquisition module is used to: based on the nodes corresponding to the current merchant and the destination merchant in the navigation graph structure, acquire at least one path node between the current merchant and the destination merchant using the navigation graph structure;
[0079] The navigation data acquisition module is used to: acquire navigation data from the current merchant to the destination merchant based on the at least one transit node.
[0080] According to a fourth aspect of the embodiments of this specification, an indoor navigation device is provided, the device comprising:
[0081] The determination module is used to: determine the user's current indoor environment;
[0082] The acquisition module is used to: acquire navigation data sent by the server from the current merchant at the user's current location to the destination merchant the user is about to visit; the navigation data is determined by the server after identifying the nodes between the current merchant and the destination merchant from the navigation map structure corresponding to the indoor environment, and acquiring at least one path node; the navigation map structure is established by identifying the relative positional relationships between POIs using image data taken of the indoor environment;
[0083] A navigation module is used to: perform navigation using the navigation data.
[0084] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method embodiments described in the first or second aspect above.
[0085] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method embodiments described in the first or second aspect above.
[0086] According to a seventh aspect of the embodiments of this specification, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method embodiments described in the first or second aspect above.
[0087] The technical solutions provided in the embodiments of this specification may include the following beneficial effects:
[0088] In this embodiment, a navigation map structure is established beforehand by identifying the relative positional relationships between Points of Interest (POIs) using image data captured from the indoor environment. The navigation map structure includes multiple nodes, each representing a POI within the indoor environment. Based on this, when a user is indoors, the current merchant and the destination merchant can be determined. Then, based on the corresponding nodes of the current and destination merchants in the navigation map structure, at least one path node between the current and destination merchants is obtained. Based on this at least one path node, navigation data from the current merchant to the destination merchant is obtained. This navigation data provides indoor navigation for users, improving their efficiency in finding merchant locations.
[0089] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0090] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this specification and, together with the specification, serve to explain the principles of this disclosure.
[0091] Figure 1 This is a schematic diagram illustrating a delivery scenario according to an exemplary embodiment of this specification.
[0092] Figure 2A This is a flowchart illustrating an indoor navigation method according to an exemplary embodiment of this specification.
[0093] Figure 2B This is a schematic diagram of some nodes in a navigation graph structure according to an exemplary embodiment of this specification.
[0094] Figure 2C This is a schematic diagram illustrating a client displaying navigation data according to an exemplary embodiment of this specification.
[0095] Figure 3 This is a flowchart illustrating an indoor navigation method according to an exemplary embodiment of this specification.
[0096] Figure 4 This specification is a hardware structure diagram of a computer device housing an indoor navigation device, according to an exemplary embodiment.
[0097] Figure 5 This is a block diagram illustrating an indoor navigation device according to an exemplary embodiment of this specification.
[0098] Figure 6 This is a block diagram illustrating another indoor navigation device according to an exemplary embodiment of this specification. Detailed Implementation
[0099] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.
[0100] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.
[0101] It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."
[0102] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points shall be provided for users to choose to authorize or refuse.
[0103] Taking the delivery scenario as an example, such as Figure 1 The diagram shown illustrates a delivery scenario according to an exemplary embodiment of this specification. The service provider establishes a server 111 and provides a client 122 to the user 121, allowing the user 121 to access the services provided by the service provider through the client 122. The service provider also provides a client 132 to the merchant 131, enabling the merchant 131 to access the services provided by the service provider through the client 132, for example, by opening a store where users can place orders. Each user can transact with the merchant through the client, and the service provider can generate transaction orders and delivery slips based on these transactions, allocating delivery capacity to the delivery slips. In this embodiment, delivery capacity refers to the party with delivery capabilities, including but not limited to delivery personnel, such as riders. Figure 1 The delivery capacity 141 and its client 142 are shown in the figure; in other examples, the delivery capacity may also include unmanned delivery electronic devices, such as drones, unmanned vehicles, etc., which may run a client that communicates with the server.
[0104] For example, after a user places an order, the server (such as an e-commerce platform or other cooperating delivery platform) needs to generate a corresponding delivery order for the delivered item and assign the delivery order to the delivery capacity. Therefore, after accepting the delivery order, the delivery capacity will go to the pickup location to collect the goods and deliver them to the user's designated location after successful pickup.
[0105] For example, in the food delivery scenario, a user places an order with a physical store on the food delivery platform through the client. The food delivery platform generates the corresponding food delivery order and assigns a delivery order to the delivery capacity (such as the client device used by the rider). The rider then goes to the physical store (i.e., the pick-up location of the delivered item) to pick up the food and delivers it to the location specified by the user.
[0106] For new riders, finding merchants is a significant challenge when heading to pickup locations, as most are unfamiliar with the delivery area. Furthermore, existing map navigation systems are ineffective in indoor environments like shopping malls, requiring riders to spend considerable time locating merchants. Even riders without delivery capabilities have a need for navigation in indoor environments.
[0107] Based on this, the embodiments of this specification provide an indoor navigation solution that can provide users with indoor navigation and improve the efficiency of users finding merchant locations. The embodiments of this specification will now be described in detail.
[0108] like Figure 2A As shown, Figure 2A This is a flowchart illustrating an indoor navigation method according to an exemplary embodiment, comprising the following steps:
[0109] In step 202, for users in an indoor environment, determine the current merchant at the user's location and the destination merchant the user is about to visit.
[0110] In step 204, obtain the navigation map structure corresponding to the indoor environment.
[0111] The navigation map structure includes multiple nodes, each node representing a point of interest (POI) within the indoor environment; the navigation map structure is established by identifying the relative positional relationships between POIs using image data captured from the indoor environment.
[0112] In step 206, based on the nodes corresponding to the current merchant and the destination merchant in the navigation map structure, at least one path node between the current merchant and the destination merchant is obtained using the navigation map structure.
[0113] In step 208, based on the at least one transit node, obtain navigation data from the current merchant to the destination merchant.
[0114] This embodiment's method can be implemented after the delivery capacity is located in an indoor environment. The indoor environment in this embodiment includes, but is not limited to, shopping malls, supermarkets, food courts, or office buildings. There are multiple ways to determine whether the delivery capacity is located indoors. For example, the delivery capacity client can determine if the delivery capacity is indoors by acquiring satellite signals collected by its device; it can also determine if the delivery capacity is indoors by acquiring changes in lighting detected by its device; it can also determine if the delivery capacity is indoors by acquiring wireless communication signals collected by its device; or it can be a combination of the above methods.
[0115] This embodiment can pre-construct a corresponding navigation map structure for the indoor environment that requires indoor navigation. The navigation map structure can include various graph structures, which can be non-heterogeneous information graphs or heterogeneous information graphs, such as meta-graphs. A meta-graph is a graph structure that contains multi-dimensional information. In this embodiment, it is used to describe the relevant positional relationships and distances of various points of interest in the indoor environment.
[0116] In this embodiment, the points of interest in the indoor environment, as nodes in the graph, can include various types; for example, they can include entrance / exit nodes representing entrances / exits to the indoor environment, or merchant nodes representing merchants located within the indoor environment. A merchant node can be any merchant in the indoor environment, or it can be... Figure 1 The illustrated embodiment includes merchants providing transactions on the platform side.
[0117] Optionally, the indoor environment may have multiple floors, and nodes can carry the floor information of the floor to which the node belongs, so that the navigation map structure reflects the information of each floor; in this case, the navigation map structure may also include floor connection nodes, such as stair nodes representing stairs, and / or elevator nodes representing elevators, where elevators may include escalators or elevators, etc.
[0118] In the navigation graph structure, each node can carry one or more types of node information as needed, such as node type information, node identifier (ID), node name, node creation time, or navigation information (e.g., an image) used to display the node. For example, a merchant node can carry node type information, merchant name, address information, merchant identifier, floor information, merchant image, or node creation time, etc. Similarly, floor connection nodes and entrance / exit nodes can also carry any of the above information.
[0119] In a navigation graph structure, the edges between two adjacent nodes can carry one or more types of edge information. For example, they can carry information representing the time required to move between two adjacent nodes, i.e., the time required to move from one node to the adjacent node.
[0120] Optionally, the navigation map structure in this embodiment can be constructed using various data sources. For example, basic information about the indoor environment can be obtained through existing map data or publicly available information about the indoor environment. For instance, some shopping malls and other indoor environments already have publicly available basic information, such as the number of floors or information about each merchant in the indoor environment (e.g., name or address information). This information is mostly textual and may not be sufficient to describe the relative location and distance of each POI in the indoor environment. Alternatively, specialized personnel can collect information about each POI on-site.
[0121] This embodiment can be constructed by combining image data captured of the indoor environment. The image data here can include image files; since video is essentially composed of multiple consecutive frames of images, the image data can also include video files.
[0122] To reduce costs, in some cases, the image data of the indoor environment is obtained in the following way:
[0123] Acquire image data of all or part of the indoor environment taken by any one of the following clients: delivery capacity client, user client, or merchant client.
[0124] This embodiment can involve data collection by a user using a user client, by delivery personnel using a delivery personnel client, or by merchants using a merchant client. For example, when delivery personnel need to pick up delivered items from a merchant, their client can provide image capture and uploading capabilities. This capability can be activated upon entering an indoor environment, allowing the delivery personnel to collect image data of all or part of the Points of Interest (POIs) within the indoor environment as they move. The delivery personnel client can then send the collected data to a server, which will then build a navigation map structure for the indoor environment.
[0125] Similarly, the merchant client can provide image capture and upload functions, allowing merchants to collect image data of POIs within the indoor environment. Optionally, the merchant client can allow merchants to open stores on the platform. The merchant client can provide this image capture and upload function when merchants open stores, enabling them to submit the collected image data when submitting merchant information. Merchants can submit image data of their store and its surrounding area, or they can submit image data of a larger range of POIs. This embodiment does not limit this.
[0126] In some examples, the navigation graph structure is established as follows:
[0127] Acquire one or more frames of images of the indoor environment and identify points of interest (POIs) in the images;
[0128] Based on the identified positional relationships of each POI in one or more image frames, determine the relative positional relationships of each POI in the indoor environment; and / or,
[0129] Obtain a video file of the indoor environment, and extract multiple frames and the shooting time of each frame from the video file;
[0130] After identifying one or more POIs in the image, the movement duration between each POI in each frame is determined based on the capture time of each frame.
[0131] In this embodiment, image data provided by multiple user clients can be obtained, and the above embodiments can be executed for each image data to establish the navigation map structure. For example, the user client provides a frame of image, which is an image taken by the user of their store and its surroundings. One or more POIs can be identified from this frame of image using various image recognition technologies.
[0132] In some examples, identifying one or more POIs in the image includes:
[0133] Identify characters in the image, determine one or more store names based on the identified characters, and use the store names to determine the corresponding merchant POI; and / or,
[0134] Identify staircase POIs or elevator POIs in the image.
[0135] One method for recognizing characters in images is through OCR (optical character recognition) text recognition. Shopfronts typically display the shop's logo (logotype). OCR can identify multiple characters, allowing the determination of the shop name based on these characters. For example, a shop name database can be pre-built, and the recognized characters can be matched against the shop names in the database to determine the corresponding shop name. Each shop name corresponds to a merchant POI.
[0136] For stairs or elevators, stair detection algorithms or elevator detection algorithms can be used for identification. Alternatively, a stair recognition model can be pre-built for identifying stairs, and an elevator recognition model can be pre-built for identifying elevators.
[0137] In practical applications, if a user provides a single image frame, one or more Points of Interest (POIs) can be identified from that frame. Taking multiple POIs as an example, the positional relationship of each POI within that frame can be determined. For instance, the relative positional relationship of each POI in the actual environment can be determined. For example, if three shop names, A, B, and C, are identified from a single image frame, and their positional relationship within the frame is determined to be ABC, then their positional relationship in the actual environment can also be determined. Based on this, corresponding nodes nodeA, nodeB, and nodeC can be created for A, B, and C respectively, and the relationships between these nodes can be established.
[0138] Optionally, one or more users can provide multiple frames of images; for example, if three POIs identified from another frame are C, D, and E, and since they share the same C with the ABC identified in the previous example, the relative positions of A, B, C, D, and E can be determined by combining ABC. Corresponding nodes nodeD and nodeE can be created for D and E, and associations can be established between nodeC, nodeD, and nodeE.
[0139] The movement duration of adjacent nodes among the above 5 nodes can be implemented in various ways. For example, in a real-world environment, if two adjacent shops are in a single image frame, it can be determined that the movement duration of these two adjacent shops is relatively short. For example, it can be set to a fixed movement duration, such as 3 seconds, as needed.
[0140] In other examples, the image data provided by the user can be a video file, i.e., a video filmed by the user while walking indoors. Therefore, the video file contains multiple frames, each corresponding to a shooting time. The movement duration between different Points of Interest (POIs) can be determined by the shooting time interval between each frame. For example, if F is identified from image p1 and G is identified from image p2, the movement duration from F to G can be determined based on the time interval between the shooting times of image p1 and image p2.
[0141] like Figure 2B The diagram shown is a schematic representation of some nodes in a navigation map structure according to an exemplary embodiment of this specification. For example, merchant BSK uploads a video file, and through OCR and elevator detection algorithms, the escalator and two other shops XBK and XC are identified. Furthermore, the path information is identified from the video file: entrance → XBK → escalator → XC → BSK. The corresponding navigation map structure is thus established. Figure 2C As shown. In this way, multiple paths can be obtained through multiple image data, and the entire navigation map structure of the indoor environment can be constructed based on these multiple paths.
[0142] Image analysis can yield a variety of information, including the time of appearance of adjacent merchants during video capture, video or images of the paths between merchants, the required movement time, and video or images of merchants being passed through. This information can be used as elements of a meta-graph. Based on the meta-graph, video / images / text between any two nodes in the graph can be generated. This embodiment reduces the data collection requirements and improves store coverage.
[0143] Optionally, data calibration can be performed on the information in the navigation map structure based on the user. For example, in a delivery scenario, indoor navigation is used to provide navigation to delivery personnel. However, image data uploaded by merchants and other non-delivery personnel differs from the behavioral characteristics of delivery personnel. Therefore, data calibration can be performed based on delivery personnel. As an example, for multiple image data sets of the same area, image processing algorithms can be used to first segment the video, and then the repeated segments after segmentation can be used for data calibration. For instance, the video uploaded by delivery personnel includes: XBK→XC; the video uploaded by merchants includes: entrance→XBK→escalator→XC→BSK. Both videos include the same movement from XBK to XC. Considering factors such as walking speed during filming, the timeline of the merchant's uploaded video file can be adjusted based on the delivery capacity video. For example, obtain the movement time 1 of XBK to XC in the delivery capacity video and the movement time 2 of XBK to XC in the merchant's video. Based on the ratio of movement time 1 to movement time 2 (e.g., 1.25, indicating movement time 1 is faster), the overall time of the merchant's video can be adjusted, speeding up the time by 1.25 times. This ensures that data from different sources can match the actual working conditions of the delivery capacity.
[0144] In some examples, the navigation graph structure is updated in the following way:
[0145] If the relocation information of the target merchant is obtained, the merchant node corresponding to the target merchant is found in the navigation map structure, and one or more node information carried by the merchant node corresponding to the target merchant is cleared.
[0146] In this embodiment, when a merchant's location changes, such as relocation, the server can collect the changes in merchant information and adjust the navigation map structure. For example, the node status corresponding to the changed merchant can be set to indicate that the node does not currently correspond to a merchant, such as a pending status. The node does not carry other information; optionally, relevant time information can be retained.
[0147] For example: In a path XBK→XC→BSK in the navigation map structure, if XC is relocated, then:
[0148] Before the update: (XBK)-15s-(XC)-15s-(BSK)
[0149] Updated: (XBK)-15s-(Pending)-15s-(BSK).
[0150] Using the above embodiments, the server can store the navigation map structure of each indoor environment. When a user enters an indoor environment, the server can provide the user with indoor navigation services. Here, "user" includes, but is not limited to, the aforementioned delivery capacity, and can also be any user. The server can provide a user client, which determines that the user needs indoor navigation services and informs the server.
[0151] Optional, Figure 2A The illustrated embodiments can be applied to either the server or the client. When applied to the server, the server can obtain navigation data and push it to the client for display. When applied to the client, the client can have a built-in navigation map structure of the indoor environment and use this structure to navigate the user.
[0152] Optionally, the user client can determine that the user is currently in an indoor environment in a variety of ways, such as by using the user's geographical location information before entering the indoor environment, or by the change in light when entering the indoor environment from the outdoors, or by using wireless signal information collected by the device. This embodiment does not limit this method.
[0153] Optionally, the electronic device containing the user client can collect wireless signal information, such as Wi-Fi (Wireless Fidelity), to locate the user's position within an indoor environment. For example, related technologies establish wireless signal fingerprints for each merchant within the indoor environment. By matching the wireless signal information collected by the user client with the wireless signal fingerprints of each merchant, the current merchant at the user's current location can be determined. Alternatively, the user client can also provide an input function, allowing the user to input current merchant information, and the current merchant at the user's current location can be determined based on the received current merchant information.
[0154] Optionally, there are multiple ways to determine the destination merchant a user is about to visit. For example, the client can receive the destination merchant information input by the user and determine the destination merchant based on the received information. In scenarios providing navigation services for delivery capacity, the current delivery order can be obtained, and the destination merchant the user is about to visit can be determined based on the merchant address in the current delivery order. In this case, the user may be about to visit one or more destination merchants. If there are multiple destination merchants, the order of the multiple destination merchants can be determined, and the path from the current merchant to each destination merchant can be determined.
[0155] In some examples, the information carried by the edges between adjacent nodes in the navigation graph structure includes: the movement time between the adjacent nodes, and the node carrying node information;
[0156] The step of obtaining at least one path node from the current merchant to the destination merchant using the navigation map structure includes:
[0157] Using the information carried by the edges between nodes in the navigation graph structure, the optimal path with the shortest travel time from the current merchant to the destination merchant is obtained;
[0158] Based on the node information of each node traversed by the optimal path, at least one path node is determined.
[0159] In this embodiment, the optimal path between two nodes can be retrieved using the nodes and edges in the navigation graph structure. The nodes in the navigation graph structure carry node information. For the current merchant and the destination merchant, information such as the merchant name or merchant identifier can be obtained. By searching the navigation graph structure, the merchant node corresponding to the current merchant and the merchant node corresponding to the destination merchant can be obtained. Once these two merchant nodes are determined, the optimal path between them can be retrieved from the graph structure. Optionally, there may be multiple optimal paths. In the case of multiple paths, one can be automatically selected as needed, or multiple optimal paths can be displayed to the user through the client for the user to choose from. This embodiment does not limit this. Based on the optimal path, the nodes along the route between the two merchants can be determined. These nodes include the merchant node corresponding to the current merchant, the merchant node corresponding to the target merchant, and any node passed between these two merchant nodes, which can be all or some of the nodes between them.
[0160] As an example, the determined optimal path is: Merchant node A of the current merchant → Merchant node 1 → Merchant node 2 → ... Escalator node ... → Merchant node n → Merchant node B corresponding to the destination merchant. Here, the arrow symbol "→" represents the edge between two nodes. The navigation map structure is constructed based on the physical merchants within the indoor environment. In the actual environment, the merchants are relatively close to each other, therefore the determined optimal path includes many nodes, such as the n nodes between node A and node B mentioned above. For example, merchant node 1 and merchant node 2 correspond to two merchants that are relatively close in the actual environment, and their movement time is 5 seconds. Path nodes are used to obtain path navigation data for display on the client-side front end. Based on this, in this embodiment, at least one path node can also be determined based on the node information of each node traversed by the optimal path. For example, the path nodes determined between merchant node A and merchant node B could be merchant node 8, elevator node, and merchant node 9, that is, the navigation data could be: Merchant node A → Merchant node 8 → Escalator node → Merchant node 9 → Merchant node B.
[0161] The identified path nodes can be all nodes traversed by the optimal path, or only a subset of them. For example, in the above example, there are nodes that are very close together, so some nodes can be removed. After removing some nodes, the travel time between two adjacent path nodes can be calculated based on the travel times of all nodes traversed between these two adjacent path nodes. For example, regarding the travel time from merchant node A to merchant node 8 in the above example, there are 9 nodes in this path, which can be calculated by adding the travel times of each of these 9 nodes.
[0162] There are various ways to determine the path nodes, and this embodiment does not limit this. For example, the number of path nodes or the travel time between path nodes can be set as needed.
[0163] As an example, determining at least one transit node may include:
[0164] At least one path node is selected based on preset guidance conditions, which include any of the following:
[0165] (1) The travel time between the two transit nodes is within the set time range;
[0166] The duration range set in this embodiment can be configured as needed. For example, an upper and / or lower duration limit can be set, but this embodiment does not limit this. By setting this duration range, some closely spaced nodes can be removed when the optimal path nodes are densely packed, thus reducing the number of nodes passed through.
[0167] (2) When the current merchant and the destination merchant are located on different floors, the nodes along the route include the floor connection nodes between the floors;
[0168] As described in the foregoing embodiments, in indoor environments with multiple floors, the navigation map structure includes floor connection nodes. If the current merchant and the destination merchant are located on different floors, the user needs to cross floors via stairs or elevators. Therefore, the floor nodes in the optimal path can be retained without being removed, making it convenient to guide the user across floors during navigation. Optionally, the path nodes may also include merchant nodes adjacent to the floor connection nodes, allowing the user to identify the location of stairs or elevators through these merchant nodes, and providing direction guidance after the user crosses floors via stairs or elevators.
[0169] (3) The popularity of the merchant node is higher than the set popularity condition.
[0170] In this embodiment, "popularity" represents a user's ability to recognize the merchant corresponding to a merchant node. Merchants with higher popularity are easier for users to identify while browsing. Optionally, merchant popularity can be obtained in various ways, such as through the merchant's order volume, which is positively correlated with popularity; or through the click-through rate of the merchant's store on the platform, which is also positively correlated with popularity. In practical applications, the popularity information of each merchant can be obtained as the node information of the corresponding merchant node and stored in the navigation graph structure.
[0171] In practical applications, it can be any of the above conditions, or a combination of at least two of the above conditions. This embodiment does not limit this.
[0172] In some examples, the navigation data includes any of the following:
[0173] The node information of the nodes passed through, the edges between adjacent nodes passed through, or the movement time of adjacent nodes passed through.
[0174] In this embodiment, the node information in the navigation data may not be all the node information carried by the nodes in the navigation graph structure. For example, the node information may include node information that is suitable for display on the client and for navigation, including but not limited to node names and node images.
[0175] like Figure 2C The diagram shown is a schematic representation of a client using navigation data to navigate, according to an exemplary embodiment of this specification. Figure 2C The navigation data shows the path from merchant "MDL" to merchant "xxmiancun". This navigation data shows the path between the two, including various nodes along the way, namely merchant node MDL, merchant node CCDK, merchant node KMBS, merchant node Mmner, merchant node LXJ, escalator node... (other nodes are not shown here for the sake of example). It also shows the name and image (store logo image) of each node along the way, and shows the image (store facade image) of the merchant node CCDK where the user is currently located.
[0176] Optionally, during the navigation process using navigation data, the client can also continuously detect whether the user is on the return navigation path. For example, the client can continuously determine whether the user will deviate from the navigation data by using the wireless signal information collected by the device. If so, the client can use the current merchant at the newly determined location of the user to execute the aforementioned steps 202 to 204 again, thereby planning new navigation data.
[0177] like Figure 3 The diagram shown is a schematic representation of an indoor navigation method according to an exemplary embodiment of this specification. The method includes:
[0178] In step 302, it is determined that the user is currently in an indoor environment;
[0179] In step 304, navigation data sent by the server is obtained from the current merchant at the user's current location to the destination merchant the user is about to visit. The navigation data is determined by the server after identifying the nodes between the current merchant and the destination merchant from the navigation map structure corresponding to the indoor environment, and obtaining at least one path node. The navigation map structure is established by identifying the relative positional relationships between POIs using image data taken of the indoor environment.
[0180] In step 306, navigation is performed using the navigation data.
[0181] In some examples, the method further includes:
[0182] Determine the current merchant at the user's current location, and / or determine the destination merchant the user is about to visit;
[0183] The current merchant at the user's current location is determined by any of the following methods:
[0184] Acquire wireless signal information and use the wireless signal information to determine the current merchant at the user's current location; or, receive current merchant information input by the user and determine the current merchant at the user's current location based on the received current merchant information.
[0185] The destination merchant that the user is about to visit is determined by any of the following methods:
[0186] When the user is a delivery driver, the system utilizes the user's current order and determines the destination merchant based on the merchant address in the current order; or, it receives the destination merchant information input by the user and determines the destination merchant based on the received destination merchant information.
[0187] In some examples, the use of navigation data for navigation includes:
[0188] Display any of the following information from the navigation data: node information of the passed-through nodes, edges between adjacent passed-through nodes, or travel time of adjacent passed-through nodes.
[0189] In some examples, the method further includes:
[0190] During the navigation process using the navigation data, the current merchant at the user's current location is determined. If it is determined that the user has deviated from the navigation data, the process returns to the step of obtaining the navigation data sent by the server from the current merchant at the user's current location to the destination merchant the user is about to visit.
[0191] In some examples, the navigation map structure includes merchant nodes representing businesses; wherein, in cases where the indoor environment has multiple floors, the navigation map structure also includes floor connection nodes, which include connection nodes representing stairs and / or connection nodes representing elevators.
[0192] In some examples, the information carried by the edges between adjacent nodes in the navigation graph structure includes: the movement time between the adjacent nodes, and the node carrying node information;
[0193] The step of obtaining at least one path node from the current merchant to the destination merchant using the navigation map structure includes:
[0194] Using the information carried by the edges between nodes in the navigation graph structure, the optimal path with the shortest travel time from the current merchant to the destination merchant is obtained;
[0195] Based on the node information of each node traversed by the optimal path, at least one path node is determined.
[0196] In some examples, determining at least one transit node includes:
[0197] At least one path node is selected based on preset guidance conditions, which include any of the following:
[0198] The travel time between the two transit nodes is within the set time range;
[0199] When the current merchant and the destination merchant are located on different floors, the nodes along the route include floor connection nodes between floors and merchant nodes adjacent to the floor connection nodes; or,
[0200] The popularity of the merchant node is higher than the set popularity condition.
[0201] In some examples, the node information includes any of the following: node type, node name, node identifier, creation time, floor or navigation information.
[0202] In some examples, the image data includes multiple frames of images; the navigation graph structure is established as follows:
[0203] Acquire multiple frames of images of the indoor environment and identify one or more points of interest (POIs) in the images;
[0204] For multiple Points of Interest (POIs) in a single image frame, determine the relative positional relationship of each POI within the indoor environment based on their positional relationships within the image; and / or,
[0205] Obtain a video file of the indoor environment, and extract multiple frames and the shooting time of each frame from the video file;
[0206] After identifying one or more POIs in the image, the movement duration between each POI in each frame is determined based on the capture time of each frame.
[0207] In some examples, identifying one or more POIs in the image includes:
[0208] Identify characters in the image, determine one or more store names based on the identified characters, and use the store names to determine the corresponding merchant POI; and / or,
[0209] Identify staircase POIs or elevator POIs in the image.
[0210] In some examples, the navigation graph structure is updated in the following way:
[0211] If the relocation information of the target merchant is obtained, the merchant node corresponding to the target merchant is found in the navigation map structure, and the node information carried by the merchant node corresponding to the target merchant is cleared.
[0212] In some examples, the method further includes:
[0213] An image data upload function is provided, which acquires image data of all or part of the indoor environment taken by the user and sends it to the server so that the server can build the navigation map structure.
[0214] Corresponding to the embodiments of the aforementioned indoor navigation methods, this specification also provides embodiments of indoor navigation devices and the computer equipment on which they are applied.
[0215] The embodiments of the indoor navigation device described in this specification can be applied to computer equipment, such as servers or terminal devices. The device embodiments can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of the indoor navigation system loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 4 The diagram shown is a hardware structure diagram of the computer equipment containing the indoor navigation device described in this manual. (Except for...) Figure 4In addition to the processor 410, memory 430, network interface 420, and non-volatile memory 440 shown, the computer device in which the indoor navigation device 431 is located in the embodiment may also include other hardware depending on the actual function of the computer device, which will not be described in detail here.
[0216] like Figure 5 As shown, Figure 5 This is a block diagram illustrating an indoor navigation device according to an exemplary embodiment of this specification, the device comprising:
[0217] The merchant determination module is used to: determine the current merchant at the user's location and the destination merchant the user is about to visit for users in an indoor environment;
[0218] The navigation map structure acquisition module is used to: acquire the navigation map structure corresponding to the indoor environment; wherein, the navigation map structure includes multiple nodes, each node representing a POI within the indoor environment; the navigation map structure is established by identifying the relative positional relationships between POIs using image data captured of the indoor environment;
[0219] The node acquisition module is used to: based on the nodes corresponding to the current merchant and the destination merchant in the navigation graph structure, acquire at least one path node between the current merchant and the destination merchant using the navigation graph structure;
[0220] The navigation data acquisition module is used to: acquire navigation data from the current merchant to the destination merchant based on the at least one transit node.
[0221] In some examples, the navigation map structure includes merchant nodes representing businesses; wherein, in cases where the indoor environment has multiple floors, the navigation map structure also includes floor connection nodes, which include connection nodes representing stairs and / or connection nodes representing elevators.
[0222] In some examples, the information carried by the edges between adjacent nodes in the navigation graph structure includes: the movement time between the adjacent nodes, and the node carrying node information;
[0223] The node acquisition module is also used for:
[0224] Using the information carried by the edges between nodes in the navigation graph structure, the optimal path with the shortest travel time from the current merchant to the destination merchant is obtained;
[0225] Based on the node information of each node traversed by the optimal path, at least one path node is determined.
[0226] In some examples, the node acquisition module is also used for:
[0227] At least one path node is selected based on preset guidance conditions, which include any of the following:
[0228] The travel time between the two transit nodes is within the set time range;
[0229] When the current merchant and the destination merchant are located on different floors, the nodes along the route include floor connection nodes between floors and merchant nodes adjacent to the floor connection nodes; or,
[0230] The popularity of the merchant node is higher than the set popularity condition.
[0231] In some examples, the node information includes any of the following: node type, node name, node identifier, creation time, floor or navigation information.
[0232] In some examples, the navigation graph structure acquisition module is also used for:
[0233] The navigation graph structure is established as follows:
[0234] Acquire one or more frames of images of the indoor environment and identify points of interest (POIs) in the images;
[0235] Based on the identified positional relationships of each POI in one or more image frames, determine the relative positional relationships of each POI in the indoor environment; and / or,
[0236] Obtain a video file of the indoor environment, and extract multiple frames and the shooting time of each frame from the video file;
[0237] After identifying one or more POIs in the image, the movement duration between each POI in each frame is determined based on the capture time of each frame.
[0238] In some examples, the navigation graph structure acquisition module is also used for:
[0239] Identify characters in the image, determine one or more store names based on the identified characters, and use the store names to determine the corresponding merchant POI; and / or,
[0240] Identify staircase POIs or elevator POIs in the image.
[0241] In some examples, the image data of the indoor environment is obtained in the following manner:
[0242] Acquire image data of all or part of the indoor environment taken by any one of the following clients: delivery capacity client, user client, or merchant client.
[0243] In some examples, the user includes delivery capacity, and the destination merchant to which the user is about to go is determined in the following way:
[0244] Obtain the current delivery order of the delivery capacity, and determine the destination merchant that the user is about to visit based on the merchant address in the current delivery order.
[0245] In some examples, the navigation graph structure is updated in the following way:
[0246] If the relocation information of the target merchant is obtained, the merchant node corresponding to the target merchant is found in the navigation map structure, and the node information carried by the merchant node corresponding to the target merchant is cleared.
[0247] In some examples, the navigation data includes any of the following:
[0248] The node information of the nodes passed through, the edges between adjacent nodes passed through, or the movement time of adjacent nodes passed through.
[0249] In some examples, the device further includes a transmitting module for:
[0250] The navigation data is sent to the user client so that the user client can use the navigation data for navigation.
[0251] like Figure 6 As shown, Figure 6 This is a block diagram illustrating another indoor navigation device according to an exemplary embodiment of this specification, the device comprising:
[0252] The determination module is used to: determine the user's current indoor environment;
[0253] The acquisition module is used to: acquire navigation data sent by the server from the current merchant at the user's current location to the destination merchant the user is about to visit; the navigation data is determined by the server after identifying the nodes between the current merchant and the destination merchant from the navigation map structure corresponding to the indoor environment, and acquiring at least one path node; the navigation map structure is established by identifying the relative positional relationships between POIs using image data taken of the indoor environment;
[0254] A navigation module is used to: perform navigation using the navigation data.
[0255] In some examples, the determining module is also used for:
[0256] Determine the current merchant at the user's current location, and / or determine the destination merchant the user is about to visit;
[0257] The current merchant at the user's current location is determined by any of the following methods:
[0258] The system acquires wireless signal information collected by the client and uses the wireless signal information to determine the current merchant at the user's current location; or, it receives current merchant information input by the user and determines the current merchant at the user's current location based on the received current merchant information.
[0259] The destination merchant that the user is about to visit is determined by any of the following methods:
[0260] When the user is a delivery driver, the system utilizes the user's current order and determines the destination merchant based on the merchant address in the current order; or, it receives the destination merchant information input by the user and determines the destination merchant based on the received destination merchant information.
[0261] In some examples, the navigation module is also used for:
[0262] Display any of the following information from the navigation data: node information of the passed-through nodes, edges between adjacent passed-through nodes, or travel time of adjacent passed-through nodes.
[0263] In some examples, the device further includes a deviation determination module for:
[0264] During the navigation process using the navigation data, the current merchant at the user's current location is determined. If it is determined that the user has deviated from the navigation data, the process returns to the step of obtaining the navigation data sent by the server from the current merchant at the user's current location to the destination merchant the user is about to visit.
[0265] In some examples, the navigation map structure includes merchant nodes representing businesses; wherein, in cases where the indoor environment has multiple floors, the navigation map structure also includes floor connection nodes, which include connection nodes representing stairs and / or connection nodes representing elevators.
[0266] In some examples, the information carried by the edges between adjacent nodes in the navigation graph structure includes: the movement time between the adjacent nodes, and the node carrying node information;
[0267] The server is also used for:
[0268] Using the information carried by the edges between nodes in the navigation graph structure, the optimal path with the shortest travel time from the current merchant to the destination merchant is obtained;
[0269] Based on the node information of each node traversed by the optimal path, at least one path node is determined.
[0270] In some examples, the server is also used for:
[0271] At least one path node is selected based on preset guidance conditions, which include any of the following:
[0272] The travel time between the two transit nodes is within the set time range;
[0273] When the current merchant and the destination merchant are located on different floors, the nodes along the route include floor connection nodes between floors and merchant nodes adjacent to the floor connection nodes; or,
[0274] The popularity of the merchant node is higher than the set popularity condition.
[0275] In some examples, the node information includes any of the following: node type, node name, node identifier, creation time, floor or navigation information.
[0276] In some examples, the server is also used to establish the navigation graph structure in the following ways:
[0277] Acquire one or more frames of images of the indoor environment and identify points of interest (POIs) in the images;
[0278] Based on the identified positional relationships of each POI in one or more image frames, determine the relative positional relationships of each POI in the indoor environment; and / or,
[0279] Obtain a video file of the indoor environment, and extract multiple frames and the shooting time of each frame from the video file;
[0280] After identifying one or more POIs in the image, the movement duration between each POI in each frame is determined based on the capture time of each frame.
[0281] In some examples, the server is also used for:
[0282] Identify characters in the image, determine one or more store names based on the identified characters, and use the store names to determine the corresponding merchant POI; and / or,
[0283] Identify staircase POIs or elevator POIs in the image.
[0284] In some examples, the navigation graph structure is updated in the following way:
[0285] If the relocation information of the target merchant is obtained, the merchant node corresponding to the target merchant is found in the navigation map structure, and the node information carried by the merchant node corresponding to the target merchant is cleared.
[0286] In some examples, the device also includes an upload module for:
[0287] An image data upload function is provided, which acquires image data of all or part of the indoor environment taken by the user and sends it to the server so that the server can build the navigation map structure.
[0288] The specific implementation process of the functions and roles of each module in the above-mentioned indoor navigation device can be found in the implementation process of the corresponding steps in the above-mentioned indoor navigation method, and will not be repeated here.
[0289] Accordingly, this specification also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the aforementioned indoor navigation method embodiments.
[0290] Accordingly, embodiments of this specification also provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the indoor navigation method embodiment.
[0291] Accordingly, embodiments of this specification also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the indoor navigation method embodiments.
[0292] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the solution in this specification according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0293] The above embodiments can be applied to one or more computer devices. The computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. The hardware of the computer device includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.
[0294] The computer device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.
[0295] The computer equipment may also include network equipment and / or user equipment. The network equipment includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.
[0296] The network in which the computer device is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, and virtual private network (VPN).
[0297] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.
[0298] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.
[0299] The terms "specific example" or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with the embodiments or examples, which are included in at least one embodiment or example of this specification. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0300] Other embodiments of this specification will readily occur to those skilled in the art upon consideration of the specification and practice of the invention claimed herein. This specification is intended to cover any variations, uses, or adaptations that follow the general principles of this specification and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this specification are indicated by the following claims.
[0301] It should be understood that this specification is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this specification is limited only by the appended claims.
[0302] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.
Claims
1. An indoor navigation method, the method comprising: For users in indoor environments, determine the current merchant at the user's location and the destination merchant the user is about to visit; Obtain the navigation map structure corresponding to the indoor environment; wherein the navigation map structure includes multiple nodes, each node representing a POI within the indoor environment; the navigation map structure is established by identifying the relative positional relationships between POIs using image data captured from the indoor environment, including: determining the relative positional relationships of each POI in the indoor environment based on the positional relationships of each POI identified from one or more frames of images of the indoor environment; and / or, identifying one or more POIs in multiple frames of images obtained from a video file of the indoor environment, and determining the movement duration between each POI in each frame of images based on the capture time of each frame of images; Based on the nodes corresponding to the current merchant and the destination merchant in the navigation graph structure, at least one path node between the current merchant and the destination merchant is obtained using the navigation graph structure; Based on the at least one transit node, obtain navigation data from the current merchant to the destination merchant.
2. The method according to claim 1, wherein the navigation graph structure includes merchant nodes representing merchants; wherein, In the case where the indoor environment has multiple floors, the navigation map structure further includes floor connection nodes, which include connection nodes representing stairs and / or connection nodes representing elevators.
3. The method according to claim 1 or 2, wherein the information carried by the edges between adjacent nodes in the navigation graph structure includes: The movement time between adjacent nodes, and the node carrying node information; The step of obtaining at least one path node from the current merchant to the destination merchant using the navigation map structure includes: Using the information carried by the edges between nodes in the navigation graph structure, the optimal path with the shortest travel time from the current merchant to the destination merchant is obtained; Based on the node information of each node traversed by the optimal path, at least one path node is determined.
4. The method according to claim 3, wherein determining at least one transit node comprises: At least one path node is selected based on preset guidance conditions, which include any of the following: The travel time between the two transit nodes is within the set time range; When the current merchant and the destination merchant are located on different floors, the nodes along the route include floor connection nodes between floors and merchant nodes adjacent to the floor connection nodes. or, The popularity of the merchant node is higher than the set popularity condition.
5. The method according to claim 3, wherein the node information includes any one of the following: node type, node name, node identifier, creation time, floor or navigation information.
6. The method according to claim 1, wherein identifying one or more POIs in the image comprises: Identify characters in the image, determine one or more store names based on the identified characters, and use the store names to determine the corresponding merchant POI; And / or, Identify staircase POIs or elevator POIs in the image.
7. The method according to claim 1, wherein the image data of the indoor environment captured is obtained in the following manner: Acquire image data of all or part of the indoor environment taken by any one of the following clients: delivery capacity client, user client, or merchant client.
8. The method according to claim 1, wherein the user includes delivery capacity, and the destination merchant to which the user is about to go is determined in the following manner: Obtain the current delivery order of the delivery capacity, and determine the destination merchant that the user is about to visit based on the merchant address in the current delivery order.
9. The method according to claim 2, wherein the navigation graph structure is updated in the following manner: If the relocation information of the target merchant is obtained, the merchant node corresponding to the target merchant is found in the navigation map structure, and one or more node information carried by the merchant node corresponding to the target merchant is cleared.
10. The method according to claim 1, wherein the navigation data includes any of the following: The node information of the nodes passed through, the edges between adjacent nodes passed through, or the movement time of adjacent nodes passed through.
11. The method according to claim 1 or 10, further comprising: The navigation data is sent to the user client so that the user client can use the navigation data for navigation.
12. An indoor navigation method, the method comprising: Determine that the user is currently in an indoor environment; Obtain navigation data sent by the server from the current merchant at the user's current location to the destination merchant the user is about to visit; The navigation data is determined by the server after identifying the nodes between the current merchant and the destination merchant from the navigation map structure corresponding to the indoor environment, and obtaining at least one path node. The navigation map structure is established by identifying the relative positional relationships between POIs using image data captured from the indoor environment. It includes: determining the relative positional relationship of each POI in the indoor environment based on the positional relationship of each POI identified from one or more frames of images of the indoor environment; and / or, identifying one or more POIs in multiple frames of images obtained from a video file of the indoor environment, and determining the movement duration between each POI in each frame of images based on the capture time of each frame of images. Navigation is performed using the navigation data.
13. The method according to claim 12, further comprising: Determine the current merchant at the user's current location, and / or determine the destination merchant the user is about to visit; The current merchant at the user's current location is determined by any of the following methods: Acquire wireless signal information and use the wireless signal information to determine the current merchant at the user's current location; or, receive current merchant information input by the user and determine the current merchant at the user's current location based on the received current merchant information. The destination merchant that the user is about to visit is determined by any of the following methods: When the user is a delivery driver, the system utilizes the user's current order and determines the destination merchant based on the merchant address in the current order; or, it receives the destination merchant information input by the user and determines the destination merchant based on the received destination merchant information.
14. The method according to claim 12, wherein the navigation using navigation data comprises: Display any of the following information from the navigation data: node information of the passed-through nodes, edges between adjacent passed-through nodes, or travel time of adjacent passed-through nodes.
15. The method according to claim 13, further comprising: During the navigation process using the navigation data, the current merchant at the user's current location is determined. If it is determined that the user has deviated from the navigation data, the process returns to the step of obtaining the navigation data sent by the server from the current merchant at the user's current location to the destination merchant the user is about to visit.
16. The method according to claim 12, wherein the navigation graph structure includes merchant nodes representing merchants; wherein, In the case where the indoor environment has multiple floors, the navigation map structure further includes floor connection nodes, which include connection nodes representing stairs and / or connection nodes representing elevators.
17. The method according to claim 12 or 16, wherein the information carried by the edges between adjacent nodes in the navigation graph structure includes: The movement time between adjacent nodes, and the node carrying node information; The step of obtaining at least one path node from the current merchant to the destination merchant using the navigation map structure includes: Using the information carried by the edges between nodes in the navigation graph structure, the optimal path with the shortest travel time from the current merchant to the destination merchant is obtained; Based on the node information of each node traversed by the optimal path, at least one path node is determined.
18. The method of claim 17, wherein determining at least one transit node comprises: At least one path node is selected based on preset guidance conditions, which include any of the following: The travel time between the two transit nodes is within the set time range; When the current merchant and the destination merchant are located on different floors, the nodes along the route include floor connection nodes between floors and merchant nodes adjacent to the floor connection nodes. or, The popularity of the merchant node is higher than the set popularity condition.
19. The method according to claim 17, wherein the node information includes any one of the following: node type, node name, node identifier, creation time, floor or navigation information.
20. The method of claim 12, wherein identifying one or more POIs in the image comprises: Identify characters in the image, determine one or more store names based on the identified characters, and use the store names to determine the corresponding merchant POI; And / or, Identify staircase POIs or elevator POIs in the image.
21. The method of claim 16, wherein the navigation graph structure is updated in the following manner: If the relocation information of the target merchant is obtained, the merchant node corresponding to the target merchant is found in the navigation map structure, and the node information carried by the merchant node corresponding to the target merchant is cleared.
22. The method according to claim 12, further comprising: An image data upload function is provided, which acquires image data of all or part of the indoor environment taken by the user and sends it to the server so that the server can build the navigation map structure.
23. An indoor navigation device, the device comprising: The merchant determination module is used to: determine the current merchant at the user's location and the destination merchant the user is about to visit for users in an indoor environment; A navigation map structure acquisition module is used to: acquire a navigation map structure corresponding to the indoor environment; wherein the navigation map structure includes multiple nodes, each node representing a POI within the indoor environment; the navigation map structure is established by identifying the relative positional relationships between POIs using image data captured of the indoor environment, including: determining the relative positional relationship of each POI in the indoor environment based on the positional relationships of each POI identified from one or more frames of images of the indoor environment; and / or, identifying one or more POIs in multiple frames of images acquired from a video file of the indoor environment, and determining the movement duration between each POI in each frame of images based on the capture time of each frame of images; The node acquisition module is used to: based on the nodes corresponding to the current merchant and the destination merchant in the navigation graph structure, acquire at least one path node between the current merchant and the destination merchant using the navigation graph structure; The navigation data acquisition module is used to: acquire navigation data from the current merchant to the destination merchant based on the at least one transit node.
24. An indoor navigation device, the device comprising: The determination module is used to: determine the user's current indoor environment; The acquisition module is used to: acquire navigation data sent by the server from the current merchant at the user's current location to the destination merchant the user is about to visit; the navigation data is determined by the server after determining the nodes between the current merchant and the destination merchant from the navigation map structure corresponding to the indoor environment, and acquiring at least one path node; The navigation map structure is established by identifying the relative positional relationships between POIs using image data captured from the indoor environment. It includes: determining the relative positional relationship of each POI in the indoor environment based on the positional relationship of each POI identified from one or more frames of images of the indoor environment; and / or, identifying one or more POIs in multiple frames of images obtained from a video file of the indoor environment, and determining the movement duration between each POI in each frame of images based on the capture time of each frame of images. A navigation module is used to: perform navigation using the navigation data.
25. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the method of any one of claims 1 to 22.
26. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any one of claims 1 to 22.
27. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 22.
Citation Information
Patent Citations
Indoor electronic map generation system, indoor navigation method and system
CN103674017A
Indoor positioning navigation method based on image recognition and pedestrian dead reckoning
CN113188546A
Indoor navigation method, device, equipment and medium
CN115567874A
Indoor navigation method, apparatus and device, and storage medium
CN115640468A
