Navigation map generation method, device, computer equipment and storage medium

Abstract

The present disclosure provides a navigation map generation method, device, computer equipment and storage medium, wherein the navigation map generation method includes: obtaining a scene image obtained by performing image acquisition on a target scene through an image acquisition device; generating a scene map based on the scene image; determining the position information of the image acquisition device in the target scene when acquiring the scene image based on the scene image, and determining the moving path of the image acquisition device in the target scene when performing image acquisition on the target scene based on the position information; and generating a navigation map of the target scene based on the moving path and the scene map. The embodiments of the present disclosure can automatically generate a navigation map of the target scene, thereby avoiding the high cost and low efficiency caused by manually drawing path information.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a navigation map generation method, device, computer equipment and storage medium. Background Art

[0002] Augmented Reality (AR) technology is a technology that integrates virtual information with the real world. It widely uses a variety of technical means such as multimedia and three-dimensional modeling to simulate computer-generated text, images, three-dimensional and other virtual information, and then displays them to users in the real world. With the continuous maturity of AR technology, AR technology has also been widely used in many scenarios. AR navigation, as an important aspect of AR technology application, requires pre-generation of navigation maps. During navigation, navigation is based on the navigation map. The current navigation map generation method has the problems of high cost and low efficiency. Summary of the invention

[0003] The embodiments of the present disclosure at least provide a navigation map generation method, apparatus, computer equipment and storage medium.

[0004] In a first aspect, an embodiment of the present disclosure provides a navigation map generation method, comprising: acquiring a scene image obtained by performing image acquisition on a target scene through an image acquisition device; generating a scene map based on the scene image; determining, based on the scene image, position information of the image acquisition device in the target scene when acquiring the scene image, and determining, based on the position information, a moving path of the image acquisition device in the target scene when performing image acquisition on the target scene; and generating a navigation map of the target scene based on the moving path and the scene map.

[0005] In this way, the moving path of the target scene and the collected scene image are collected by the image acquisition device to automatically generate a navigation map of the target scene map, thereby avoiding the high cost and low efficiency caused by manually drawing path information.

[0006] In a possible implementation, generating a scene map based on the scene image includes: using the scene image to generate a three-dimensional point cloud map of the target scene; the three-dimensional point cloud map includes: three-dimensional position information of multiple point cloud points in the target scene, and semantic information corresponding to the point cloud points; generating the scene map using the three-dimensional position information and the semantic information.

[0007] In this way, the scene map can be automatically generated, reducing the time required for manual scene map drawing, improving the efficiency of generating navigation maps, and reducing time costs.

[0008] In a possible implementation, the three-dimensional position information and the semantic information are used to generate the scene map, and based on the three-dimensional position information of the multiple point cloud points, the multiple point cloud points are respectively projected onto a preset plane to obtain a projection image; based on the projection image and the semantic information corresponding to the multiple point cloud points, the scene map is generated.

[0009] In this way, by projecting the point cloud points in the three-dimensional point cloud map onto a preset plane, a projection image is generated, and the projection image is used to generate a scene map. Compared with the scene map obtained by manual drawing, the scene map has higher accuracy.

[0010] In one possible implementation, the projection points corresponding to the multiple point cloud points, and the two-dimensional coordinate value of each projection point in the projection image; generating the scene map based on the projection image and the semantic information of each point cloud point, includes: clustering the projection points based on the two-dimensional coordinate values ​​of the projection points of the multiple point cloud points in the projection image, and the semantic information corresponding to the multiple point cloud points; generating the scene map based on the clustering results of the projection points.

[0011] In this way, by clustering the projection points, a scene map can be accurately generated, and various objects in the scene map are more detailed and accurate.

[0012] In a possible implementation, determining the movement path of the image acquisition device in the target scene when capturing images of the target scene based on the position information includes: based on the position information, projecting the sampling position of the image acquisition device when capturing the scene image onto a preset plane to obtain projection positions of multiple frames of scene images corresponding to the sampling positions on the preset plane; generating the movement path based on the projection positions of the sampling positions corresponding to the multiple frames of scene images on the preset plane and the timestamp corresponding to each frame of the scene image.

[0013] In this way, by projecting the sampling position onto the preset plane, the projection position is obtained, so that the moving path can be automatically generated.

[0014] In a possible implementation, the generating of the navigation map of the target scene based on the moving path and the scene map includes: determining a plurality of areas based on semantic information of each pixel in the scene map; determining accessibility information between each two adjacent areas based on the moving path; generating the navigation map of the target scene based on the accessibility information between each two adjacent areas; the accessibility information between each two adjacent areas includes: whether each two adjacent areas are reachable, and a reachable path when each two adjacent areas are reachable.

[0015] In this way, the accessibility information between areas can be used to plan navigation routes based on the navigation map, and the planning results are faster and more accurate.

[0016] In a possible implementation, it also includes: obtaining a navigation route generation instruction triggered by a user through an augmented reality (AR) device; the navigation route generation instruction carries the end point location information of the navigation route; and generating a navigation route based on the current position information of the AR device in the target scene, the end point location information, and the navigation map.

[0017] In this way, by obtaining the navigation route generation instruction triggered by the AR device, the navigation route generated based on the destination location information carried in the navigation route generation instruction, the current location information of the AR device, and the navigation map generation method provided by the embodiment of the present disclosure has lower costs and higher navigation accuracy.

[0018] In a possible implementation, it also includes: obtaining a video frame image obtained by the AR device performing real-time image acquisition of the target scene; and determining the current position information of the AR device in the target scene based on the video frame image and a three-dimensional point cloud map of the target scene.

[0019] In a possible implementation, the generating of a navigation route based on the current position information of the AR device in the target scene, the destination position information, and the navigation map includes: determining a first area where a navigation starting point is located based on the current position information, and determining a second area where a navigation destination is located based on the destination position information; determining a target path between the first area and the second area based on the navigation map, and generating the navigation route based on the current position information, the destination position information, and the target path.

[0020] In this way, a navigation route can be generated quickly and accurately.

[0021] In a second aspect, the embodiment of the present disclosure also provides a navigation map generating device, including: an acquisition module, used to acquire a scene image obtained by performing image acquisition on a target scene through an image acquisition device;

[0022] A first generating module, used for generating a scene map based on the scene image;

[0023] A second generating module is used to determine, based on the scene image, the position information of the image acquisition device in the target scene when acquiring the scene image, and determine, according to the position information, the moving path of the image acquisition device in the target scene when acquiring the image of the target scene;

[0024] The third generating module is used to generate a navigation map of the target scene based on the moving path and the scene map.

[0025] In a possible implementation manner, when generating the scene map based on the scene image, the first generating module is used to:

[0026] Using the scene image, a three-dimensional point cloud map of the target scene is generated; the three-dimensional point cloud map includes: three-dimensional position information of a plurality of point cloud points in the target scene, and semantic information corresponding to the point cloud points;

[0027] The scene map is generated using the three-dimensional position information and the semantic information.

[0028] In a possible implementation manner, when the first generating module generates the scene map using the three-dimensional position information and the semantic information, it is configured to:

[0029] Based on the three-dimensional position information of the plurality of point cloud points, the plurality of point cloud points are projected onto a preset plane respectively to obtain a projection image;

[0030] The scene map is generated based on the projection image and the semantic information respectively corresponding to the plurality of point cloud points.

[0031] In a possible implementation manner, the projection image includes: projection points corresponding to a plurality of the point cloud points, and a two-dimensional coordinate value of each projection point in the projection image;

[0032] The first generating module, when generating the scene map based on the projection image and the semantic information of each point cloud point, is used to:

[0033] Clustering the projection points based on two-dimensional coordinate values ​​of the projection points of the plurality of point cloud points in the projection image and semantic information corresponding to the plurality of point cloud points;

[0034] The scene map is generated based on the clustering result of the projection points.

[0035] In a possible implementation manner, when determining, based on the position information, a moving path of the image acquisition device in the target scene when acquiring images of the target scene, the second generation module is configured to:

[0036] Based on the position information, the sampling position of the image acquisition device when acquiring the scene image is projected onto a preset plane to obtain a plurality of frames of scene images corresponding to the projection positions of the sampling positions on the preset plane respectively;

[0037] The moving path is generated based on the projection positions of the sampling positions corresponding to the multiple frames of scene images on the preset plane and the timestamp corresponding to each frame of the scene image.

[0038] In a possible implementation manner, the third generating module, when generating the navigation map of the target scene based on the moving path and the scene map, is used to:

[0039] Determining a plurality of regions based on semantic information of each pixel in the scene map;

[0040] Based on the moving path, determine the reachability information between every two adjacent areas;

[0041] Based on the accessibility information between each two adjacent areas, generating a navigation map of the target scene;

[0042] The reachability information between each two adjacent areas includes: whether the two adjacent areas are reachable, and a reachable path when the two adjacent areas are reachable.

[0043] In a possible implementation, the present invention further includes a navigation module, which is used to:

[0044] Obtaining a navigation route generation instruction triggered by a user through an augmented reality (AR) device; the navigation route generation instruction carries the end point location information of the navigation route;

[0045] A navigation route is generated based on the current position information of the AR device in the target scene, the destination position information, and the navigation map.

[0046] In a possible implementation manner, the navigation module is further used to:

[0047] Acquire a video frame image obtained by the AR device acquiring an image of the target scene in real time;

[0048] Based on the video frame image and the three-dimensional point cloud map of the target scene, current position information of the AR device in the target scene is determined.

[0049] In a possible implementation, the navigation module, when generating a navigation route based on the current position information of the AR device in the target scene, the destination position information, and the navigation map, is used to:

[0050] Based on the current location information, determining a first area where the navigation starting point is located, and

[0051] Based on the endpoint location information, determining a second area where the navigation endpoint is located;

[0052] Based on the navigation map, a target path between the first area and the second area is determined, and based on the current position information, the destination position information, and the target path, the navigation route is generated.

[0053] In a third aspect, an optional implementation of the present disclosure further provides a computer device, a processor, and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the machine-readable instructions perform the steps of the above-mentioned first aspect, or any possible implementation of the first aspect.

[0054] In a fourth aspect, an optional implementation of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are executed.

[0055] For a description of the effects of the above-mentioned navigation map generating apparatus, computer device, and computer-readable storage medium, please refer to the description of the above-mentioned navigation map generating method, which will not be repeated here.

[0056] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0058] Figure 1 A flowchart of a navigation map generation method provided by an embodiment of the present disclosure is shown;

[0059] Figure 2 A flowchart showing a specific method for generating a scene map based on a scene image provided by an embodiment of the present disclosure is shown;

[0060] Figure 3 A flowchart showing a specific method for generating a navigation map of a target scene provided by an embodiment of the present disclosure is shown;

[0061] Figure 4 A flowchart of a specific method for generating a navigation route using a navigation map provided by an embodiment of the present disclosure is shown;

[0062] Figure 5 A schematic diagram of a navigation map generating device provided by an embodiment of the present disclosure is shown;

[0063] Figure 6 A schematic diagram of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0065] Research has found that AR navigation uses visual perception technology to locate AR devices, and then combines the Global Positioning System (GPS) to achieve real-life driving navigation. However, when applying AR navigation to smaller scenes, such as exhibition halls, scenic spots, amusement park scenes, etc., due to the accuracy of GPS, it is difficult to use GPS combined with visual perception technology to achieve AR navigation in these scenes. Currently, for AR navigation of these smaller scenes, a navigation map is usually established first, and then the image taken by the AR device is used to locate the position of the AR device in the navigation map, and the navigation map is used for navigation. The current navigation map generation method requires the use of manual drawing to draw the scene map of the target scene, and the manually drawn scene map is used to manually plan the path information within the scene; this method of using manual assistance to generate navigation maps is costly and inefficient.

[0066] Based on the above research, the present disclosure provides a navigation map generation method, which uses an image acquisition device to capture the moving path of a target scene and the captured scene image to automatically generate a navigation map of the target scene map, thereby avoiding the high cost and low efficiency problems caused by manually drawing path information.

[0067] The defects existing in the above solutions are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure for the above problems below should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0068] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0069] To facilitate understanding of this embodiment, a navigation map generation method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the navigation map generation method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example: an AR device or a server or other processing device. The AR device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the navigation map generation method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0070] The navigation map generation method provided by the embodiment of the present disclosure is described below.

[0071] See also Figure 1 FIG. 1 is a flowchart of a navigation map generation method provided by an embodiment of the present disclosure, wherein the method comprises steps S101 to S104, wherein:

[0072] S101: Acquire a scene image obtained by performing image acquisition on a target scene through an image acquisition device;

[0073] S102: Generate a scene map based on the scene image;

[0074] S103: Based on the scene image, determine the position information of the image acquisition device in the target scene when acquiring the scene image, and determine, according to the position information, the moving path of the image acquisition device in the target scene when acquiring the image of the target scene;

[0075] S104: Generate a navigation map of the target scene based on the moving path and the scene map.

[0076] There is no execution order logic for the above S102 and S103, and they can be executed synchronously or asynchronously.

[0077] The disclosed embodiment acquires a scene image obtained by an image acquisition device when acquiring a target scene, generates a scene map based on the scene image, determines a moving path of the image acquisition device when acquiring the target scene based on the scene image, and then generates a navigation map of the target scene based on the scene map and the path information. The process of generating the navigation map does not require manual drawing of the path within the target scene, thereby avoiding the high cost and low efficiency problems caused by manual drawing of path information.

[0078] The above S101 to S104 are described in detail below.

[0079] I: With respect to the above S101, the image acquisition device includes, for example, a depth camera and a color camera. When capturing the target scene, the image acquisition device can acquire multiple frames of scene images, including depth images and color images.

[0080] For example, when the target scene is captured by the image acquisition device, the depth camera and the color camera can be controlled to synchronously expose the target scene to obtain a frame of depth image and a frame of color image with the same timestamp. The shooting fields of the depth camera and the color camera are consistent or close, so that the objects contained in the depth image and the color image with the same timestamp are consistent or close.

[0081] II: Regarding S102 above:

[0082] See also Figure 2 As shown, the embodiment of the present disclosure also provides a specific method for generating a scene map based on a scene image, including:

[0083] S201: Generate a three-dimensional point cloud map of the target scene using the scene image; the three-dimensional point cloud map includes: three-dimensional position information of a plurality of point cloud points in the target scene, and semantic information corresponding to the point cloud points.

[0084] In a specific implementation, (1): For example, the following method can be used to obtain a three-dimensional point cloud map:

[0085] Key point detection and key point matching are performed on the scene image, and based on the results of key point matching, the target key points are determined. Then, the three-dimensional coordinate values ​​of the point cloud points corresponding to the target key points in the target scene and the three-dimensional coordinate values ​​of the point cloud points corresponding to the target key points in the camera coordinate system corresponding to the scene image are used to obtain the three-dimensional position information of other point cloud points in the target scene in the scene coordinate system.

[0086] Specifically, the camera coordinate system corresponding to the image acquisition device when acquiring the first frame of the scene image can be used as the scene coordinate system; and based on the coordinate values ​​of each point cloud point in the depth image in the first frame of the scene image in the camera coordinate system, the coordinate values ​​of each point cloud point in the depth image in the scene coordinate system are obtained.

[0087] Here, the camera coordinate system is, for example, a three-dimensional coordinate system established using the optical center of the image acquisition device as the coordinate origin, the depth direction as the z-axis, and the plane where the image acquired by the image acquisition device is located as the plane where the x-axis and y-axis are located.

[0088] After acquiring the second frame of scene image, the second frame of scene image and the color image in the first frame of scene image are subjected to key point detection and key point matching to determine the target key point pair representing the same position in the first frame of scene image and the second frame of scene image, wherein the target key point pair includes the first key point located in the first frame of scene image and the second key point located in the second frame of scene image, wherein the first key point is the point cloud point in the target scene projected in the first frame of scene image, and the second key point is the same point cloud point in the target scene projected in the second frame of scene, therefore, the three-dimensional position information of the point cloud point corresponding to the first key point in the scene coordinate system is the three-dimensional position information of the point cloud point corresponding to the second key point in the scene coordinate system. Then, the three-dimensional position information of the point cloud point corresponding to the second key point in the camera coordinate system corresponding to the second frame of scene image is used to obtain the conversion relationship information between the camera coordinate system corresponding to the second frame of scene image and the scene coordinate system, and then the three-dimensional position information of all the point cloud points corresponding to the second frame of scene image in the scene coordinate system is obtained using the conversion relationship information.

[0089] Then, for the third frame scene image acquired by the image acquisition device, based on the third frame scene image and the second frame scene image, the three-dimensional position information of all corresponding point cloud points in the third positive scene image in the scene coordinate system is obtained.

[0090] Through continuous iteration of the above process, a three-dimensional point cloud map can be generated.

[0091] (2): For example, the following method can be used to obtain the semantic information corresponding to the point cloud points in the target scene:

[0092] The scene image is semantically segmented to obtain the semantic information corresponding to each pixel in the scene image; then the projection relationship between the point cloud points in the target scene and the pixel points in the scene image is used to obtain the semantic information corresponding to the point cloud points in the target scene.

[0093] In a specific implementation, for example, a pre-trained semantic segmentation model can be used to perform semantic segmentation processing on a scene image; here, semantic segmentation processing is usually performed on a color image. After the scene image is semantically segmented, the semantic segmentation result of each pixel in the color image, that is, the semantic information, can be obtained.

[0094] The semantic segmentation model can, for example, determine the semantic classification to which each pixel in a color image belongs. Here, the semantic classification includes, for example: the name of each object in the target scene; for example, when three-dimensionally modeling a historical exhibition hall, there will be multiple venues in the historical exhibition hall, each of which includes entrances and exits, exhibits in the venue, walls of the exhibition hall, etc. In the venue, there are also trash cans, plants, furnishings, lockers, etc. The semantics of the above objects can be set as walls, obstacles (including other objects other than walls that can segment the space in the exhibition hall). The semantic segmentation model can identify walls and obstacles, and mark the semantic category to which each pixel belongs.

[0095] Following the above S201, the method for generating a scene map based on a scene image provided by the embodiment of the present disclosure further includes:

[0096] S202: Generate the scene map using the three-dimensional position information and the semantic information.

[0097] In a specific implementation, when generating a scene map using three-dimensional position information and semantic information, for example, based on the three-dimensional position information of multiple point cloud points, the multiple point cloud points are respectively projected onto a preset plane to obtain a projection image; based on the projection image and the semantic information corresponding to the multiple point cloud points, the scene map is generated.

[0098] Here, the preset plane when performing planar projection on the point cloud points is, for example, a horizontal plane, or a plane determined based on the ground of the target scene.

[0099] The projection image includes: projection points corresponding to a plurality of point cloud points respectively, and a two-dimensional coordinate value of each projection point in the projection image.

[0100] The embodiment of the present disclosure takes the horizontal plane as the projection plane as an example. When projecting the point cloud point onto the horizontal plane, the projection is performed on the plane formed by the x-axis and z-axis of the scene coordinate system. If the coordinate value of the point cloud point in the scene coordinate system is: (w, e, r), then after projection, the two-dimensional coordinate value of the projection point formed in the projection image is: (w, r). That is, in the projection image, the x-axis coordinate value is w, and the y-axis coordinate value is r.

[0101] Through the above process, after multiple point cloud points are projected onto the projection surface respectively, the set of all projection points constitutes a projection image.

[0102] After the projection image is generated, the following method can be used to generate a scene map using the projection image:

[0103] Based on the two-dimensional coordinate values ​​of the projection points of the plurality of point cloud points in the projection image and the semantic information respectively corresponding to the plurality of point cloud points, the projection points are clustered; based on the clustering results of the projection points, the scene map is generated.

[0104] Here, when clustering is performed based on the two-dimensional coordinate values ​​of each projection point in the projection image and the semantic information of each point cloud point, for example, the distance between different projection points can be calculated by the two-dimensional coordinate values ​​of different projection points, and then based on the distances between different projection points and the corresponding semantic information, it is determined whether the different projection points should belong to the same category, thereby grouping all the projection points.

[0105] After all the projection points are grouped, the projection area of ​​the object in the target scene on the projection surface is determined based on each group of projection points, and then a scene map can be generated according to the projection area.

[0106] When generating a scene map based on the projection area, for example, corresponding point of interest (POI) information and display effects can be added to each projection area. For example, when the target scene is a history museum, different exhibition hall names will be added to different projection areas, such as "Han Dynasty Exhibition Hall", "X Planet X Cultural Relics Exhibition Hall", etc., and different colors, historical landmark effects, etc. will be added to the corresponding projection areas. Then, based on the added display effects and POI information, a scene map is generated.

[0107] III: With respect to the above S103, the moving path of the image acquisition device when acquiring the target scene can be acquired, for example, by an inertial navigation device installed in the image acquisition device. When the image acquisition device acquires the target scene, the inertial navigation device will record the moving distance and moving direction during the acquisition process; when the moving starting point is determined, the moving path of the image acquisition device can be determined based on the moving distance and moving direction recorded by the inertial navigation device.

[0108] In another embodiment, the moving path may also be determined based on the posture of the image acquisition device when capturing the scene image. The posture acquisition method of the image acquisition device may also be as follows:

[0109] A camera coordinate system is established based on the first frame of scene image acquired by the image acquisition device, and the coordinate system is used as the scene coordinate system. The scene coordinate system uses the optical axis of the image acquisition device when acquiring the first frame of scene image as the z-axis, and the position of the optical center of the image acquisition device when acquiring the first frame of scene image as the origin. At this time, the position and posture of the image acquisition device when acquiring the first frame of scene image can be determined, that is, the position and posture of the image acquisition device in the scene coordinate system when acquiring the first frame of scene image.

[0110] After the image acquisition device acquires the second frame of the scene image, the first frame of the scene image and the second frame of the scene image are subjected to feature extraction and key point matching processing, and based on the results of the feature extraction and key point matching processing, the movement distance and rotation angle of the image acquisition device in the target scene can be determined. According to the movement distance, rotation angle, and the position and posture of the image acquisition device in the scene coordinate system when acquiring the first frame of the scene image, the position and posture of the image acquisition device in the scene coordinate system when acquiring the second frame of the scene image can be determined.

[0111] Then, for the third frame scene image acquired by the image acquisition device, based on the movement distance and rotation angle of the image acquisition device relative to the second frame scene image, and the corresponding posture of the second frame scene image, the position of the image acquisition device in the scene coordinate system when acquiring the third frame scene image can be determined.

[0112] By continuously iterating the above process, the positions and postures corresponding to all scene images can be determined. After the positions and postures corresponding to all scene images are determined, the moving path is obtained based on the data representing the position of the scene image in the target scene.

[0113] Exemplarily, an embodiment of the present disclosure provides a specific method for a moving path, including: based on the position information, projecting the sampling position of the image acquisition device when collecting the scene image to a preset plane, and obtaining projection positions of multiple frames of scene images corresponding to the sampling positions on the preset plane respectively; based on the projection positions of the multiple frames of scene images corresponding to the sampling positions on the preset plane respectively, and the timestamp corresponding to each frame of the scene image, generating the moving path.

[0114] IV: With respect to the above S104, after obtaining the scene map and path information based on the above S101 to S103, the scene map and path information are combined to determine whether any two location points in the scene map are reachable. The scene map and path information are then combined to obtain a navigation map.

[0115] Specifically, see Figure 3 As shown, the embodiment of the present disclosure provides a specific method for generating a navigation map of a target scene based on a moving path and a scene map, including:

[0116] S301: Determine multiple areas based on semantic information of each pixel in the scene map.

[0117] In a specific implementation, when multiple areas are determined based on the semantic information of each pixel in the scene map, for example, a target pixel whose semantic information is the target semantic information is determined from the semantic information of each pixel in the scene map.

[0118] Exemplarily, when the target scene is an exhibition hall, the target semantic information is: wall; based on the walls between different exhibition halls, the scene map is divided into multiple areas.

[0119] Here, each determined area generally includes at least one projection area in the scene map.

[0120] S302: Determine the reachability information between every two adjacent areas based on the moving path.

[0121] Here, the reachability information between each two adjacent areas includes: whether each two adjacent areas are reachable, and a reachable path when each two adjacent areas are reachable.

[0122] Here, if the moving path crosses the area interval between two adjacent areas, the two adjacent areas are considered to be reachable, and the path crossing the interval between the two adjacent areas is determined as a reachable path between the two areas.

[0123] S303: Generate a navigation map of the target scene based on the accessibility information between each two adjacent areas.

[0124] After the navigation map is generated, when navigating based on the navigation map, for any location point in the target scene, the first area of ​​the navigation map can be determined for the location point; then the second area of ​​the navigation map where the destination location is located is used, and then the navigation map is used to generate a navigation route.

[0125] For example, see Figure 4 As shown, the embodiment of the present disclosure also provides a specific method for generating a navigation route using a navigation map, including:

[0126] S401: Acquire a navigation route generation instruction triggered by a user through an augmented reality (AR) device; the navigation route generation instruction carries the end point location information of the navigation route;

[0127] S402: Generate a navigation route based on the current position information of the AR device in the target scene, the destination position information, and a pre-generated navigation map.

[0128] The embodiment of the present disclosure obtains a navigation route generation instruction triggered by an AR device, and generates a navigation route based on the destination location information carried in the navigation route generation instruction, the current location information of the AR device, and the navigation map generation method provided by the embodiment of the present disclosure, thereby reducing costs and achieving higher navigation accuracy.

[0129] The above S401 and S402 are described in detail below.

[0130] In the above S401, the destination location information can be represented, for example, as any location point in the target scene or a POI of an object in the target scene. When the destination location information is represented as a POI, the object in the target scene can have Point of Interest (POI) information generated in advance.

[0131] The navigation map generation method provided by the embodiment of the present disclosure can be executed in an AR device or on a server side.

[0132] Exemplarily, (1): In the case where the navigation map generation method provided by the embodiment of the present disclosure is executed in an AR device, a target scene map of the target scene can be displayed to the user in the graphical user interface of the AR device; the target scene map can be a two-dimensional map or a three-dimensional map; in the case where the target scene is a two-dimensional map, it can be, for example, the scene map in the above embodiment; in the case where the target scene is a three-dimensional map, it can be, for example, a three-dimensional point cloud map obtained based on the above embodiment, which is composed of point cloud points in the target scene, and by rendering different special effects for objects with different semantic information in the three-dimensional point cloud map, a more detailed target scene can be displayed to the user. For example, trash cans, street lights, etc. arranged in the target scene can all be displayed in the three-dimensional point cloud map in the form of special effects. The AR device responds to the user's triggering of any position in the scene map, determines a terminal location information for the user based on the position triggered by the user, and generates a navigation route generation instruction based on the terminal location information.

[0133] Here, exemplarily, when determining a destination location for the user based on the location triggered by the user, it can be determined whether the location triggered by the user is reachable, for example, based on the navigation map provided in the embodiment of the present disclosure; if reachable, the location is determined as the destination location; if the location is unreachable, a reachable location closest to the location can be determined as the destination location based on the navigation map, or multiple alternative locations can be determined based on the location, and the multiple alternative locations can be displayed to the user through a graphical user interface, so that the user can select one from the multiple alternative locations as the destination location.

[0134] In addition, when it is determined that the location triggered by the user is unreachable, the user may be prompted through the AR device to re-enter the destination to be reached.

[0135] In another embodiment, an information input box may be displayed to the user through an AR device; the user inputs the POI of any object in the target scene through the information input box, and a navigation route generation instruction is generated based on the POI input by the user.

[0136] Alternatively, the POIs of the objects in the target scene may be displayed in the graphical user interface in the form of triggerable controls; after the user triggers any of the POIs, a navigation route generation instruction is generated based on the POI triggered by the user.

[0137] (2): In the case where the navigation route generation method provided by the embodiment of the present disclosure is executed in a service, the AR device generates a navigation route generation instruction based on the process in (1) above, and transmits the navigation route generation instruction to the server.

[0138] In the above S402, when generating a navigation route based on the current position information of the AR device in the target scene, the destination position information, and a pre-generated navigation map, for example, a first area where the navigation starting point is located can be determined based on the current position information, and a second area where the navigation destination is located can be determined based on the destination position information; a target path between the first area and the second area can be determined based on the navigation map, and the navigation route can be generated based on the current position information, the destination position information, and the target path.

[0139] The current position information of the AR device in the target scene can be obtained, for example, in the following manner:

[0140] Obtain the video frame image obtained by the AR device capturing the target scene in real time;

[0141] Based on the video frame image and the three-dimensional point cloud map of the target scene, current position information of the AR device in the target scene is determined.

[0142] In a specific implementation, when the navigation map generation method provided by the embodiment of the present disclosure is executed on a server, when a user carries an AR device to a target scene, the user can use the image acquisition device in the AR device to shoot the target scene to obtain a video frame image captured in real time; the AR device sends the video frame image captured in real time to the server. The server uses the video frame image and the pre-built three-dimensional point cloud map of the target scene to determine the current location information of the AR device in the target scene.

[0143] Exemplarily, for example, the current position information of the AR device in the target scene may be determined in the following manner:

[0144] Key point recognition is performed on the scene image to obtain a first key point in the scene image.

[0145] Based on the first key point, determine the target second key point that matches the first key point from the second key points in the three-dimensional point cloud map, and determine the current position information of the AR device in the scene coordinate system based on the three-dimensional coordinate value of the target second key point in the scene coordinate system.

[0146] After the current position information is determined, the current position information may be the three-dimensional position of the AR device in the target scene. The three-dimensional position may be projected onto a preset plane to obtain a one-dimensional position, and then based on the division of regions in the navigation map, the first region where the navigation start point is located may be obtained. Similarly, the second region where the navigation end point is located may be obtained.

[0147] Among them, the three-dimensional point cloud map can be generated, for example, by the method in the above embodiment, which will not be repeated here.

[0148] Then, by utilizing the accessible paths between every two adjacent areas in the navigation map, at least one alternative path from the first area to the second area can be obtained.

[0149] A target path can be determined from at least one alternative path based on certain screening conditions, and the target path can overlap with a moving path in a target scene captured by an image acquisition device, for example. Based on the initial position information, a first route of a navigation starting point and a moving path is determined, and based on the end point position information, a second route of a navigation end point and a moving path is determined. The navigation route is obtained by combining the target path, the first route, and the second route.

[0150] Here, the screening conditions include, for example: the shortest route, the most areas passed through, passing through a specific area, etc., which can be set according to actual needs.

[0151] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0152] Based on the same inventive concept, a navigation map generating device corresponding to the navigation map generating method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned navigation map generating method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0153] Reference Figure 5 FIG. 1 is a schematic diagram of a navigation map generating device provided by an embodiment of the present disclosure, wherein the device comprises: an acquisition module 51, a first generating module 52, a second generating module 53, and a third generating module 54; wherein,

[0154] An acquisition module 51 is used to acquire a scene image obtained by acquiring an image of a target scene through an image acquisition device;

[0155] A first generating module 52, configured to generate a scene map based on the scene image;

[0156] A second generating module 53 is used to determine, based on the scene image, the position information of the image acquisition device in the target scene when acquiring the scene image, and determine, according to the position information, the moving path of the image acquisition device in the target scene when acquiring the image of the target scene;

[0157] The third generating module 54 is used to generate a navigation map of the target scene based on the moving path and the scene map.

[0158] In a possible implementation manner, when generating a scene map based on the scene image, the first generating module 52 is configured to:

[0159] Using the scene image, a three-dimensional point cloud map of the target scene is generated; the three-dimensional point cloud map includes: three-dimensional position information of a plurality of point cloud points in the target scene, and semantic information corresponding to the point cloud points;

[0160] The scene map is generated using the three-dimensional position information and the semantic information.

[0161] In a possible implementation manner, when the first generating module 52 generates the scene map using the three-dimensional position information and the semantic information, it is configured to:

[0162] Based on the three-dimensional position information of the plurality of point cloud points, the plurality of point cloud points are projected onto a preset plane respectively to obtain a projection image;

[0163] The scene map is generated based on the projection image and the semantic information respectively corresponding to the plurality of point cloud points.

[0164] In a possible implementation manner, the projection image includes: projection points corresponding to the plurality of point cloud points respectively, and a two-dimensional coordinate value of each projection point in the projection image;

[0165] The first generating module 52, when generating the scene map based on the projection image and the semantic information of each point cloud point, is used to:

[0166] Clustering the projection points based on two-dimensional coordinate values ​​of the projection points of the plurality of point cloud points in the projection image and semantic information corresponding to the plurality of point cloud points;

[0167] The scene map is generated based on the clustering result of the projection points.

[0168] In a possible implementation manner, the second generating module 53, when determining, according to the position information, the moving path of the image acquisition device in the target scene when acquiring images of the target scene, is used to:

[0169] Based on the position information, the sampling position of the image acquisition device when acquiring the scene image is projected onto a preset plane to obtain a plurality of frames of scene images corresponding to the projection positions of the sampling positions on the preset plane respectively;

[0170] The moving path is generated based on the projection positions of the sampling positions corresponding to the multiple frames of scene images on the preset plane and the timestamp corresponding to each frame of the scene image.

[0171] In a possible implementation manner, the third generating module 54, when generating the navigation map of the target scene based on the moving path and the scene map, is configured to:

[0172] Determining a plurality of regions based on semantic information of each pixel in the scene map;

[0173] Based on the moving path, determine the reachability information between every two adjacent areas;

[0174] Based on the accessibility information between each two adjacent areas, generating a navigation map of the target scene;

[0175] The reachability information between each two adjacent areas includes: whether the two adjacent areas are reachable, and a reachable path when the two adjacent areas are reachable.

[0176] In a possible implementation, the present invention further includes a navigation module 55 for:

[0177] Obtaining a navigation route generation instruction triggered by a user through an augmented reality (AR) device; the navigation route generation instruction carries the end point location information of the navigation route;

[0178] A navigation route is generated based on the current position information of the AR device in the target scene, the destination position information, and the navigation map.

[0179] In a possible implementation manner, the navigation module 55 is further used to:

[0180] Acquire a video frame image obtained by the AR device acquiring an image of the target scene in real time;

[0181] Based on the video frame image and the three-dimensional point cloud map of the target scene, current position information of the AR device in the target scene is determined.

[0182] In a possible implementation manner, the navigation module 55, when generating a navigation route based on the current position information of the AR device in the target scene, the destination position information, and the navigation map, is used to:

[0183] Based on the current location information, determining a first area where the navigation starting point is located, and

[0184] Based on the endpoint location information, determining a second area where the navigation endpoint is located;

[0185] Based on the navigation map, a target path between the first area and the second area is determined, and based on the current position information, the destination position information, and the target path, the navigation route is generated.

[0186] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0187] The present disclosure also provides a computer device, such as Figure 6 FIG. 1 is a schematic diagram of a computer device structure provided in an embodiment of the present disclosure, including:

[0188] A processor 61 and a memory 62; the memory 62 stores machine-readable instructions executable by the processor 61, and the processor 61 is used to execute the machine-readable instructions stored in the memory 62. When the machine-readable instructions are executed by the processor 61, the processor 61 performs the following steps:

[0189] Acquire a scene image obtained by performing image acquisition on a target scene through an image acquisition device;

[0190] Based on the scene image, generating a scene map;

[0191] Based on the scene image, determining the position information of the image acquisition device in the target scene when acquiring the scene image, and determining, according to the position information, the moving path of the image acquisition device in the target scene when acquiring the image of the target scene; and

[0192] A navigation map of the target scene is generated based on the moving path and the scene map.

[0193] The above-mentioned memory 62 includes internal memory 621 and external memory 622; the memory 621 here is also called internal memory, which is used to temporarily store the calculation data in the processor 61, as well as the data exchanged with the external memory 622 such as the hard disk. The processor 61 exchanges data with the external memory 622 through the internal memory 621.

[0194] The specific execution process of the above instructions can refer to the steps of the navigation map generation method described in the embodiment of the present disclosure, which will not be repeated here.

[0195] The present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the navigation map generation method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0196] The present disclosure also provides a computer program product that carries a program code. The program code includes instructions that can be used to execute the steps of the navigation map generation method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.

[0197] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0198] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0199] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0200] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0201] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0202] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A navigation map generation method, characterized in that: include: Acquire a scene image obtained by performing image acquisition on a target scene through an image acquisition device; Based on the scene image, generating a scene map; Based on the scene image, determining the position information of the image acquisition device in the target scene when acquiring the scene image, and determining, according to the position information, the moving path of the image acquisition device in the target scene when acquiring the image of the target scene; as well as Based on the moving path and the scene map, generating a navigation map of the target scene; The generating a navigation map of the target scene based on the moving path and the scene map includes: Determining a plurality of regions based on semantic information of each pixel in the scene map; Based on the moving path, determine the reachability information between every two adjacent areas; Based on the accessibility information between each two adjacent areas, a navigation map of the target scene is generated.

2. The navigation map generation method according to claim 1, characterized in that: The step of generating a scene map based on the scene image comprises: Using the scene image, a three-dimensional point cloud map of the target scene is generated; the three-dimensional point cloud map includes: three-dimensional position information of a plurality of point cloud points in the target scene, and semantic information corresponding to the point cloud points; The scene map is generated using the three-dimensional position information and the semantic information.

3. The navigation map generation method according to claim 2, characterized in that: The step of generating the scene map by using the three-dimensional position information and the semantic information includes: Based on the three-dimensional position information of the plurality of point cloud points, the plurality of point cloud points are projected onto a preset plane respectively to obtain a projection image; The scene map is generated based on the projection image and the semantic information respectively corresponding to the plurality of point cloud points.

4. The navigation map generation method according to claim 3, characterized in that: The projection image includes: projection points corresponding to a plurality of the point cloud points, and a two-dimensional coordinate value of each projection point in the projection image; The generating the scene map based on the projection image and the semantic information respectively corresponding to the plurality of point cloud points comprises: Clustering the projection points based on two-dimensional coordinate values ​​of the projection points of the plurality of point cloud points in the projection image and semantic information corresponding to the plurality of point cloud points; The scene map is generated based on the clustering result of the projection points.

5. The navigation map generation method according to any one of claims 1 to 4, characterized in that: The step of determining, based on the position information, a moving path of the image acquisition device in the target scene when acquiring an image of the target scene comprises: Based on the position information, the sampling position of the image acquisition device when acquiring the scene image is projected onto a preset plane to obtain a plurality of frames of scene images corresponding to the projection positions of the sampling positions on the preset plane respectively; The moving path is generated based on the projection positions of the sampling positions corresponding to the multiple frames of scene images on the preset plane and the timestamp corresponding to each frame of the scene image.

6. The navigation map generation method according to any one of claims 1 to 4, characterized in that: The reachability information between each two adjacent areas includes: whether the two adjacent areas are reachable, and a reachable path when the two adjacent areas are reachable.

7. The navigation map generation method according to any one of claims 1 to 4, characterized in that: Also includes: Obtain navigation route generation instructions triggered by the user through the augmented reality (AR) device; The navigation route generation instruction carries the end point location information of the navigation route; A navigation route is generated based on the current position information of the AR device in the target scene, the destination position information, and the navigation map.

8. The navigation map generation method according to claim 7, characterized in that: Also includes: Acquire a video frame image obtained by the AR device acquiring an image of the target scene in real time; Based on the video frame image and the three-dimensional point cloud map of the target scene, current position information of the AR device in the target scene is determined.

9. The navigation map generation method according to claim 7, characterized in that: The generating a navigation route based on the current position information of the AR device in the target scene, the destination position information, and the navigation map includes: Based on the current location information, determining a first area where the navigation starting point is located, and Based on the endpoint location information, determining a second area where the navigation endpoint is located; Based on the navigation map, a target path between the first area and the second area is determined, and based on the current position information, the end point position information, and the target path, the navigation route is generated.

10. A navigation map generating device, characterized in that: include: An acquisition module is used to acquire a scene image obtained by acquiring an image of a target scene through an image acquisition device; A first generating module, used for generating a scene map based on the scene image; A second generating module is used to determine, based on the scene image, the position information of the image acquisition device in the target scene when acquiring the scene image, and determine, according to the position information, the moving path of the image acquisition device in the target scene when acquiring the image of the target scene; A third generating module, configured to generate a navigation map of the target scene based on the moving path and the scene map; The third generating module is specifically used to: determine a plurality of areas based on semantic information of each pixel in the scene map; Based on the moving path, determine the reachability information between every two adjacent areas; Based on the accessibility information between each two adjacent areas, a navigation map of the target scene is generated.

11. A computer device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the processor executes the navigation map generation method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program. When the computer program is executed by a computer device, the computer device executes the navigation map generating method according to any one of claims 1 to 9.

Images