Three-dimensional positioning method, system, device and storage medium

Abstract

In the embodiments of the present disclosure, a three-dimensional positioning method, system, device, and storage medium are provided. By collecting binocular images and obtaining a depth map of the binocular images, where the binocular images include a first image and a second image, aligning the depth map with the first image to obtain an aligned region of the first image and the depth map, and obtaining the depth values of the feature points in the aligned region according to the depth map, determining the non-aligned region of the first image and the depth map, determining the depth values of the corresponding feature points in the non-aligned region according to the matched feature points, and performing three-dimensional positioning according to the depth values of the feature points in the first image. Since the field of view of the depth map is smaller than that of the binocular images, for the aligned region of the first image and the depth map, the depth values of the feature points in the first image can be directly obtained from the depth map, and there is no need to perform feature point matching between the feature points in this aligned region and the second image, which greatly reduces the consumption time of feature point matching, shortens the three-dimensional positioning duration, and improves the timeliness of three-dimensional positioning.

Description

Technical Field

[0001] The present disclosure relates to the technical field of three-dimensional positioning, and in particular to a three-dimensional positioning method, system, device, and storage medium. Background Art

[0002] There are various modes of vision-based simultaneous localization and mapping (VSLAM, full English name: Simultaneous Localization and Mapping), such as solutions based on monocular, binocular, and RGB-D (RGB monocular camera and depth map). In the binocular solution, some input cameras are fish-eye, so that a larger field of view can be obtained, and more feature points can be generated, thereby greatly improving the accuracy of the VSLAM system.

[0003] However, since binocular vision is mostly feature point-based, feature point matching needs to be performed between the left and right images to generate three-dimensional map points, and this process is time-consuming. Summary of the Invention

[0004] In view of the above-mentioned disadvantages of the related art, the purpose of the present disclosure is to provide a three-dimensional positioning method, system, device, and storage medium to solve the technical problem of too long three-dimensional positioning time in the related art.

[0005] The first aspect of the present disclosure provides a three-dimensional positioning method, which includes:

[0006] Collect binocular images and obtain the depth map of the binocular images, where the binocular images include a first image and a second image;

[0007] Align the depth map with the first image to obtain the alignment area of the first image and the depth map, and obtain the depth values of the feature points in the alignment area according to the depth map;

[0008] Determine the non-alignment area of the first image and the depth map, perform feature point matching on the non-alignment area and the second image, and determine the depth values of the corresponding feature points in the non-alignment area according to the matched feature points;

[0009] Perform three-dimensional positioning according to the depth values of the feature points in the first image.

[0010] Optionally, performing feature point matching on the non-alignment area and the second image includes:

[0011] Extract feature descriptors for the feature points in the non-alignment area and the second image respectively;

[0012] Use the feature descriptors of the feature points in the non-alignment area to perform feature descriptor matching in the second image to obtain matching feature points.

[0013] Optionally, determining the depth value of the corresponding feature points in the non-aligned region according to the matched feature points includes:

[0014] Calculating the parallax information between the corresponding feature points in the non-aligned region and the matched feature points;

[0015] Determining the depth value of the matched feature points according to the parallax information.

[0016] Optionally, performing three-dimensional positioning according to the depth value of the feature points in the first image includes:

[0017] Performing back-projection transformation on the corresponding feature points according to the depth value of the feature points in the first image to obtain the three-dimensional coordinates of the corresponding feature points.

[0018] Optionally, the three-dimensional positioning method further includes:

[0019] Constructing a grid map according to the three-dimensional positioning information of the corresponding feature points in the first image.

[0020] Optionally, constructing a grid map according to the three-dimensional positioning information of the corresponding feature points in the first image includes:

[0021] Determining the non-ground points corresponding to the corresponding feature points and the passable points between the non-ground points and the image acquisition device according to the three-dimensional positioning information of the corresponding feature points in the first image and the coordinates of the current image acquisition device;

[0022] Constructing a grid map by using the non-ground points and the passable points.

[0023] Optionally, constructing a grid map according to the three-dimensional positioning information of the corresponding feature points in the first image includes:

[0024] In the case of collecting multiple pairs of binocular image frames and corresponding depth maps, using the three-dimensional positioning method for each pair of binocular image frames and corresponding depth maps to obtain three-dimensional positioning information, and constructing a grid map by using the three-dimensional positioning information.

[0025] The second aspect of the present disclosure further provides a three-dimensional positioning system, which includes:

[0026] An acquisition module, which acquires binocular images and obtains depth maps of the binocular images, and the binocular images include a first image and a second image;

[0027] An alignment module, which aligns the depth map with the first image to obtain the aligned region of the first image and the depth map, and obtains the depth value of the feature points in the aligned region according to the depth map;

[0028] A matching module, which determines the non-aligned region of the first image and the depth map, matches the non-aligned region with the second image for feature points, and determines the depth value of the corresponding feature points in the non-aligned region according to the matched feature points;

[0029] A positioning module performs three-dimensional positioning based on the depth values of feature points in the first image.

[0030] The third aspect of the present disclosure provides a computer device, including: a communicator, a memory, and a processor; the communicator is used for external communication; the memory stores program instructions; the processor is used for running the program instructions to execute the three-dimensional positioning method according to any one of the first aspect.

[0031] The fourth aspect of the present disclosure provides a computer-readable storage medium storing program instructions, and the program instructions are run to execute the three-dimensional positioning method according to any one of the first aspect.

[0032] As described above, the embodiments of the present disclosure provide a three-dimensional positioning method, system, device, and storage medium. By collecting binocular images and obtaining the depth map of the binocular images, where the binocular images include a first image and a second image, aligning the depth map with the first image to obtain the alignment region between the first image and the depth map, obtaining the depth values of the feature points in the alignment region according to the depth map, determining the non-alignment region between the first image and the depth map, performing feature point matching between the non-alignment region and the second image, and determining the depth values of the corresponding feature points in the non-alignment region according to the matched feature points, and performing three-dimensional positioning based on the depth values of the feature points in the first image. In this embodiment, by combining the binocular images and the depth map aligned with one of the images, since the field of view of the depth map is smaller than that of the binocular images, for the alignment region between the first image and the depth map, the depth values of the feature points in the first image can be directly obtained from the depth map, and for the feature points in this alignment region, there is no need to perform feature point matching with the second image, but only need to perform feature point matching on the remaining non-alignment region, which greatly reduces the consumption time of feature point matching, shortens the three-dimensional positioning duration, and improves the timeliness of three-dimensional positioning. Description of the Drawings

[0033] Figure 1 A flowchart showing the three-dimensional positioning method according to an embodiment of the present disclosure.

[0034] Figure 2 A schematic diagram showing the alignment between the first image and the depth map according to an embodiment of the present disclosure.

[0035] Figure 3 A schematic diagram showing the principle of determining the depth value according to the parallax in the embodiment of the present disclosure.

[0036] Figure 4 A flowchart showing the three-dimensional positioning method according to an embodiment of the present disclosure.

[0037] Figure 5 A flowchart showing the three-dimensional positioning method according to a specific embodiment of the present disclosure.

[0038] Figure 6Schematic diagram of modules of the three-dimensional positioning system according to embodiments of the present disclosure.

[0039] Figure 7 Schematic diagram of the structure of a computer device in an embodiment of the present disclosure. Detailed implementation manners

[0040] The following uses specific specific examples to illustrate the implementation manners of the present disclosure. Those skilled in the art can easily understand other advantages and effects of the present disclosure from the information disclosed in the present disclosure. The present disclosure can also be implemented or applied through other different specific implementation manners. Various details in the present disclosure can also be modified or changed according to different viewpoints and application systems without departing from the spirit of the present disclosure. It should be noted that, without conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0041] The following takes the accompanying drawings as a reference and details the embodiments of the present disclosure so that those skilled in the technical field to which the present disclosure belongs can easily implement it. The present disclosure can be embodied in many different forms and is not limited to the embodiments described herein.

[0042] In the description of the present disclosure, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics represented in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. Moreover, the specific features, structures, materials, or characteristics represented can be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples represented in the present disclosure and the features of the different embodiments or examples.

[0043] In addition, the terms "first" and "second" are only used for illustrative purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" can explicitly or implicitly include at least one of the features. In the description of the present disclosure, "a plurality" means two or more unless otherwise specifically defined.

[0044] Figure 1 Flowchart showing the three-dimensional positioning method provided by an embodiment of the present disclosure. The execution subject of this method can be a three-dimensional positioning system, specifically corresponding to a server or a three-dimensional positioning terminal, such as Figure 1 As shown, this method includes the following steps:

[0045] Step 110: Collect binocular images and obtain depth maps of the binocular images. The binocular images include a first image and a second image;

[0046] Step 120: Align the depth map with the first image to obtain the aligned region of the first image and the depth map, and obtain the depth values of the feature points in the aligned region according to the depth map;

[0047] Step 130: Determine the non-aligned region of the first image and the depth map, perform feature point matching between the non-aligned region and the second image, and determine the depth values of the corresponding feature points in the non-aligned region according to the matched feature points;

[0048] Step 140: Perform three-dimensional positioning according to the depth values of the feature points in the first image.

[0049] The input of this embodiment is a binocular image and a depth map aligned with one of the images. Since the field of view of the depth map is smaller than that of the binocular image, as shown in combination Figure 2 The virtual frame area is the depth map, and the area outside the depth map is the field of view of the binocular image. Therefore, for the aligned region of the first image and the depth map, the depth values of the feature points in the first image can be directly obtained from the depth map, and the feature points in this aligned region do not need to be feature point-matched with the second image, but only the remaining non-aligned region needs to be feature point-matched, which greatly reduces the consumption time of feature point matching, shortens the three-dimensional positioning duration, and improves the timeliness of three-dimensional positioning.

[0050] The method of the embodiment of the present disclosure can be applied to a camera device or an image processing device. Through the above device, a binocular image can be collected, for example, collected through a camera device, or a binocular image can be transmitted through other devices.

[0051] Among them, according to the different structures of the camera device, the first image and the second image in the binocular image can be respectively constructed as the left image and the right image, or can also be constructed as the upper side view and the lower side view, which can be specifically determined according to the position of the camera lens of the camera device for collecting the binocular image. The embodiment of the present disclosure does not specifically limit this.

[0052] In the embodiment of the present disclosure, aligning the depth map with the first image specifically means aligning the depth map to the coordinate system of the first image, and then the depth values of the corresponding feature points in the aligned region can be directly obtained in the depth map.

[0053] In the embodiment of the present disclosure, performing feature point matching between the non-aligned region and the second image can obtain mutually matched feature points. Specifically, performing feature point matching between the non-aligned region and the second image includes:

[0054] Extract feature descriptors for the feature points in the non-aligned region and the second image respectively;

[0055] Use the feature descriptors of the feature points in the non-aligned region to perform feature descriptor matching in the second image to obtain matching feature points.

[0056] Among them, the feature descriptor is a quantitative description of the feature point, and its manifestation form is a feature vector. In this way, feature point matching is to compare the feature descriptors of the feature points in two images to determine the matching feature points according to the comparison results. Specifically, the brief of the feature descriptor of the feature points calculated based on the FAST feature algorithm is used to obtain the mutually associated and matching feature points by comparing the similarity of the brief of each pixel in the two images. The matching feature points include the feature points of a certain measured point on the object to be measured in the two binocular images respectively.

[0057] In the embodiment of the present disclosure, determining the depth value of the corresponding feature point in the misaligned area according to the matched feature points includes:

[0058] Calculating the parallax information between the corresponding feature point in the misaligned area and the matched feature point;

[0059] Determining the depth value of the matched feature point according to the parallax information.

[0060] Specifically, the parallax information between the mutually matching feature points can be calculated according to the triangulation principle, and then the depth value used to characterize the object in the application scenario can be determined based on the parallax information.

[0061] Combined with Figure 3 as shown, the distance between the projection centers of the two cameras O L and O R is b, which is also called the baseline. The imaging point of any point P in the three-dimensional space on the left camera is P L , and the imaging point on the right camera is P R . According to the principle of the straight-line propagation of light, the three-dimensional space point P is the intersection point of the connection lines of the projection centers and the imaging points of the two cameras. The line segments X L and X R are the distances from the imaging points of the left and right cameras to the left and right imaging planes respectively. Then the parallax of point P in the left and right cameras can be defined as follows:

[0062] d = |X L - X R |

[0063] The distance between the two imaging points P L and P R is:

[0064]

[0065] According to the similar triangle theory, it can be obtained that:

[0066]

[0067] Then the distance Z from point P to the projection center plane can be obtained:

[0068]

[0069] When the point P moves in three - dimensional space, the imaging positions of the point P on the left and right cameras will also change, and thus the parallax will also change accordingly. As can be seen from the above formula, the parallax is inversely proportional to the distance from the point in three - dimensional space to the projection center plane. Therefore, as long as the parallax of a certain point is known, the depth information of that point can be known.

[0070] In the implementation of the present disclosure, when the depth values of the feature points in the first image are obtained, three - dimensional positioning of the points to be measured on the object corresponding to the feature points can be performed. Specifically, according to the depth values of the feature points in the first image, inverse projection transformation is performed on the corresponding feature points to obtain the three - dimensional coordinates of the corresponding feature points. Furthermore, for all the feature points with depth values in the first image, inverse projection transformation is performed according to their respective depth values, and the overall three - dimensional coordinates of the object to be measured can be obtained.

[0071] The depth values of the feature points in the first image represent the distances from them to the image acquisition device. Inverse projection means projecting the feature points in the first image into three - dimensional space to obtain the three - dimensional coordinates of the corresponding feature points in the world coordinate system.

[0072] Figure 4 The flowchart showing the three - dimensional positioning method of the embodiments of the present disclosure is as Figure 4 shown, and the method includes but is not limited to the following steps:

[0073] Step 410: Collect binocular images and obtain the depth map of the binocular images. The binocular images include the first image and the second image;

[0074] Step 420: Align the depth map with the first image to obtain the aligned area of the first image and the depth map, and obtain the depth values of the feature points in the aligned area according to the depth map;

[0075] Step 430: For the non - aligned area of the first image and the depth map, perform feature point matching between the non - aligned area and the second image, and determine the depth values of the corresponding feature points in the non - aligned area according to the matched feature points;

[0076] Step 440: Perform three - dimensional positioning according to the depth values of the feature points in the first image;

[0077] Step 450: Construct a grid map according to the three - dimensional positioning information of the corresponding feature points in the first image.

[0078] In the embodiments of the present disclosure, the grid map can be used for navigation.

[0079] In the embodiments of the present disclosure, constructing a grid map according to the three - dimensional positioning information of the corresponding feature points in the first image includes:

[0080] Based on the three-dimensional positioning information of the corresponding feature points in the first image and the coordinates of the current image acquisition device, determine the non-ground points corresponding to the corresponding feature points and the passable points between the non-ground points and the image acquisition device;

[0081] Construct a grid map using the non-ground points and the passable points.

[0082] The current image acquisition device is used to acquire binocular images. Non-ground points are considered as points lower than the current image acquisition device. Therefore, according to the three-dimensional positioning information of the corresponding feature points, the height of the point to be measured corresponding to the corresponding feature point can be obtained, and it is compared with the height of the current image acquisition device to determine the ground points and non-ground points. Non-ground points are regarded as obstacle points. At this time, all points between the non-ground points and the image acquisition device are regarded as passable points.

[0083] In the embodiments of the present disclosure, in the case of acquiring multiple pairs of binocular image frames and corresponding depth maps, for each pair of binocular image frames and corresponding depth maps, three-dimensional positioning information is obtained using a three-dimensional positioning method, and a grid map is constructed using the three-dimensional positioning information.

[0084] Figure 5 Show the flowchart of the three-dimensional positioning method of an embodiment of the present disclosure. This method specifically includes the following steps:

[0085] Step 510: Input the left image, right image of the binocular image, and the depth map aligned to the left image;

[0086] Step 520: Extract the positions of the feature points from the left and right binocular images respectively.

[0087] Step 530: For a feature point in the left image, if there is a corresponding depth value in the depth map, directly take its depth value as the depth value of the corresponding feature point;

[0088] Step 540: If a certain feature point in the left image has no corresponding depth value, extract a descriptor for this feature point to perform matching in the right image based on the feature descriptor. If a feature point that meets the corresponding conditions is matched, calculate the disparity (for example, the difference in the abscissa), so as to obtain the depth value. If there is no feature point that meets the corresponding conditions, discard it.

[0089] Step 550: For the feature points with depth values, calculate their three-dimensional coordinates through back-projection transformation.

[0090] Step 560: Obtain the three-dimensional points corresponding to the corresponding feature points according to the three-dimensional coordinates, and judge whether the three-dimensional points are higher than the binocular camera, that is, judge whether the three-dimensional points are ground points. If they are ground points, discard them;

[0091] Step 570: Form a straight line between the three-dimensional point and the current coordinate point of the binocular camera, and project the straight line onto the ground plane, where the position of the three-dimensional point is the obstacle point, and all points between the straight line connections are passable points.

[0092] Perform the above operations on each image frame collected in sequence, so that a grid map for navigation can be obtained.

[0093] As Figure 6 shown, a module schematic diagram of a three-dimensional positioning system according to an embodiment of the present disclosure is presented. It should be noted that the principle of the three-dimensional positioning system can refer to the three-dimensional positioning method in the previous embodiment, so the same technical content will not be repeated here.

[0094] The three-dimensional positioning system 600 may include:

[0095] An acquisition module 610 that acquires binocular images and obtains a depth map of the binocular images, where the binocular images include a first image and a second image;

[0096] An alignment module 620 that aligns the depth map with the first image to obtain an alignment region of the first image and the depth map, and obtains the depth values of the feature points in the alignment region according to the depth map;

[0097] A matching module 630 that determines the non-alignment region of the first image and the depth map, performs feature point matching on the non-alignment region and the second image, and determines the depth values of the corresponding feature points in the non-alignment region according to the matched feature points;

[0098] A positioning module 640 that performs three-dimensional positioning according to the depth values of the feature points in the first image.

[0099] In some embodiments, the matching module 630 is specifically configured to:

[0100] Extract feature descriptors for the feature points in the non-alignment region and the second image respectively;

[0101] Use the feature descriptors of the feature points in the non-alignment region to perform feature descriptor matching in the second image to obtain matched feature points.

[0102] In some embodiments, the matching module 630 is specifically configured to:

[0103] Calculate the parallax information between the corresponding feature points in the non-alignment region and the matched feature points;

[0104] Determine the depth values of the matched feature points according to the parallax information.

[0105] In some embodiments, the positioning module 640 is specifically further configured to:

[0106] Perform back-projection transformation on the corresponding feature points according to the depth values of the feature points in the first image to obtain the three-dimensional coordinates of the corresponding feature points.

[0107] In some embodiments, the positioning module 640 is further specifically configured to:

[0108] Construct a grid map according to the three-dimensional positioning information of the corresponding feature points in the first image.

[0109] In some embodiments, the positioning module 640 is specifically configured to:

[0110] Determine the non-ground points corresponding to the corresponding feature points and the passable points between the non-ground points and the image acquisition device according to the three-dimensional positioning information of the corresponding feature points in the first image and the coordinates of the current image acquisition device;

[0111] Construct a grid map using the non-ground points and the passable points.

[0112] In some embodiments, the positioning module 640 is specifically configured to:

[0113] In the case of acquiring multiple pairs of binocular image frames and corresponding depth maps, use a three-dimensional positioning method for each pair of binocular image frames and corresponding depth maps to obtain three-dimensional positioning information, and construct a grid map using the three-dimensional positioning information.

[0114] It should be specifically noted that in Figure 6 Each functional module in the embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a program instruction product. The program instruction product includes one or more program instructions. When the program instructions are loaded and executed on a computer, the processes or functions according to the present disclosure are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The program instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0115] And, Figure 6 The systems disclosed in the embodiments can be implemented by other module partitioning methods. The system embodiments shown above are merely illustrative. For example, the module partitioning is only a logical function partitioning. In actual implementation, there can be other partitioning methods, such as multiple modules or modules can be combined or can be dynamically adjusted to another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections between each other can be through some interfaces, and the indirect couplings or communication connections of devices or modules can be in an electrical or other form.

[0116] In addition, Figure 6In the embodiments, each functional module and sub-module can be dynamically located in a processing component, or each module can exist physically alone, or two or more modules can be dynamically located in one component. The above-mentioned dynamic component can be implemented in the form of hardware or in the form of a software functional module. When the above-mentioned dynamic component is implemented in the form of a software functional module and executed as an independent product for sale or use, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a magnetic disk, an optical disc, etc.

[0117] In the embodiments of the present disclosure, a computer-readable storage medium can also be provided, storing program instructions that, when run, execute the method steps before Figure 1 in the embodiments.

[0118] The method steps in the above embodiments are implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disc), or as computer code that is originally stored in a remote recording medium or a non-transitory machine-readable medium and downloaded through a network and will be stored in a local recording medium, so that the method represented herein can be stored on such a software process on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA).

[0119] It should be specifically noted that the flowcharts representing the processes or methods in the above embodiments of the present disclosure can be understood as representing modules, segments, or parts of code including one or more executable instructions for implementing specific logical functions or processes. And the scope of the preferred embodiments of the present disclosure includes additional implementations, where the functions can be executed in a substantially simultaneous manner or in the reverse order according to the functions involved, rather than in the order shown or discussed.

[0120] For example, Figure 1 the order of the steps in the illustrated embodiments may be changed in a specific scenario and is not limited to the above representation.

[0121] As Figure 7 shown, a schematic structural diagram of a computer device in an embodiment of the present disclosure is presented.

[0122] In some embodiments, the computer device is used to load program instructions for implementing the three-dimensional positioning method. The computer device can be specifically implemented as, for example, a server, a desktop computer, a laptop computer, a mobile terminal, etc., and may be used by implementers who store and / or run this program instruction for commercial purposes such as development and testing.

[0123] Figure 7The computer device 700 shown is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

[0124] As Figure 7 shown, the computer device 700 is presented in the form of a general-purpose computing device. The components of the computer device 700 may include, but are not limited to: at least one of the above-mentioned processing units 710, at least one of the above-mentioned storage units 720, and a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710).

[0125] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 710, so that the computer device is used to implement the method steps described in the above embodiments of the present disclosure (for example Figure 2 embodiments).

[0126] In some embodiments, the storage unit 720 may include a volatile storage unit, such as a random access storage unit (RAM) 721 and / or a cache storage unit 722, and may further include a read-only storage unit (ROM) 723.

[0127] In some embodiments, the storage unit 720 may further include a program / utilities 724 having a set (at least one) of program modules 725. Such program modules 725 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0128] In some embodiments, the bus 730 may include a data bus, an address bus, and a control bus.

[0129] In some embodiments, the computer device 700 may also communicate with one or more external devices 900 (such as a keyboard, a pointing device, a Bluetooth device, etc.). Such communication may be carried out through an input / output (I / O) interface 750. Optionally, the computer device 700 further includes a display unit 740, which is connected to the input / output (I / O) interface 750 for display. And, the computer device 700 may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 760. As shown in the figure, the network adapter 760 communicates with other modules of the computer device 700 through the bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in combination with the computer device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0130] In summary, the three-dimensional positioning method, system, device, and storage medium provided in the embodiments of the present disclosure collect binocular images and obtain depth maps of the binocular images. The binocular images include a first image and a second image. The depth map is aligned with the first image to obtain an alignment region between the first image and the depth map, and the depth values of the feature points in the alignment region are obtained according to the depth map. The non-alignment region between the first image and the depth map is determined, and the non-alignment region is subjected to feature point matching with the second image, and the depth values of the corresponding feature points in the non-alignment region are determined according to the matched feature points. Three-dimensional positioning is performed according to the depth values of the feature points in the first image. In this embodiment, the binocular image and the depth map aligned with one of the images are combined. Since the field of view of the depth map is smaller than that of the binocular image, for the alignment region between the first image and the depth map, the depth values of the feature points in the first image can be directly obtained from the depth map, and the feature points in this alignment region do not need to be subjected to feature point matching with the second image, but only the remaining non-alignment region needs to be subjected to feature point matching. This greatly reduces the consumption time of feature point matching, shortens the three-dimensional positioning duration, and improves the timeliness of three-dimensional positioning.

[0131] The above embodiments are only illustrative of the principles and effects of the present disclosure, and are not intended to limit the present disclosure. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present disclosure. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical ideas disclosed by the present disclosure should still be covered by the claims of the present disclosure.

Claims

1. A three-dimensional positioning method, characterized in that, it includes: collect binocular images and obtain the depth map of the binocular images, where the binocular images include a first image and a second image; align the depth map with the first image to obtain the alignment region of the first image and the depth map, and obtain the depth values of the feature points in the alignment region according to the depth map; determine the non-alignment region of the first image and the depth map, perform feature point matching between the non-alignment region and the second image, and determine the depth values of the corresponding feature points in the non-alignment region according to the matched feature points; perform three-dimensional positioning according to the depth values of the feature points in the first image.

2. The three-dimensional positioning method according to claim 1, characterized in that, the performing feature point matching between the non-alignment region and the second image includes: extracting feature descriptors for the feature points in the non-alignment region and the second image respectively; using the feature descriptors of the feature points in the non-alignment region to perform feature descriptor matching in the second image to obtain matching feature points.

3. The three-dimensional positioning method according to claim 1, characterized in that, the determining the depth values of the corresponding feature points in the non-alignment region according to the matched feature points includes: calculating the parallax information between the corresponding feature points in the non-alignment region and the matched feature points; determining the depth value of the matched feature points according to the parallax information.

4. The three-dimensional positioning method according to claim 1, characterized in that, the performing three-dimensional positioning according to the depth values of the feature points in the first image includes: performing back-projection transformation on the corresponding feature points according to the depth values of the feature points in the first image to obtain the three-dimensional coordinates of the corresponding feature points.

5. The three-dimensional positioning method according to claim 1, characterized in that, the three-dimensional positioning method further includes: constructing a grid map according to the three-dimensional positioning information of the corresponding feature points in the first image.

6. The three-dimensional positioning method according to claim 5, characterized in that, the constructing a grid map according to the three-dimensional positioning information of the corresponding feature points in the first image includes: determining the non-ground points corresponding to the corresponding feature points and the passable points between the non-ground points and the image acquisition device according to the three-dimensional positioning information of the corresponding feature points in the first image and the coordinates of the current image acquisition device; constructing the grid map by using the non-ground points and the passable points.

7. The three-dimensional positioning method according to claim 5, characterized in that, the constructing a grid map according to the three-dimensional positioning information of the corresponding feature points in the first image includes: in the case of collecting multiple pairs of binocular image frames and corresponding depth maps, using the three-dimensional positioning method for each pair of binocular image frames and corresponding depth maps to obtain three-dimensional positioning information, and constructing a grid map by using the three-dimensional positioning information.

8. A three-dimensional positioning system, characterized in that, it includes: an acquisition module that acquires binocular images and obtains the depth map of the binocular images, where the binocular images include a first image and a second image; An alignment module that aligns the depth map with the first image to obtain an aligned region of the first image and the depth map, and obtains depth values of feature points in the aligned region according to the depth map; A matching module that determines an unaligned region of the first image and the depth map, performs feature point matching on the unaligned region and the second image, and determines depth values of corresponding feature points in the unaligned region according to the matched feature points; A positioning module that performs three-dimensional positioning according to the depth values of feature points in the first image.

9. A computer device, characterized in that it includes: a communicator, a memory, and a processor; the communicator is used for external communication; the memory stores program instructions; the processor is used for running the program instructions to execute the three-dimensional positioning method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that it stores program instructions, and the program instructions are run to execute the three-dimensional positioning method according to any one of claims 1 to 7.

Images