Three-dimensional map construction method and device, equipment and storage medium

Abstract

The invention discloses a three-dimensional map construction method and device, equipment and a storage medium, and relates to the technical field of intelligent equipment, and the method comprises the steps: determining a target region consistent with a current region in current frame image data when detecting that a positioning system in the intelligent equipment carries out spatial positioning, target feature points are extracted from the frame image data after the target area is removed, and a three-dimensional map is constructed according to the target feature points. Through the above mode, the current area with relatively low extraction value of the overall feature points is determined from the number and distribution quality dimensions, and because the difference between the adjacent previous frame of image data and the previous frame of image data is relatively small, the target area which does not participate in extraction of the target feature points in the current frame of image data is determined according to the current area; according to the method, the calculation amount of extracting the target feature points can be effectively reduced, and then the three-dimensional map is constructed according to the target feature points, so that the efficiency of constructing the three-dimensional map can be effectively improved on the basis of ensuring the tracking effect.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent devices, and particularly to a three-dimensional map construction method, apparatus, device, and storage medium. Background Art

[0002] With the continuous improvement of intelligent device technology, the interaction between people and intelligent devices has become more frequent. During the interaction process, the intelligent device will use the internal Simultaneous Localization and Mapping (SLAM) system for spatial positioning and construct a three-dimensional map. Obviously, when constructing a three-dimensional map in the above manner, it is necessary to perform complete calculations on each frame of image data to find feature points that meet the requirements, and the computational complexity of the complete calculations of all frames of image data is extremely large, which will waste a lot of time. Therefore, the efficiency of constructing a three-dimensional map in the above manner is relatively low.

[0003] The above content is only used to assist in understanding the technical solution of the present application, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of the present application is to provide a three-dimensional map construction method, apparatus, device, and storage medium, aiming to solve the technical problem of relatively low efficiency in constructing a three-dimensional map in the prior art.

[0005] To achieve the above object, the present application proposes a three-dimensional map construction method, and the method includes: When it is detected that the positioning system in the intelligent device performs spatial positioning, obtain the feature points of each region of the current frame of image data and the previous frame of image data; Respectively count the number of feature points in each region of the previous frame of image data, and determine the current region where the number of feature points is less than a preset threshold; Determine the target region in the current frame of image data that is consistent with the current region, and remove the target region from the current frame of image data; Extract target feature points from the image data after removal, and construct a three-dimensional map based on the target feature points.

[0006] In an embodiment, the step of obtaining the feature points of each region of the current frame of image data and the previous frame of image data when it is detected that the positioning system in the intelligent device performs spatial positioning includes: When it is detected that the positioning system in the intelligent device performs spatial positioning, respectively obtain the current frame of image data and the previous frame of image data; Determine the feature information of the frame of image data collected by the intelligent device, and determine the region division structure according to the feature information; Based on the region division structure, respectively perform region division on the current frame image data and the previous frame image data to obtain regions of the current frame image data and the previous frame image data; Obtain feature points of each region of the previous frame image data.

[0007] In one embodiment, the step of respectively counting the number of feature points of each region of the previous frame image data and determining the current region where the number of feature points is less than a preset threshold includes: Respectively count the number of feature points of each region of the previous frame image data; Compare the number of feature points with the preset threshold, and determine the region where the number of feature points is less than the preset threshold according to the comparison result; Determine other regions adjacent to the region in the previous frame image data, and calculate the area of the region with feature points and the area of the region without feature points in the other regions; Determine the current region according to the area of the region with feature points and the area of the region without feature points.

[0008] In one embodiment, the step of determining the current region according to the area of the region with feature points and the area of the region without feature points includes: Calculate the area ratio between the area of the region with feature points and the area of the region without feature points; When the area ratio is less than the target value, determine the region where the number of feature points is less than the preset threshold as the current region.

[0009] In one embodiment, the step of determining the target region in the current frame image data that is consistent with the current region and removing the target region from the current frame image data includes: Obtain the position information of the current region relative to the camera component of the intelligent device; When it is detected that the current frame image data and the previous frame image data are completely aligned, determine the target region in the current frame image data that is consistent with the current region according to the position information; Based on a preset marking component, mark the target region according to the target optimization label; Remove the target region with the target optimization label from the current frame image data.

[0010] In one embodiment, the step of extracting target feature points from the frame image data after removal and constructing a 3D map according to the target feature points includes: Extract target feature points from the frame image data after removal and match the target feature points; Extract the target inlier matching pairs from the feature point matching results, and determine the relative pose transformation information according to the target inlier matching pairs; Construct 3D map points according to the relative pose transformation information; Construct a 3D map based on the 3D map points.

[0011] In one embodiment, after the step of extracting target feature points from the filtered frame image data and constructing a 3D map according to the target feature points, the method further includes: Obtain the constructed current 3D map and the sequence number of the current frame image data; When the sequence number is not the sequence number of the last frame image data, obtain the next frame image data; Determine a target area according to the feature points of each area of the next frame image data and the current frame image data; And continue to execute the step of extracting target feature points from the filtered frame image data according to the target area.

[0012] In addition, to achieve the above object, the present application further provides a 3D map construction device, where the 3D map construction device includes: An acquisition module, configured to acquire the feature points of each area of the current frame image data and the previous frame image data when detecting that the positioning system in the intelligent device performs spatial positioning; A determination module, configured to respectively count the number of feature points of each area of the previous frame image data, and determine the current area where the number of feature points is less than a preset threshold; The determination module is further configured to determine a target area in the current frame image data that is consistent with the current area, and remove the target area from the current frame image data; A construction module, configured to extract target feature points from the filtered frame image data and construct a 3D map according to the target feature points.

[0013] In addition, to achieve the above object, the present application further provides a 3D map construction device, where the device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the 3D map construction method as described above.

[0014] In addition, to achieve the above object, the present application further provides a storage medium, where the storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the 3D map construction method as described above are implemented.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: When it is detected that the positioning system in the intelligent device performs spatial positioning, the feature points of each region of the current frame image data and the previous frame image data are obtained; the number of feature points of each region of the previous frame image data is respectively counted, and the current region where the number of feature points is less than the preset threshold is determined; the target region in the current frame image data that is consistent with the current region is determined, and the target region is removed from the current frame image data; target feature points are extracted from the frame image data after removal, and a three-dimensional map is constructed based on the target feature points. By the above method, after obtaining the feature points of each region of the previous frame image data, the current region with relatively low extraction value for the overall feature points is determined from the dimensions of quantity and distribution quality. Since the difference between adjacent current frame image data and the previous frame image data is small, from this moment on, the target region in the current frame image data that no longer participates in the extraction of target feature points can be determined according to the current region, which can effectively reduce the computational amount of extracting target feature points and basically does not affect the effect of extracting the feature points of the entire image data. Then, a three-dimensional map is constructed based on the target feature points, so that the efficiency of constructing the three-dimensional map can be effectively improved on the basis of ensuring the tracking effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings herein are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application and, together with the specification, are used to explain the principles of the present application.

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a schematic flowchart provided for the first embodiment of the three-dimensional map construction method of the present application; Figure 2 It is a schematic diagram of target region marking provided for the first embodiment of the three-dimensional map construction method of the present application; Figure 3 It is a schematic flowchart provided for the second embodiment of the three-dimensional map construction method of the present application; Figure 4 It is a schematic diagram of the module structure of the three-dimensional map construction device according to the embodiment of the present application; Figure 5 It is a schematic diagram of the device structure of the hardware operating environment involved in the three-dimensional map construction method according to the embodiment of the present application.

[0019] The implementation, functional features, and advantages of the present application will be further described in conjunction with the embodiments with reference to the drawings. Specific implementation manner

[0020] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a three-dimensional map construction device, etc. that can implement the above functions. Hereinafter, taking a three-dimensional map construction device as an example, this embodiment and the following embodiments will be described.

[0021] Based on this, the embodiment of the present application provides a three-dimensional map construction method. Refer to Figure 1 , Figure 1 which is a schematic flowchart of the first embodiment of the three-dimensional map construction method of the present application.

[0022] In this embodiment, the three-dimensional map construction method includes steps S10 to S40: Step S10, when it is detected that the positioning system in the intelligent device performs spatial positioning, obtain the feature points of each region of the current frame image data and the previous frame image data.

[0023] It should be noted that the current frame image data and the previous frame image data are adjacent frame image data. In order to avoid affecting the effect of extracting the feature points of the entire image data, an adjacent dimension gap processing method is used to determine the target region in the current frame image data that no longer participates in extracting the target feature points. The intelligent device can be a head-mounted helmet, and the positioning system can be a SLAM system.

[0024] Further, step S10 includes: when it is detected that the positioning system in the intelligent device performs spatial positioning, respectively obtain the current frame image data and the previous frame image data; determine the feature information of the frame image data collected by the intelligent device, and determine the region division structure according to the feature information; based on the region division structure, respectively perform region division on the current frame image data and the previous frame image data to obtain each region of the current frame image data and the previous frame image data; obtain the feature points of each region of the previous frame image data.

[0025] It should be understood that the region division structure refers to the structure for dividing each frame of image data. The region division structure can be a quadtree, an octree, etc. For intelligent devices with different feature information, the region division structures used are different. After determining the region division structure according to the feature information of the frame image data collected by the intelligent device, the current frame image data and the previous frame image data can be respectively divided into regions, and there are different numbers of feature points in each divided region.

[0026] Step S20, respectively count the number of feature points in each region of the previous frame image data, and determine the current region where the number of feature points is less than the preset threshold.

[0027] It can be understood that the current region refers to the region with relatively low extraction value for the overall feature points. From the analysis of the image data of each frame, it can be seen that the difference between adjacent frame image data is small. If the complete calculation is performed on each frame of image data, and considering the multi-layer pyramid images added for scale invariance, similar calculations are performed on each layer of image data. Therefore, the computational complexity of a single frame of image data is extremely large. At this time, first determine the region where the number of feature points is less than the preset threshold from the quality dimension, and the preset threshold can be 1.

[0028] Step S30: Determine the target region in the current frame of image data that is consistent with the current region, and remove the target region from the current frame of image data.

[0029] It should be understood that the target region refers to the region in the current frame of image data that is in the same position as the current region, that is, optimize the region in the current frame of image data according to the feature points of each region in the previous frame of image data. For example, extract the region with relatively low extraction value for the overall feature points to achieve the purpose of reducing the computational complexity.

[0030] Further, step S30 includes: obtaining the position information of the current region relative to the camera component of the intelligent device; when it is detected that the current frame of image data and the previous frame of image data are completely aligned, determining the target region in the current frame of image data that is consistent with the current region according to the position information; based on a preset marking component, marking the target region according to the target optimization label; and removing the target region carrying the target optimization label from the current frame of image data.

[0031] It can be understood that the position information represents the position of the current region relative to the camera component of the intelligent device, and this position information can be represented by coordinates. When it is detected that the current frame of image data and the previous frame of image data are completely aligned, determine the target region in the current frame of image data that is in the same position as the current region. To effectively improve the efficiency of removing the region, each determined target region needs to be marked. At this time, the target optimization label can be used for marking. The marked target region will no longer participate in the extraction of target feature points. After marking all the target regions that meet the requirements, the target regions carrying the target optimization label are removed from the current frame of image data at one time.

[0032] It should be noted that refer to Figure 2 , Figure 2Schematic diagram for marking the target area. Taking the octree as an example of the area division structure, the specific process is as follows: After the current frame image data and the previous frame image data are respectively divided into areas based on the octree, at this time, the current frame image data and the previous frame image data are divided into multiple 8×8 areas. The difference between the current frame image data and the previous frame image data is extremely small. The upper side represents the previous frame image data for area division, and the lower side represents the current frame image data with the target optimization label. After determining the current area that meets the requirements from the quantity dimension and the area dimension of the region, and when it is detected that the current frame image data and the previous frame image data are completely aligned, the target area in the current frame image data that is in the same position as the current area is determined. The target area is the area with the target optimization label. At this time, the target area with the target optimization label can be removed from the current frame image data. Among them, the "×" symbol represents the target optimization label for marking, and the small circle represents the feature points of each area.

[0033] Step S40: Extract target feature points from the frame image data after removal, and construct a 3D map based on the target feature points.

[0034] It can be understood that the target feature points refer to the feature points used to determine the relative pose transformation information. Compared with the method of performing complete calculations and extracting feature points for each frame of image data, the number of regions for extracting feature points in this embodiment is less, which can effectively reduce the computational amount of extracting target feature points. After extracting the target feature points from the frame image data after removal, a 3D map is constructed based on the target feature points. By extracting the target feature points through the above method of optimizing the region, the computational amount of extracting the target feature points can be effectively reduced, and the quantity and distribution quality of the feature points are considered during optimization, which can ensure the quality of the feature points of the entire image data, and thus ensure the tracking effect.

[0035] Further, step S40 includes: extracting target feature points from the frame image data after removal, and matching the target feature points; extracting target inlier matching pairs from the feature point matching results, and determining relative pose transformation information based on the target inlier matching pairs; constructing 3D map points based on the relative pose transformation information; and constructing a 3D map based on the 3D map points.

[0036] It should be understood that after extracting the target feature points from the frame image data after rejection, the corresponding relationship of the same object in different perspectives in the three-dimensional space is established by matching the target feature points. The target inlier matching pair refers to the point pair that matches the target feature points. The matching algorithm used can be the approximate nearest neighbor matching algorithm. The relative pose transformation information includes but is not limited to rotation, translation information, etc. After determining the target inlier matching pair, the relative pose transformation information can be determined by the epipolar geometry algorithm. On the basis of the already constructed three-dimensional map, the three-dimensional map is updated according to the newly constructed three-dimensional map points, and then the next frame of image data is processed, and so on in a cycle. In addition, the images of each layer of the multi-layer pyramid can also be processed according to the technical solution of this embodiment.

[0037] Further, after step S40, it further includes: obtaining the constructed current three-dimensional map and the serial number of the current frame of image data; when the serial number is not the serial number of the last frame of image data, obtaining the next frame of image data; determining the target area according to the feature points of each area of the next frame of image data and the current frame of image data; and continuing to execute the step of extracting the target feature points from the frame image data after rejection according to the target area.

[0038] It can be understood that for each frame of image data, a corresponding serial number will be carried, and this serial number can be set sequentially according to the order in which the intelligent device collects the frame image data. For example, the serial number of the previous frame of image data is 2, and the serial number of the current frame of image data is 3. When it is determined that the serial number of the current frame of image data is not the serial number of the last frame of image data, it indicates that there is unprocessed frame image data. At this time, the target feature points are extracted from the frame image data after rejection, the three-dimensional map is constructed and updated in the same way as above until all the frame image data is processed.

[0039] When the positioning system in the intelligent device performs spatial positioning in this embodiment, the feature points of each region of the current frame image data and the previous frame image data are obtained; the number of feature points of each region of the previous frame image data is respectively counted, and the current region where the number of the feature points is less than a preset threshold is determined; the target region in the current frame image data that is consistent with the current region is determined, and the target region is removed from the current frame image data; the target feature points are extracted from the frame image data after removal, and a three-dimensional map is constructed based on the target feature points. In the above manner, after obtaining the feature points of each region of the previous frame image data, the current region with relatively low extraction value for the overall feature points is determined from the dimensions of quantity and distribution quality. Since the difference between adjacent previous frame image data and the previous frame image data is small, from this moment on, the target region in the current frame image data that no longer participates in the extraction of target feature points can be determined according to the current region, which can effectively reduce the computational amount of extracting target feature points and basically does not affect the effect of extracting the feature points of the entire image data. Then, a three-dimensional map is constructed based on the target feature points, so that the efficiency of constructing the three-dimensional map can be effectively improved while ensuring the tracking effect.

[0040] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar content as in the above-mentioned first embodiment can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 3 , step S20 includes steps S201 to S204: Step S201, respectively count the number of feature points of each region of the previous frame image data.

[0041] Step S202, compare the number of the feature points with a preset threshold, and determine the region where the number of the feature points is less than the preset threshold according to the comparison result.

[0042] It can be understood that after obtaining the number of feature points of each region of the previous frame image data, the number of feature points of each region of the previous frame image data is compared with a preset threshold, and the region where the number of feature points is less than the preset threshold is determined according to the comparison result. The preset threshold is 1, that is, there are no feature points in this region. At this time, the region with relatively low extraction value for the overall feature points is determined from the quantity dimension.

[0043] Step S203, determine the other regions adjacent to the region in the previous frame image data, and calculate the area of the regions where there are feature points and the area of the regions where there are no feature points in the other regions.

[0044] It should be understood that the other region refers to the region adjacent to the region where the number of feature points is less than the preset threshold. This other region can be located on the left, right, upper, or lower side of this region. At this time, the area of the region with feature points and the area of the region without feature points in the other region are calculated respectively, that is, the region with relatively low extraction value for the overall feature points is determined from the dimension of the region area.

[0045] Step S204: Determine the current region according to the area of the region with feature points and the area of the region without feature points.

[0046] It can be understood that the current region refers to the region with relatively low extraction value for the overall feature points. After calculating the area of the region with feature points and the area of the region without feature points in the other region, the current region with relatively low extraction value for the overall feature points is determined according to the ratio of the region areas.

[0047] Further, step S204 includes: calculating the area ratio between the area of the region with feature points and the area of the region without feature points; when the area ratio is less than the target value, determining the region where the number of feature points is less than the preset threshold as the current region.

[0048] It should be understood that after calculating the area ratio between the area of the region with feature points and the area of the region without feature points, it is judged whether the area ratio is less than the target value. If so, it indicates that the region with feature points in the adjacent region is larger than the region without feature points. At this time, it can be further determined that the distribution of feature points in the region where the number of feature points is less than the preset threshold is uneven and the extraction value for the overall feature points is relatively low. At this time, the region where the number of feature points is less than the preset threshold is determined as the current region, and the target region in the current frame image data that is the same as the current region no longer participates in the extraction of target feature points.

[0049] In this embodiment, the number of feature points in each region of the previous frame of image data is respectively counted; the number of feature points is compared with a preset threshold, and the region where the number of feature points is less than the preset threshold is determined according to the comparison result; other regions adjacent to the region in the previous frame of image data are determined, and the area of the region with feature points and the area of the region without feature points in the other regions are calculated; the current region is determined according to the area of the region with feature points and the area of the region without feature points. In the above manner, after respectively counting the number of feature points in each region of the previous frame of image data, the region where the number of feature points is less than the preset threshold is determined according to the comparison result, that is, the region with relatively low extraction value for the overall feature points is determined from the quantity dimension, and after calculating the area of the region with feature points and the area of the region without feature points in the other regions, the region with relatively low extraction value for the overall feature points is determined from the region area dimension, so as to effectively improve the accuracy of determining the current region.

[0050] This application also provides a three-dimensional map construction device. Please refer to Figure 4 , and the three-dimensional map construction device includes: An acquisition module 10, configured to acquire the feature points of each region of the current frame of image data and the previous frame of image data when it is detected that the positioning system in the intelligent device performs spatial positioning.

[0051] A determination module 20, configured to respectively count the number of feature points in each region of the previous frame of image data and determine the current region where the number of feature points is less than a preset threshold.

[0052] The determination module 20 is further configured to determine a target region in the current frame of image data that is consistent with the current region, and remove the target region from the current frame of image data.

[0053] A construction module 30, configured to extract target feature points from the frame image data after removal and construct a three-dimensional map according to the target feature points.

[0054] When the positioning system in the intelligent device performs spatial positioning in this embodiment, the feature points of each region of the current frame image data and the previous frame image data are obtained; the number of feature points of each region of the previous frame image data is respectively counted, and the current region where the number of feature points is less than the preset threshold is determined; the target region in the current frame image data that is consistent with the current region is determined, and the target region is removed from the current frame image data; the target feature points are extracted from the frame image data after removal, and a three-dimensional map is constructed based on the target feature points. In the above manner, after the feature points of each region of the previous frame image data are obtained, the current region with relatively low extraction value for the overall feature points is determined from the dimensions of quantity and distribution quality. Since the difference between adjacent current frame image data and the previous frame image data is small, from this moment on, the target region in the current frame image data that no longer participates in the extraction of target feature points can be determined according to the current region, which can effectively reduce the computational amount of extracting target feature points and basically does not affect the effect of extracting the feature points of the entire image data. Then, a three-dimensional map is constructed based on the target feature points, so that the efficiency of constructing the three-dimensional map can be effectively improved on the basis of ensuring the tracking effect.

[0055] The three-dimensional map construction device provided in this application adopts the three-dimensional map construction method in the above embodiment, and can solve the technical problem of low efficiency in constructing a three-dimensional map in the prior art. Compared with the prior art, the beneficial effects of the three-dimensional map construction device provided in this application are the same as those of the three-dimensional map construction method provided in the above embodiment, and other technical features in the three-dimensional map construction device are the same as those disclosed in the above embodiment method, and will not be elaborated here.

[0056] In one embodiment, the obtaining module 10 is further configured to, when detecting that the positioning system in the intelligent device performs spatial positioning, respectively obtain the current frame image data and the previous frame image data; determine the feature information of the frame image data collected by the intelligent device, and determine the region division structure according to the feature information; based on the region division structure, respectively perform region division on the current frame image data and the previous frame image data to obtain each region of the current frame image data and the previous frame image data; obtain the feature points of each region of the previous frame image data.

[0057] In one embodiment, the determining module 20 is further configured to respectively count the number of feature points of each region of the previous frame image data; compare the number of feature points with a preset threshold, and determine the region where the number of feature points is less than the preset threshold according to the comparison result; determine other regions adjacent to the region in the previous frame image data, and calculate the area of the region where feature points exist and the area of the region where no feature points exist in the other regions; determine the current region according to the area of the region where feature points exist and the area of the region where no feature points exist.

[0058] In one embodiment, the determining module 20 is further configured to calculate the area ratio between the area of the region with feature points and the area of the region without feature points; when the area ratio is less than a target value, determine the region where the number of feature points is less than a preset threshold as the current region.

[0059] In one embodiment, the determining module 20 is further configured to obtain the position information of the current region relative to the camera component of the intelligent device; when it is detected that the current frame image data is completely aligned with the previous frame image data, determine the target region in the current frame image data that is consistent with the current region according to the position information; based on a preset marking component, mark the target region with a target optimization label; and remove the target region with the target optimization label from the current frame image data.

[0060] In one embodiment, the constructing module 30 is further configured to extract target feature points from the frame image data after removal and match the target feature points; extract target inlier matching pairs from the feature point matching results, and determine relative pose transformation information according to the target inlier matching pairs; construct three-dimensional map points according to the relative pose transformation information; and construct a three-dimensional map according to the three-dimensional map points.

[0061] In one embodiment, the constructing module 30 is further configured to obtain the currently constructed three-dimensional map and the sequence number of the current frame image data; when the sequence number is not the sequence number of the last frame image data, obtain the next frame image data; determine the target region according to the feature points of each region of the next frame image data and the current frame image data; and continue to execute the step of extracting target feature points from the frame image data after removal according to the target region.

[0062] The present application provides a three-dimensional map construction device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the three-dimensional map construction method in the first embodiment above.

[0063] Next, refer to Figure 5, which shows a schematic structural diagram of a three-dimensional map construction device suitable for implementing the embodiments of the present application. The three-dimensional map construction device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description: tablet computers), PMPs (Portable Media Player: portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The three-dimensional map construction device shown is merely an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0064] As Figure 5 shown, the three-dimensional map construction device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a ROM (Read Only Memory) 1002 or a program loaded from a storage device 1003 into a RAM (Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the three-dimensional map construction device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the three-dimensional map construction device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a three-dimensional map construction device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or had alternatively.

[0065] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. The computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by a processing device 1001, the above-mentioned functions defined in the methods of the disclosed embodiments of the present application are executed.

[0066] The three-dimensional map construction device provided by the present application adopts the three-dimensional map construction method in the above embodiment, and can solve the technical problem of low efficiency in constructing a three-dimensional map in the prior art. Compared with the prior art, the beneficial effects of the three-dimensional map construction device provided by the present application are the same as those of the three-dimensional map construction method provided by the above embodiment, and other technical features in the three-dimensional map construction device are the same as those disclosed in the method of the previous embodiment, and will not be elaborated here.

[0067] It should be understood that the various parts disclosed in the present application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0068] As described above, only the specific embodiments of the present application are provided, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all of them should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0069] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the three-dimensional map construction method in the above embodiment.

[0070] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0071] The above computer-readable storage medium can be included in a three-dimensional map construction device; or it can exist independently without being assembled into the three-dimensional map construction device.

[0072] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++; and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0073] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems and methods according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0074] The modules described in the embodiments of the present application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation to the unit itself in some cases.

[0075] The readable storage medium provided by the present application is a computer-readable storage medium, and the computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned three-dimensional map construction method, which can solve the technical problem of low efficiency in constructing three-dimensional maps in the prior art. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as those of the three-dimensional map construction method provided in the above embodiments, and will not be elaborated here.

[0076] The above are only some embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural transformation made by using the specification and drawings of the present application under the technical concept of the present application, or direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A three-dimensional map construction method, characterized in that: The method comprises: When the positioning system in the smart device is detected to perform spatial positioning, feature points of each area of ​​the current frame image data and the previous frame image data are obtained; Counting the number of feature points in each region of the previous frame of image data respectively, and determining a current region where the number of feature points is less than a preset threshold; Determine a target region in the current frame image data that is consistent with the current region, and remove the target region from the current frame image data; Target feature points are extracted from the eliminated frame image data, and a three-dimensional map is constructed based on the target feature points.

2. The method according to claim 1, characterized in that The step of acquiring feature points of each region of the current frame image data and the previous frame image data when the positioning system in the smart device is detected to perform spatial positioning comprises: When the positioning system in the smart device is detected to perform spatial positioning, the current frame image data and the previous frame image data are respectively obtained; Determine feature information of frame image data collected by the smart device, and determine a region division structure according to the feature information; Based on the region division structure, the current frame image data and the previous frame image data are respectively divided into regions to obtain regions of the current frame image data and the previous frame image data; Acquire feature points of each region of the previous frame image data.

3. The method according to claim 1, characterized in that The step of respectively counting the number of feature points of each region of the previous frame image data and determining the current region where the number of feature points is less than a preset threshold comprises: Counting the number of feature points in each area of ​​the previous frame of image data respectively; Comparing the number of the feature points with a preset threshold, and determining an area where the number of the feature points is less than the preset threshold according to the comparison result; Determine other regions adjacent to the region in the previous frame of image data, and calculate the area of ​​the region where the feature points exist and the area where the feature points do not exist in the other regions; The current region is determined according to the area of ​​the region where the feature points exist and the area of ​​the region where the feature points do not exist.

4. The method according to claim 3, characterized in that The step of determining the current area according to the area of ​​the area where the feature points exist and the area of ​​the area where the feature points do not exist comprises: Calculating the area ratio between the area of ​​the region where the feature points exist and the area of ​​the region where the feature points do not exist; When the area ratio is less than the target value, the region in which the number of feature points is less than a preset threshold is determined as the current region.

5. The method according to claim 1, characterized in that The step of determining a target region in the current frame image data that is consistent with the current region and removing the target region from the current frame image data comprises: Obtaining position information of the current area relative to a camera component of a smart device; When it is detected that the current frame image data and the previous frame image data are completely aligned, determining a target area in the current frame image data that is consistent with the current area according to the position information; Based on a preset marking component, marking the target area according to a target optimization label; The target area carrying the target optimization label is removed from the current frame image data.

6. The method according to claim 1, characterized in that The step of extracting target feature points from the eliminated frame image data and constructing a three-dimensional map according to the target feature points includes: Extracting target feature points from the eliminated frame image data, and matching the target feature points; Extracting target inner point matching pairs from the feature point matching results, and determining relative pose transformation information according to the target inner point matching pairs; Constructing a three-dimensional map point according to the relative posture transformation information; A three-dimensional map is constructed according to the three-dimensional map points.

7. The method according to any one of claims 1 to 6, characterized in that After the step of extracting target feature points from the eliminated frame image data and constructing a three-dimensional map according to the target feature points, the method further includes: Obtaining the sequence number of the constructed current three-dimensional map and the current frame image data; When the sequence number is not the sequence number of the last frame of image data, acquiring the next frame of image data; Determine the target area according to the feature points of each area of ​​the next frame image data and the current frame image data; And continue to perform the step of extracting target feature points from the eliminated frame image data according to the target area.

8. A three-dimensional map construction device, characterized in that: The device comprises: An acquisition module, used to acquire feature points of each area of ​​the current frame image data and the previous frame image data when the positioning system in the smart device is detected to perform spatial positioning; A determination module, used to count the number of feature points in each area of ​​the previous frame image data, and determine a current area where the number of feature points is less than a preset threshold; The determination module is further used to determine a target region in the current frame image data that is consistent with the current region, and remove the target region from the current frame image data; The construction module is used to extract target feature points from the eliminated frame image data and construct a three-dimensional map based on the target feature points.

9. A three-dimensional map construction device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the three-dimensional map construction method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the three-dimensional map construction method according to any one of claims 1 to 7 are implemented.

Images