3D Map Construction Method, Device, Equipment and Storage Medium

Abstract

The present application discloses a three-dimensional map construction method, device, equipment and storage medium, relating to the technical field of intelligent devices, including: when detecting that the positioning system in the intelligent device performs spatial positioning, determining a target area in the current frame image data that is consistent with the current area, extracting target feature points from the frame image data after removing the target area, and constructing a three-dimensional map based on the target feature points. By the above method, the current area with relatively low extraction value for the overall feature points is determined from the dimensions of quantity and distribution quality. Since the difference between the adjacent previous frame image data and the previous frame image data is small, the target area in the current frame image data that no longer participates in the extraction of target feature points is determined according to the current area, which can effectively reduce the computational amount of extracting target feature points, and then a three-dimensional map is constructed based on the target feature points, so as to effectively improve the efficiency of constructing the three-dimensional map on the basis of ensuring the tracking effect.

Description

Technical Field

[0001] This application relates to the technical field of intelligent devices, and particularly to a three-dimensional map construction method, device, equipment, and storage medium. Background Art

[0002] With the continuous improvement of intelligent device technology, the interaction between people and intelligent devices has become more frequent. During the interaction process, the intelligent device will use the internal Simultaneous Localization and Mapping (SLAM) system for spatial positioning and construct a three-dimensional map. Obviously, when constructing a three-dimensional map in the above manner, it is necessary to perform complete calculations on each frame of image data to find feature points that meet the requirements, and the computational workload of complete calculations for all frames of image data is extremely large, which will waste a lot of time. Therefore, the efficiency of constructing a three-dimensional map in the above manner is relatively low.

[0003] The above content is only used to assist in understanding the technical solution of this application, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a three-dimensional map construction method, device, equipment, and storage medium, aiming to solve the technical problem of relatively low efficiency in constructing a three-dimensional map in the prior art.

[0005] To achieve the above purpose, this application proposes a three-dimensional map construction method, and the method includes:

[0006] When detecting that the positioning system in the intelligent device performs spatial positioning, obtain the feature points of each region of the current frame of image data and the previous frame of image data;

[0007] Respectively count the number of feature points of each region of the previous frame of image data, and determine the current region where the number of feature points is less than a preset threshold;

[0008] Determine the target region in the current frame of image data that is consistent with the current region, and remove the target region from the current frame of image data;

[0009] Extract target feature points from the image data after removal, and construct a three-dimensional map based on the target feature points.

[0010] In an embodiment, the step of obtaining the feature points of each region of the current frame of image data and the previous frame of image data when detecting that the positioning system in the intelligent device performs spatial positioning includes:

[0011] When detecting that the positioning system in the intelligent device performs spatial positioning, respectively obtain the current frame of image data and the previous frame of image data;

[0012] Determine the feature information of the frame image data collected by the intelligent device, and determine the region division structure according to the feature information;

[0013] Based on the region division structure, respectively perform region division on the current frame image data and the previous frame image data to obtain each region of the current frame image data and the previous frame image data;

[0014] Obtain the feature points of each region of the previous frame image data.

[0015] In one embodiment, the step of respectively counting the number of feature points in each region of the previous frame image data and determining the current region where the number of feature points is less than the preset threshold includes:

[0016] Respectively count the number of feature points in each region of the previous frame image data;

[0017] Compare the number of feature points with the preset threshold, and determine the region where the number of feature points is less than the preset threshold according to the comparison result;

[0018] Determine the other regions adjacent to the region in the previous frame image data, and calculate the area of the regions with feature points and the area of the regions without feature points in the other regions;

[0019] Determine the current region according to the area of the regions with feature points and the area of the regions without feature points.

[0020] In one embodiment, the step of determining the current region according to the area of the regions with feature points and the area of the regions without feature points includes:

[0021] Calculate the area ratio between the area of the regions with feature points and the area of the regions without feature points;

[0022] When the area ratio is less than the target value, determine the region where the number of feature points is less than the preset threshold as the current region.

[0023] In one embodiment, the step of determining the target region in the current frame image data that is consistent with the current region and removing the target region from the current frame image data includes:

[0024] Obtain the position information of the current region relative to the camera component of the intelligent device;

[0025] When it is detected that the current frame image data and the previous frame image data are completely aligned, determine the target region in the current frame image data that is consistent with the current region according to the position information;

[0026] Based on a preset marking component, mark the target area according to the target optimization label;

[0027] Remove the target area with the target optimization label from the current frame image data.

[0028] In one embodiment, the step of extracting target feature points from the frame image data after removal and constructing a three-dimensional map according to the target feature points includes:

[0029] Extract target feature points from the frame image data after removal and match the target feature points;

[0030] Extract target inlier matching pairs from the feature point matching results and determine relative pose transformation information according to the target inlier matching pairs;

[0031] Construct three-dimensional map points according to the relative pose transformation information;

[0032] Construct a three-dimensional map according to the three-dimensional map points.

[0033] In one embodiment, after the step of extracting target feature points from the frame image data after removal and constructing a three-dimensional map according to the target feature points, it further includes:

[0034] Obtain the constructed current three-dimensional map and the serial number of the current frame image data;

[0035] When the serial number is not the serial number of the last frame image data, obtain the next frame image data;

[0036] Determine the target area according to the feature points of each area of the next frame image data and the current frame image data;

[0037] And continue to execute the step of extracting target feature points from the frame image data after removal according to the target area.

[0038] In addition, to achieve the above object, the present application also proposes a three-dimensional map construction device, and the three-dimensional map construction device includes:

[0039] An acquisition module, configured to acquire the feature points of each area of the current frame image data and the previous frame image data when detecting that the positioning system in the intelligent device performs spatial positioning;

[0040] A determination module, configured to respectively count the number of feature points of each area of the previous frame image data and determine the current area where the number of feature points is less than a preset threshold;

[0041] The determining module is further configured to determine a target area in the current frame image data that is consistent with the current area, and remove the target area from the current frame image data;

[0042] The constructing module is configured to extract target feature points from the frame image data after removal, and construct a three-dimensional map based on the target feature points.

[0043] In addition, to achieve the above object, the present application further provides a three-dimensional map construction device, where the device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the three-dimensional map construction method as described above.

[0044] In addition, to achieve the above object, the present application further provides a storage medium, where the storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the three-dimensional map construction method as described above are implemented.

[0045] One or more technical solutions provided by the present application have at least the following technical effects: when it is detected that the positioning system in the intelligent device performs spatial positioning, the feature points of each area of the current frame image data and the previous frame image data are obtained; the number of feature points of each area of the previous frame image data is respectively counted, and the current area where the number of feature points is less than a preset threshold is determined; a target area in the current frame image data that is consistent with the current area is determined, and the target area is removed from the current frame image data; target feature points are extracted from the frame image data after removal, and a three-dimensional map is constructed based on the target feature points. By the above method, after obtaining the feature points of each area of the previous frame image data, the current area with relatively low extraction value for the overall feature points is determined from the dimensions of quantity and distribution quality. Since the difference between adjacent current frame image data and the previous frame image data is small, from this moment on, the target area that no longer participates in the extraction of target feature points in the current frame image data can be determined according to the current area, which can effectively reduce the computational amount of extracting target feature points and basically does not affect the effect of extracting the feature points of the entire image data. Then, a three-dimensional map is constructed based on the target feature points, so that the efficiency of constructing the three-dimensional map can be effectively improved on the basis of ensuring the tracking effect. Description of the Drawings

[0046] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0047] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0048] Figure 1 The flowchart provided for the first embodiment of the three-dimensional map construction method of the present application;

[0049] Figure 2 The schematic diagram of target area marking provided for the first embodiment of the three-dimensional map construction method of the present application;

[0050] Figure 3 The flowchart provided for the second embodiment of the three-dimensional map construction method of the present application;

[0051] Figure 4 The schematic diagram of the module structure of the three-dimensional map construction device according to the embodiment of the present application;

[0052] Figure 5 The schematic diagram of the device structure of the hardware operating environment involved in the three-dimensional map construction method according to the embodiment of the present application.

[0053] The realization of the purpose, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. Specific embodiments

[0054] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a three-dimensional map construction device, etc. that can realize the above functions. Hereinafter, taking the three-dimensional map construction device as an example, this embodiment and the following embodiments will be described.

[0055] Based on this, the embodiments of the present application provide a three-dimensional map construction method, referring to Figure 1 , Figure 1 The flowchart of the first embodiment of the three-dimensional map construction method of the present application.

[0056] In this embodiment, the three-dimensional map construction method includes steps S10 to S40:

[0057] Step S10, when detecting that the positioning system in the intelligent device performs spatial positioning, obtain the feature points of each area of the current frame image data and the previous frame image data.

[0058] It should be noted that the current frame image data and the previous frame image data are adjacent frame image data. In order to avoid affecting the effect of extracting feature points of the entire image data, an adjacent dimension gap processing method is used to determine the target area in the current frame image data that no longer participates in the extraction of target feature points. The intelligent device can be a head-mounted helmet, and the positioning system can be a SLAM system.

[0059] Further, step S10 includes: when it is detected that the positioning system in the intelligent device performs spatial positioning, respectively obtaining the current frame image data and the previous frame image data; determining the feature information of the frame image data collected by the intelligent device, and determining the region division structure according to the feature information; based on the region division structure, respectively performing region division on the current frame image data and the previous frame image data to obtain each region of the current frame image data and the previous frame image data; and obtaining the feature points of each region of the previous frame image data.

[0060] It should be understood that the region division structure refers to the structure for dividing each frame of image data. The region division structure can be a quadtree, an octree, etc. For intelligent devices with different feature information, the region division structures used are different. After determining the region division structure according to the feature information of the frame image data collected by the intelligent device, the current frame image data and the previous frame image data can be respectively divided into regions, and there are different numbers of feature points in each divided region.

[0061] Step S20: respectively count the number of feature points in each region of the previous frame image data, and determine the current region where the number of feature points is less than the preset threshold.

[0062] It can be understood that the current region refers to the region with relatively low extraction value for the overall feature points. From the analysis of each frame of image data, the difference between adjacent frame image data is small. If each frame of image data is calculated completely, and considering the multi-layer pyramid images added for scale invariance and similar calculations are performed on each layer of image data, the computational amount of an entire frame of image data is extremely large. At this time, first determine the region where the number of feature points is less than the preset threshold from the quality dimension. The preset threshold can be 1.

[0063] Step S30: determine the target area in the current frame image data that is consistent with the current area, and remove the target area from the current frame image data.

[0064] It should be understood that the target area refers to the area in the current frame image data that is in the same position as the current area, that is, optimize the area in the current frame image data according to the feature points of each area of the previous frame image data. For example, extract the area with relatively low extraction value for the overall feature points to achieve the purpose of reducing the computational amount.

[0065] Further, step S30 includes: obtaining position information of the current area relative to the camera component of the intelligent device; when it is detected that the current frame image data and the previous frame image data are completely aligned, determining a target area in the current frame image data that is the same as the current area according to the position information; based on a preset marking component, marking the target area with a target optimization label; and removing the target area with the target optimization label from the current frame image data.

[0066] It can be understood that the position information represents the position of the current area relative to the camera component of the intelligent device, and this position information can be represented by coordinates. When it is detected that the current frame image data and the previous frame image data are completely aligned, a target area in the current frame image data that is in the same position as the current area is determined. In order to effectively improve the efficiency of the removed area, each determined target area needs to be marked. At this time, a target optimization label can be used for marking. The marked target area no longer participates in the extraction of target feature points. After marking all target areas that meet the requirements, the target areas with the target optimization label are removed from the current frame image data at one time.

[0067] It should be noted that Figure 2 , Figure 2 is a schematic diagram of marking the target area. Taking the octree as an example of the area division structure for illustration, specifically: after the current frame image data and the previous frame image data are respectively divided into areas based on the octree, at this time, the current frame image data and the previous frame image data are divided into multiple 8×8 areas, and the difference between the current frame image data and the previous frame image data is extremely small. The upper side represents the previous frame image data for area division, and the lower side represents the current frame image data with the target optimization label. After determining the current area that meets the requirements from the quantity dimension and the area dimension of the area, and when it is detected that the current frame image data and the previous frame image data are completely aligned, a target area in the current frame image data that is in the same position as the current area is determined. The target area is the area with the target optimization label. At this time, the target area with the target optimization label can be removed from the current frame image data. Among them, the "×" symbol represents the target optimization label for marking, and the small circle represents the feature points of each area.

[0068] Step S40: Extract target feature points from the frame image data after removal, and construct a three-dimensional map based on the target feature points.

[0069] It can be understood that the target feature points refer to the feature points used to determine the relative pose transformation information. Compared with the method of performing complete calculations and extracting feature points for each frame of image data, the number of regions for extracting feature points in this embodiment is less, which can effectively reduce the computational complexity of extracting the target feature points. After extracting the target feature points from the frame image data after rejection, a three-dimensional map is constructed based on the target feature points. By extracting the target feature points in the above optimized region manner, the computational complexity of extracting the target feature points can be effectively reduced, and when optimizing, the number and distribution quality of the feature points are considered, which can ensure the quality of the feature points of the entire image data, and thus ensure the tracking effect.

[0070] Further, step S40 includes: extracting target feature points from the frame image data after rejection, and matching the target feature points; extracting target inlier matching pairs from the feature point matching results, and determining relative pose transformation information according to the target inlier matching pairs; constructing three-dimensional map points according to the relative pose transformation information; and constructing a three-dimensional map according to the three-dimensional map points.

[0071] It should be understood that after extracting the target feature points from the frame image data after rejection, the corresponding relationship of the same object in the three-dimensional space under different perspectives is established by matching the target feature points. The target inlier matching pairs refer to the point pairs that match the target feature points. The matching algorithm used can be the approximate nearest neighbor matching algorithm. The relative pose transformation information includes, but is not limited to, rotation, translation information, etc. After determining the target inlier matching pairs, the relative pose transformation information can be determined by the epipolar geometry algorithm. On the basis of the already constructed three-dimensional map, the three-dimensional map is updated according to the newly constructed three-dimensional map points, and then the next frame of image data is processed, and so on in a loop. In addition, each layer of the multi-layer pyramid can also be processed according to the technical solution of this embodiment.

[0072] Further, after step S40, it further includes: obtaining the constructed current three-dimensional map and the serial number of the current frame image data; when the serial number is not the serial number of the last frame of image data, obtaining the next frame of image data; determining a target region according to the feature points of each region of the next frame of image data and the current frame of image data; and continuing to execute the step of extracting target feature points from the frame image data after rejection according to the target region.

[0073] It can be understood that for each frame of image data, a corresponding serial number will be carried. This serial number can be set sequentially according to the order in which the intelligent device collects the frame image data. For example, if the serial number of the previous frame of image data is 2 and the serial number of the current frame of image data is 3, when it is determined that the serial number of the current frame of image data is not the serial number of the last frame of image data, it indicates that there is unprocessed frame image data. At this time, the target feature points are extracted from the frame image data after rejection, and the three-dimensional map is constructed and updated in the same way as above until all the frame image data is processed.

[0074] In this embodiment, when it is detected that the positioning system in the intelligent device performs spatial positioning, the feature points of each region of the current frame of image data and the previous frame of image data are obtained; the number of feature points in each region of the previous frame of image data is respectively counted, and the current region where the number of the feature points is less than the preset threshold is determined; the target region in the current frame of image data that is consistent with the current region is determined, and the target region is removed from the current frame of image data; the target feature points are extracted from the frame image data after rejection, and the three-dimensional map is constructed according to the target feature points. By the above method, after obtaining the feature points of each region of the previous frame of image data, the current region with relatively low extraction value for the overall feature points is determined from the dimensions of quantity and distribution quality. Since the difference between adjacent previous frame image data and the previous frame of image data is small, from this moment on, the target region in the current frame of image data that no longer participates in the extraction of target feature points can be determined according to the current region, which can effectively reduce the computational amount of extracting target feature points and basically does not affect the effect of extracting the feature points of the entire image data. Then, the three-dimensional map is constructed according to the target feature points, so that the efficiency of constructing the three-dimensional map can be effectively improved on the basis of ensuring the tracking effect.

[0075] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar content as that in the above-mentioned first embodiment can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 3 , step S20 includes steps S201 to S204:

[0076] Step S201, respectively count the number of feature points in each region of the previous frame of image data.

[0077] Step S202, compare the number of the feature points with the preset threshold, and determine the region where the number of the feature points is less than the preset threshold according to the comparison result.

[0078] It can be understood that after obtaining the number of feature points in each region of the previous frame of image data, the number of feature points in each region of the previous frame of image data is compared with a preset threshold, and the regions where the number of feature points is less than the preset threshold are determined according to the comparison result. The preset threshold is 1, that is, there are no feature points in this region. At this time, the regions with relatively low extraction value for the overall feature points are determined from the quantity dimension.

[0079] Step S203: Determine other regions adjacent to the region in the previous frame of image data, and calculate the area of the regions with feature points and the area of the regions without feature points in the other regions.

[0080] It should be understood that the other regions refer to the regions adjacent to the regions where the number of feature points is less than the preset threshold. These other regions can be located on the left, right, upper or lower side of this region. At this time, calculate the area of the regions with feature points and the area of the regions without feature points in the other regions respectively, that is, determine the regions with relatively low extraction value for the overall feature points from the regional area dimension.

[0081] Step S204: Determine the current region according to the area of the regions with feature points and the area of the regions without feature points.

[0082] It can be understood that the current region refers to the region with relatively low extraction value for the overall feature points. After calculating the area of the regions with feature points and the area of the regions without feature points in the other regions, the current region with relatively low extraction value for the overall feature points is determined according to the ratio of the regional areas.

[0083] Further, step S204 includes: calculating the area ratio between the area of the regions with feature points and the area of the regions without feature points; when the area ratio is less than the target value, determining the region where the number of feature points is less than the preset threshold as the current region.

[0084] It should be understood that after calculating the area ratio between the area of the regions with feature points and the area of the regions without feature points, it is judged whether the area ratio is less than the target value. If so, it indicates that the regions with feature points in the adjacent regions are larger than the regions without feature points. At this time, it can be further determined that the distribution of feature points in the region where the number of feature points is less than the preset threshold is uneven, and the extraction value for the overall feature points is relatively low. At this time, determine the region where the number of feature points is less than the preset threshold as the current region, and the target region in the current frame of image data that is the same as the current region will no longer participate in the extraction of target feature points.

[0085] In this embodiment, the number of feature points in each region of the previous frame of image data is respectively counted; the number of feature points is compared with a preset threshold, and the regions where the number of feature points is less than the preset threshold are determined according to the comparison result; other regions adjacent to the regions in the previous frame of image data are determined, and the area of the regions with feature points and the area of the regions without feature points in the other regions are calculated; the current region is determined according to the area of the regions with feature points and the area of the regions without feature points. In the above manner, after the number of feature points in each region of the previous frame of image data is respectively counted, the regions where the number of feature points is less than the preset threshold are determined according to the comparison result, that is, the regions with relatively low extraction value for the overall feature points are determined from the quantity dimension, and after the area of the regions with feature points and the area of the regions without feature points in the other regions are calculated, the regions with relatively low extraction value for the overall feature points are determined from the regional area dimension, so as to effectively improve the accuracy of determining the current region.

[0086] This application also provides a three-dimensional map construction device. Please refer to Figure 4 , the three-dimensional map construction device includes:

[0087] An acquisition module 10, configured to acquire the feature points of each region of the current frame of image data and the previous frame of image data when it is detected that the positioning system in the intelligent device performs spatial positioning.

[0088] A determination module 20, configured to respectively count the number of feature points in each region of the previous frame of image data and determine the current region where the number of feature points is less than a preset threshold.

[0089] The determination module 20 is further configured to determine a target region in the current frame of image data that is the same as the current region, and remove the target region from the current frame of image data.

[0090] A construction module 30, configured to extract target feature points from the image data after removal and construct a three-dimensional map according to the target feature points.

[0091] When the positioning system in the intelligent device performs spatial positioning is detected in this embodiment, the feature points of each region of the current frame image data and the previous frame image data are obtained; the number of feature points of each region of the previous frame image data is respectively counted, and the current region where the number of the feature points is less than a preset threshold is determined; the target region in the current frame image data that is consistent with the current region is determined, and the target region is removed from the current frame image data; the target feature points are extracted from the frame image data after removal, and a three-dimensional map is constructed according to the target feature points. By the above method, after the feature points of each region of the previous frame image data are obtained, the current region with relatively low extraction value for the overall feature points is determined from the dimensions of quantity and distribution quality. Since the difference between adjacent current frame image data and the previous frame image data is small, from this moment on, the target region in the current frame image data that no longer participates in the extraction of the target feature points can be determined according to the current region, which can effectively reduce the computational amount of extracting the target feature points and basically does not affect the effect of extracting the feature points of the entire image data. Then, a three-dimensional map is constructed according to the target feature points, so that the efficiency of constructing the three-dimensional map can be effectively improved on the basis of ensuring the tracking effect.

[0092] The three-dimensional map construction device provided by the present application adopts the three-dimensional map construction method in the above embodiment, and can solve the technical problem of low efficiency of constructing a three-dimensional map in the prior art. Compared with the prior art, the beneficial effects of the three-dimensional map construction device provided by the present application are the same as those of the three-dimensional map construction method provided by the above embodiment, and other technical features in the three-dimensional map construction device are the same as those disclosed in the method of the above embodiment, and will not be elaborated here.

[0093] In one embodiment, the obtaining module 10 is further configured to respectively obtain the current frame image data and the previous frame image data when the positioning system in the intelligent device performs spatial positioning is detected; determine the feature information of the frame image data collected by the intelligent device, and determine the region division structure according to the feature information; based on the region division structure, respectively perform region division on the current frame image data and the previous frame image data to obtain each region of the current frame image data and the previous frame image data; obtain the feature points of each region of the previous frame image data.

[0094] In one embodiment, the determining module 20 is further configured to respectively count the number of feature points of each region of the previous frame image data; compare the number of the feature points with a preset threshold, and determine the region where the number of the feature points is less than the preset threshold according to the comparison result; determine other regions adjacent to the region in the previous frame image data, and calculate the area of the region where there are feature points and the area of the region where there are no feature points in the other regions; determine the current region according to the area of the region where there are feature points and the area of the region where there are no feature points.

[0095] In one embodiment, the determining module 20 is further configured to calculate an area ratio between the area of the region with feature points and the area of the region without feature points; when the area ratio is less than a target value, determine the region where the number of feature points is less than a preset threshold as the current region.

[0096] In one embodiment, the determining module 20 is further configured to obtain position information of the current region relative to the camera component of the intelligent device; when it is detected that the current frame image data is completely aligned with the previous frame image data, determine a target region in the current frame image data that is consistent with the current region according to the position information; based on a preset marking component, mark the target region with a target optimization label; and remove the target region with the target optimization label from the current frame image data.

[0097] In one embodiment, the constructing module 30 is further configured to extract target feature points from the frame image data after removal, and match the target feature points; extract target inlier matching pairs from the feature point matching results, and determine relative pose transformation information according to the target inlier matching pairs; construct three-dimensional map points according to the relative pose transformation information; and construct a three-dimensional map according to the three-dimensional map points.

[0098] In one embodiment, the constructing module 30 is further configured to obtain the currently constructed three-dimensional map and the sequence number of the current frame image data; when the sequence number is not the sequence number of the last frame image data, obtain the next frame image data; determine a target region according to the feature points of each region of the next frame image data and the current frame image data; and continue to execute the step of extracting target feature points from the frame image data after removal according to the target region.

[0099] The present application provides a three-dimensional map construction device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the three-dimensional map construction method in the first embodiment above.

[0100] Next, refer to Figure 5, which shows a schematic structural diagram of a three-dimensional map construction device suitable for implementing the embodiments of the present application. The three-dimensional map construction device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The three-dimensional map construction device shown is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present application.

[0101] As Figure 5 shown, the three-dimensional map construction device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the ROM (Read Only Memory) 1002 or the program loaded from the storage device 1003 into the RAM (Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the three-dimensional map construction device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the three-dimensional map construction device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a three-dimensional map construction device with various systems, it should be understood that it is not required to implement or have all the shown systems. Instead, more or fewer systems may be implemented or had.

[0102] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. The computer programs contain program codes for performing the methods shown in the flowcharts. In such embodiments, the computer programs can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer programs are executed by a processing device 1001, the above functions defined in the methods of the disclosed embodiments of the present application are performed.

[0103] The three-dimensional map construction device provided by the present application adopts the three-dimensional map construction method in the above embodiments, and can solve the technical problem of low efficiency in constructing a three-dimensional map in the prior art. Compared with the prior art, the beneficial effects of the three-dimensional map construction device provided by the present application are the same as those of the three-dimensional map construction method provided by the above embodiments, and other technical features in the three-dimensional map construction device are the same as those disclosed in the method of the previous embodiment, which will not be elaborated here.

[0104] It should be understood that the various parts disclosed in the present application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0105] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0106] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the three-dimensional map construction method in the above embodiments.

[0107] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0108] The above computer-readable storage medium can be included in a three-dimensional map construction device; it can also exist independently without being assembled into the three-dimensional map construction device.

[0109] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++; they also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0110] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems and methods according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0111] The modules described in the embodiments of the present application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation on the unit itself in some cases.

[0112] The readable storage medium provided by the present application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned three-dimensional map construction method, and can solve the technical problem of low efficiency in constructing three-dimensional maps in the prior art. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as those of the three-dimensional map construction method provided in the above embodiments, and will not be elaborated here.

[0113] The above are only some embodiments of the present application, and do not limit the patent scope of the present application accordingly. Any equivalent structural transformation made by using the content of the specification and drawings of the present application under the technical concept of the present application, or any direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A three-dimensional map construction method, characterized in that, The method includes: When it is detected that the positioning system in the intelligent device performs spatial positioning, obtaining the feature points of each region of the current frame image data and the previous frame image data; Respectively counting the number of feature points of each region of the previous frame image data, and determining the current region where the number of feature points is less than a preset threshold; Determining the target region in the current frame image data that is consistent with the current region, and removing the target region from the current frame image data; Extracting target feature points from the frame image data after removal, and constructing a three-dimensional map based on the target feature points.

2. The method according to claim 1, characterized in that The step of, when it is detected that the positioning system in the intelligent device performs spatial positioning, obtaining the feature points of each region of the current frame image data and the previous frame image data includes: When it is detected that the positioning system in the intelligent device performs spatial positioning, respectively obtaining the current frame image data and the previous frame image data; Determining the feature information of the frame image data collected by the intelligent device, and determining the region division structure according to the feature information; Based on the region division structure, respectively dividing the current frame image data and the previous frame image data into regions, obtaining each region of the current frame image data and the previous frame image data; Obtaining the feature points of each region of the previous frame image data.

3. The method according to claim 1, characterized in that, The step of respectively counting the number of feature points of each region of the previous frame image data, and determining the current region where the number of feature points is less than a preset threshold includes: Respectively counting the number of feature points of each region of the previous frame image data; Comparing the number of feature points with a preset threshold, and determining the region where the number of feature points is less than the preset threshold according to the comparison result; Determining other regions adjacent to the region in the previous frame image data, and calculating the area of the region where feature points exist and the area of the region where no feature points exist in the other regions; Determining the current region according to the area of the region where feature points exist and the area of the region where no feature points exist.

4. The method according to claim 3, wherein The step of determining the current region according to the area of the region where feature points exist and the area of the region where no feature points exist includes: Calculating the area ratio between the area of the region where feature points exist and the area of the region where no feature points exist; When the area ratio is less than a target value, determining the region where the number of feature points is less than the preset threshold as the current region.

5. The method according to claim 1, characterized in that, The step of determining the target region in the current frame image data that is consistent with the current region, and removing the target region from the current frame image data includes: Obtaining the position information of the current region relative to the camera component of the intelligent device; When it is detected that the current frame image data and the previous frame image data are completely aligned, determining the target region in the current frame image data that is consistent with the current region according to the position information; Based on a preset marking component, marking the target region according to the target optimization label; Removing the target region with the target optimization label from the current frame image data.

6. The method according to claim 1, characterized in that, The step of extracting target feature points from the frame image data after removal, and constructing a three-dimensional map based on the target feature points includes: Extract target feature points from the filtered frame image data and match the target feature points; Extract target inlier matching pairs from the feature point matching results and determine relative pose transformation information according to the target inlier matching pairs; Construct 3D map points according to the relative pose transformation information; Construct a 3D map according to the 3D map points.

7. The method according to any one of claims 1 to 6, characterized in that After the steps of extracting target feature points from the filtered frame image data and constructing a 3D map according to the target feature points, the method further includes: Obtain the constructed current 3D map and the serial number of the current frame image data; When the serial number is not the serial number of the last frame image data, obtain the next frame image data; Determine a target area according to the feature points of each area of the next frame image data and the current frame image data; And continue to execute the step of extracting target feature points from the filtered frame image data according to the target area.

8. A three-dimensional map construction device, characterized in that, The device includes: An acquisition module, configured to acquire the feature points of each area of the current frame image data and the previous frame image data when detecting that the positioning system in the intelligent device performs spatial positioning; A determination module, configured to respectively count the number of feature points of each area of the previous frame image data and determine the current area where the number of feature points is less than a preset threshold; The determination module is further configured to determine a target area in the current frame image data that is consistent with the current area, and remove the target area from the current frame image data; A construction module, configured to extract target feature points from the filtered frame image data and construct a 3D map according to the target feature points.

9. A three-dimensional map construction device, characterized in that, The device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the computer program is configured to implement the steps of the 3D map construction method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the 3D map construction method according to any one of claims 1 to 7 are implemented.

Images