Map system initialization method, device, equipment and storage medium

Abstract

The present invention belongs to the field of computer technology, and discloses a map system initialization method, device, equipment and storage medium. The present invention obtains a first-frame feature point set when a non-first-frame image is collected; performs feature tracking and matching on each first-frame feature point in the first-frame feature point set according to the optical flow method, and determines the region of interest corresponding to each first-frame feature point; extracts similar feature points of interest corresponding to each first-frame feature point from the region of interest to obtain multiple pairs of matching feature point pairs; and initializes the map system based on the multiple pairs of matching feature point pairs. Because each first-frame feature point is tracked and matched first when constructing the matching feature point pair, and the region of interest corresponding to each first-frame feature point is determined, and then similar feature points of interest corresponding to the first-frame feature point are extracted from the region of interest to construct the matching feature point pair, the area to be searched is greatly reduced without affecting the feature tracking, and the matching process is accelerated, thereby improving the system initialization efficiency.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a map system initialization method, device, equipment and storage medium. Background Art

[0002] Vision-based simultaneous localization and mapping (SLAM) technology is widely used in industries such as robotics, autonomous driving, and VR / AR. When in use, the SLAM system (referred to as the map system) needs to initially construct a map of a small area of ​​the surrounding environment, and then perform incremental construction based on the initialized map. Therefore, the results of the map system initialization directly affect the subsequent tracking accuracy and stability of SLAM. The map system initialization stage is the most computationally intensive and time-consuming link. If the initialization takes a long time, subsequent image frames will not be able to track, locate, and build maps. Therefore, the processing speed of the SLAM initialization stage directly affects the real-time and stability of the subsequent tracking of the map system. Therefore, the execution efficiency of the map system during initialization is very important.

[0003] However, the existing technology has many disadvantages when initializing the map system:

[0004] 1. During feature matching, the proportion of successfully matched feature points is low. Most of the features extracted from subsequent images do not appear in the first frame image. Therefore, a large number of points cannot be matched with features.

[0005] 2. In feature matching, the position obtained from the previous frame matching is used to search for feature points in the current frame. The search range is an area with a length of 2r and a width of 2r (r is the preset search range length). If the search area is small, it is easy to fail to match feature points; if the search area is large, the matching calculation is large and time-consuming.

[0006] 3. Posture solution and 3D reconstruction are frequent and time-consuming. After each frame matching is processed, it is calculated once. Usually, it takes multiple frames before initialization is successful. The results obtained before success are invalid, resulting in low overall initialization efficiency.

[0007] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art. Summary of the Invention

[0008] The main purpose of the present invention is to provide a map system initialization method, device, equipment and storage medium, aiming to solve the technical problem of low efficiency in initializing SLAM systems in the existing technology.

[0009] To achieve the above object, the present invention provides a method for initializing a map system, the method comprising the following steps:

[0010] When a non-first frame image is collected, a first frame feature point set is obtained;

[0011] Performing feature tracking and matching on each first-frame feature point in the first-frame feature point set according to the optical flow method to determine the region of interest corresponding to each first-frame feature point;

[0012] Extract similar feature points of interest corresponding to the feature points of each first frame from the region of interest to obtain multiple pairs of matching feature points;

[0013] A map system is initialized according to the multiple pairs of matching feature points.

[0014] Optionally, the step of performing feature tracking and matching on each first-frame feature point in the first-frame feature point set according to the optical flow method to determine the region of interest corresponding to each first-frame feature point includes:

[0015] Tracking each feature point in the first frame feature point set according to the optical flow method to obtain a corresponding non-first frame feature point set;

[0016] Traversing the non-first frame feature point set, and using the traversed feature points as current non-first frame feature points;

[0017] Determine the region of interest corresponding to the feature point of the current non-first frame according to the feature point coordinates of the feature point of the current non-first frame and a preset range threshold;

[0018] At the end of the traversal, the region of interest corresponding to the feature points of each first frame is obtained.

[0019] Optionally, before the step of traversing the non-first frame feature point set and using the traversed feature points as the current non-first frame feature points, the method further includes:

[0020] Obtaining the tracking status corresponding to each feature point in the non-first frame feature point set;

[0021] The corresponding feature points whose tracking status is tracking failure are removed from the non-first frame feature point set.

[0022] Optionally, the step of extracting similar interest feature points corresponding to the feature points of each first frame from the region of interest to obtain multiple pairs of matching feature point pairs includes:

[0023] Traversing the first frame feature point set, and using the traversed first frame feature points as current first frame feature points;

[0024] Extracting feature points from the region of interest corresponding to the feature points of the current first frame to obtain multiple feature points of interest;

[0025] Calculating the feature matching value between each feature point of interest and the feature point of the current first frame;

[0026] Taking the maximum value among the feature matching values ​​as the target feature matching value;

[0027] If the target feature matching value is greater than a preset matching threshold, the interest feature point corresponding to the target feature matching value is used as a similar interest feature point corresponding to the current non-first frame feature point;

[0028] Constructing a matching feature point pair based on the similar interest feature points and the current first frame feature points;

[0029] At the end of the traversal, multiple pairs of matching feature points are obtained.

[0030] Optionally, the step of initializing a map system based on the multiple pairs of matching feature points includes:

[0031] Calculate the coordinate distance between the feature points in each pair of matching feature points to obtain multiple feature coordinate distances;

[0032] Obtaining an average value of the plurality of feature coordinate distances to obtain a feature movement distance;

[0033] If the feature movement distance is greater than or equal to a preset movement threshold, calculating the center point distance based on the multiple pairs of matching feature points;

[0034] When the feature movement distance and the center point distance meet a preset initialization condition, a map system is initialized according to the multiple pairs of matching feature points.

[0035] Optionally, the step of calculating the center point distance based on the multiple pairs of matching feature points includes:

[0036] Selecting a preset number of matching feature point pairs from the multiple matching feature point pairs to obtain target matching feature point pairs;

[0037] Determining a first center point based on the target feature point pair, and determining a center operation weight based on a preset weight matrix;

[0038] Determine a second center point according to the first center point and the center operation weight;

[0039] A center point distance is calculated based on the first center point and the second center point.

[0040] Optionally, before the step of initializing the map system according to the multiple pairs of matching feature points when the feature movement distance and the center point distance meet a preset initialization condition, the method further includes:

[0041] If the characteristic movement distance is greater than or equal to a preset movement determination threshold, and / or the center point distance is greater than or equal to a preset center determination threshold, it is determined that the characteristic movement distance and the center point distance meet a preset initialization condition.

[0042] In addition, to achieve the above-mentioned purpose, the present invention further proposes a map system initialization device, which includes the following modules:

[0043] The image acquisition module is used to obtain the feature point set of the first frame when a non-first frame image is acquired;

[0044] A feature tracking module is used to perform feature tracking and matching on each first-frame feature point in the first-frame feature point set according to an optical flow method, and determine a region of interest corresponding to each first-frame feature point;

[0045] A feature matching module is used to extract similar feature points of interest corresponding to the feature points of each first frame from the region of interest to obtain multiple pairs of matching feature points;

[0046] The system debugging module is used to initialize the map system according to the multiple pairs of matching feature points.

[0047] In addition, to achieve the above-mentioned purpose, the present invention also proposes a map system initialization device, which includes: a processor, a memory, and a map system initialization program stored in the memory and executable on the processor. When the map system initialization program is executed by the processor, the steps of the map system initialization method described above are implemented.

[0048] In addition, to achieve the above-mentioned purpose, the present invention further proposes a computer-readable storage medium, on which a map system initialization program is stored. When the map system initialization program is executed, the steps of the map system initialization method described above are implemented.

[0049] The present invention obtains a first-frame feature point set when a non-first-frame image is collected; performs feature tracking and matching on each first-frame feature point in the first-frame feature point set based on the optical flow method to determine the region of interest corresponding to each first-frame feature point; extracts similar feature points of interest corresponding to each first-frame feature point from the region of interest to obtain multiple pairs of matching feature point pairs; and initializes the map system based on the multiple pairs of matching feature point pairs. Since each first-frame feature point is first tracked and matched when constructing the matching feature point pairs, the region of interest corresponding to each first-frame feature point is determined, and then similar feature points of interest corresponding to the first-frame feature point are extracted from the region of interest to construct the matching feature point pairs, the area to be searched is greatly reduced without affecting feature tracking, and the matching process is accelerated, thereby improving the efficiency of system initialization. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic diagram of the structure of an electronic device in the hardware operating environment involved in the embodiment of the present invention;

[0051] Figure 2 This is a flow chart of a first embodiment of a method for initializing a map system according to the present invention;

[0052] Figure 3 This is a flow chart of a second embodiment of a method for initializing a map system according to the present invention;

[0053] Figure 4 This is a structural block diagram of the first embodiment of the map system initialization device of the present invention.

[0054] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0055] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0056] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a map system initialization device in the hardware operating environment involved in the embodiment of the present invention.

[0057] like Figure 1 As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0058] Those skilled in the art will understand that Figure 1The structure shown in the figure does not constitute a limitation to the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0059] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a map system initialization program.

[0060] exist Figure 1 In the electronic device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in the map system initialization device, and the electronic device calls the map system initialization program stored in the memory 1005 through the processor 1001 and executes the map system initialization method provided by the embodiment of the present invention.

[0061] The embodiment of the present invention provides a method for initializing a map system, referring to Figure 2 , Figure 2 FIG. 4 is a flow chart of a first embodiment of a method for initializing a map system according to the present invention.

[0062] In this embodiment, the map system initialization method includes the following steps:

[0063] Step S10: when a non-first frame image is captured, a first frame feature point set is obtained.

[0064] It should be noted that the execution subject of this embodiment can be the map system initialization device, and the map system initialization device can be an electronic device such as a personal computer, a server, or other devices that can achieve the same or similar functions. In this embodiment and the following embodiments, the map system initialization method of the present invention is described using the map system initialization device as an example.

[0065] It should be noted that the first frame image can be the first frame image collected by the map system initialization device when starting the SLAM system initialization, or it can be the first frame image re-collected after the previous initialization failed. The non-first frame image can be an image collected after the first frame image is collected. The first frame feature point set can be a set containing multiple first frame feature point information, wherein the first frame feature point information can be the feature point information of the image feature point in the first frame image, and the feature point information can include: the coordinates of the feature point in the image, the scale information of the feature point, the feature point descriptor, the starting frame number of the feature point, the most recent frame number of the feature point, the number of consecutive frames lost by the feature point, and other information.

[0066] Step S20: performing feature tracking and matching on each first-frame feature point in the first-frame feature point set according to the optical flow method to determine the region of interest corresponding to each first-frame feature point.

[0067] It should be noted that the optical flow method uses the temporal changes of pixels in an image sequence and the correlation between adjacent frames to find the correspondence between the previous frame and the current frame, thereby calculating the motion information of objects between adjacent frames. The optical flow method can track each feature point in the first frame feature point set, determine the image feature point corresponding to each feature point in the non-first frame image, and then determine four region vertices based on the feature point coordinates corresponding to the corresponding image feature point in the non-first frame image and a preset range threshold. Based on the region vertices, the region of interest corresponding to the first frame feature point can be determined.

[0068] Furthermore, in order to quickly and accurately determine the region of interest corresponding to each first frame feature point, step S20 in this embodiment may include:

[0069] Tracking each feature point in the first frame feature point set according to the optical flow method to obtain a corresponding non-first frame feature point set;

[0070] Traversing the non-first frame feature point set, and using the traversed feature points as current non-first frame feature points;

[0071] Determine the region of interest corresponding to the feature point of the current non-first frame according to the feature point coordinates of the feature point of the current non-first frame and a preset range threshold;

[0072] At the end of the traversal, the region of interest corresponding to the feature points of each first frame is obtained.

[0073] It should be noted that according to the optical flow method, each feature point in the first frame feature point set can be tracked, the image feature point corresponding to each feature point in the non-first frame image can be determined, and then the feature point information of the corresponding image feature points in the non-first frame image can be extracted and aggregated to obtain the corresponding non-first frame feature point set.

[0074] In actual use, traversing the non-first frame image feature point set can be traversing the non-first frame image feature point set from small to large according to the set index of the non-first frame image feature point set. Of course, other traversal methods can also be used, and this embodiment does not limit this.

[0075] It should be noted that determining the region of interest corresponding to the feature point of the current non-first frame based on the feature point coordinates of the feature point of the current non-first frame and the preset range threshold can be performed by obtaining the feature point coordinates of the feature point of the current non-first frame, and then determining four region vertices based on the feature point coordinates and the preset range threshold, and then determining the region of interest based on the region vertices. For example: assuming that the feature point coordinates of the feature point of the current non-first frame are (x, y), and the preset range threshold is r, then the coordinates of the four region vertices are (xr, yr), (x+r, yr), (xr, y+r), and (x+r, y+r), respectively. The region of interest can be determined based on the four region vertices.

[0076] In a specific implementation, in order to further narrow the scope of the area of ​​interest to be searched, before determining the area of ​​interest corresponding to the current non-first frame feature point based on the feature point coordinates of the current non-first frame feature point and the preset range threshold, the first frame feature point corresponding to the current non-first frame feature point in the first frame feature point set can also be obtained, and then the pyramid layer where the first frame feature point is located is obtained, and the tower layer range with extraction is determined based on the pyramid layer and the preset tower layer threshold, and then the area of ​​interest is further constrained by the tower layer range. For example: assuming that the golden tower layer where the first frame feature point is located is L, and the preset tower layer threshold is R, then the tower layer range is [LR, L+R], and the area occupied by the tower layer range in the image can be obtained, and then the intersection area of ​​the area occupied by the tower layer range in the image and the area of ​​interest is obtained, and the intersection area is used as the new area of ​​interest.

[0077] In a specific implementation, since feature point tracking inevitably results in feature points that fail to be tracked, in order to avoid such feature points occupying computing resources, before the step of traversing the non-first frame feature point set and using the traversed feature points as the current non-first frame feature points in this embodiment, the following steps may also be included:

[0078] Obtaining the tracking status corresponding to each feature point in the non-first frame feature point set;

[0079] The corresponding feature points whose tracking status is tracking failure are removed from the non-first frame feature point set.

[0080] It should be noted that if the tracking status is tracking failure, it means that the feature point is not actually tracked in the non-first frame image. Therefore, the feature point with the tracking status of tracking failure can be removed from the non-first frame feature point set to avoid traversing such feature points and reduce unnecessary performance overhead.

[0081] Step S30: extracting similar interest feature points corresponding to the feature points of each first frame from the region of interest to obtain multiple pairs of matching feature point pairs.

[0082] It should be noted that the similar feature point of interest may be the feature point in the region of interest that has the highest similarity to the feature point of the first frame. Extracting similar feature points of interest corresponding to the feature points of the first frame from the region of interest to obtain matching feature point pairs may be performed by extracting similar feature points of interest corresponding to each first frame feature point from the region of interest corresponding to each first frame feature point, obtaining feature point information of the similar feature points, and then aggregating the feature point information with the feature point information of the first frame feature point to obtain matching feature point pairs.

[0083] Furthermore, in order to ensure the validity of the obtained similar interest feature points and to ensure that the obtained matching feature point pairs are valid matching feature point pairs, step S30 in this embodiment may include:

[0084] Traversing the first frame feature point set, and using the traversed first frame feature points as current first frame feature points;

[0085] Extracting feature points from the region of interest corresponding to the feature points of the current first frame to obtain multiple feature points of interest;

[0086] Calculating the feature matching value between each feature point of interest and the feature point of the current first frame;

[0087] Taking the maximum value among the feature matching values ​​as the target feature matching value;

[0088] If the target feature matching value is greater than a preset matching threshold, the interest feature point corresponding to the target feature matching value is used as a similar interest feature point corresponding to the current non-first frame feature point;

[0089] Constructing a matching feature point pair based on the similar interest feature points and the current first frame feature points;

[0090] At the end of the traversal, multiple pairs of matching feature points are obtained.

[0091] In actual use, calculating the feature matching value between each feature point of interest and the feature point of the current first frame can be to extract the descriptor of the feature point of interest, obtain the descriptor of the feature point of the current first frame, and then perform similarity matching between the descriptor of the feature point of interest and the descriptor of the feature point of the current first frame, so as to obtain the feature matching value. In particular, when performing similarity matching, a preset similarity model can be used for similarity matching, and the preset similarity model can be a pre-trained neural network model. Taking the maximum value among the feature matching values ​​as the target feature matching value can be to sort the obtained feature matching values ​​from large to small, and then use the feature matching value ranked first as the target matching value.

[0092] It should be noted that the preset matching threshold can be pre-set by the administrator of the map system initialization device. If the target feature matching value is greater than the preset matching threshold, it means that the interest feature point corresponding to the target feature matching value has a high similarity with the feature point of the current first frame. In this case, the interest feature point corresponding to the target feature matching value can be used as a similar interest feature point.

[0093] In actual use, if the target feature matching value is less than or equal to the preset matching threshold, it means that the similarity between the feature points of interest extracted at this time and the feature points of the current first frame is low. At this time, it can be determined that the feature points corresponding to the feature points of the current first frame are not matched. At this time, it is not necessary to construct a matching feature point pair and directly perform subsequent traversal.

[0094] Step S40: Initializing a map system according to the plurality of matching feature point pairs.

[0095] In actual use, initializing the map system based on multiple pairs of matching feature points can be done by using epipolar constraints to screen out the correct matching feature point pairs from the multiple matching feature point pairs, then calculating the basic matrix F based on the correct matching feature point pairs, decomposing the essential matrix, and obtaining the pose parameters: rotation R and translation t; using the pose parameters, calculating the three-dimensional coordinates of the matching feature point pairs, and calculating the back-projection error. If the error is less than a set threshold, the three-dimensional coordinates are considered correct; counting the number of points with correct three-dimensional coordinates. If the number is greater than the threshold, the initialization is considered successful.

[0096] This embodiment obtains a first-frame feature point set when a non-first-frame image is captured; performs feature tracking and matching on each first-frame feature point in the first-frame feature point set based on the optical flow method to determine the region of interest corresponding to each first-frame feature point; extracts similar feature points of interest corresponding to each first-frame feature point from the region of interest to obtain multiple pairs of matching feature point pairs; and initializes the map system based on the multiple pairs of matching feature point pairs. Because each first-frame feature point is first tracked and matched when constructing the matching feature point pairs, the region of interest corresponding to each first-frame feature point is determined, and then similar feature points of interest corresponding to the first-frame feature point are extracted from the region of interest to construct the matching feature point pairs, the area to be searched is greatly reduced without affecting feature tracking, and the matching process is accelerated, thereby improving the efficiency of system initialization.

[0097] refer to Figure 3 , Figure 3 FIG. 4 is a flow chart of a second embodiment of a method for initializing a map system according to the present invention.

[0098] Based on the first embodiment, step S40 of the map system initialization method of this embodiment includes:

[0099] Step S401: Calculate the coordinate distances between feature points in each pair of matching feature points to obtain multiple feature coordinate distances.

[0100] It should be noted that calculating the coordinate distance between the feature points in each pair of matching feature points to obtain multiple feature coordinate distances can be done by obtaining the first-frame feature point coordinates corresponding to the first-frame feature point information in each pair of matching feature points and the non-first-frame feature point coordinates corresponding to the non-first-frame feature point information, and then calculating the coordinate distance based on the first-frame feature point coordinates and the non-first-frame feature point coordinates through the distance formula between two points, thereby obtaining multiple feature coordinate distances.

[0101] In actual use, the number of matching feature point pairs can also be detected. If the number of matching feature point pairs is less than a preset pair threshold, it indicates that the number of matching feature point pairs is insufficient for system initialization. In this case, the process can directly return to step S10 and re-execute the initialization operation. If the number of matching feature point pairs is greater than or equal to the preset pair threshold, step S401 is executed. The preset pair threshold can be pre-set by the administrator of the map system initialization device based on actual conditions.

[0102] Step S402: Obtain an average value of the plurality of feature coordinate distances to obtain a feature movement distance.

[0103] In actual use, obtaining the average value of multiple feature coordinate distances and obtaining the feature movement distance may be performed by calculating the average value of the multiple feature coordinate distances and using the calculated average value as the feature movement distance. The preset movement threshold may be pre-set by the administrator of the map system initialization device.

[0104] Step S403: If the feature movement distance is greater than or equal to a preset movement threshold, then calculating the center point distance based on the multiple pairs of matching feature points.

[0105] It should be noted that the characteristic movement distance can be a value used to represent the difference in acquisition position when acquiring images. A larger characteristic movement distance indicates a larger difference in acquisition position between the first frame image and the non-first frame image. The center point distance can be a value used to represent the movement distance between the image center of the first frame image and the image center of the non-first frame image. A larger center point distance indicates a larger distance between the image center of the first frame image and the image center of the non-first frame image.

[0106] It is understood that if the feature movement distance is greater than or equal to the preset movement threshold, it indicates that the center point distance may meet the preset initialization condition at this time. Therefore, the center point distance can be calculated based on the multiple pairs of matching feature points. If the feature movement distance is less than the preset movement threshold, it indicates that the center point distance cannot meet the preset initialization condition at this time. Therefore, the center point distance can be calculated without re-capturing the non-first frame image and then executing step S10.

[0107] In a specific implementation, in order to accurately calculate the center point distance, the step of calculating the center point distance based on the multiple pairs of matching feature points in this embodiment includes:

[0108] Selecting a preset number of matching feature point pairs from the multiple matching feature point pairs to obtain target matching feature point pairs;

[0109] Determining a first center point based on the target feature point pair, and determining a center operation weight based on a preset weight matrix;

[0110] Determine a second center point according to the first center point and the center operation weight;

[0111] A center point distance is calculated based on the first center point and the second center point.

[0112] It should be noted that the preset number can be pre-set by the administrator of the map system initialization device according to actual needs, for example, the preset number can be set to 4. Selecting a preset number of matching feature point pairs from multiple pairs of matching feature point pairs to obtain target matching feature point pairs can be a random preset number of matching feature point pairs from the multiple pairs of matching feature point pairs, and the selected matching feature point pairs are used as target matching feature point pairs. Determining the first center point based on the target feature point pairs can be extracting the first frame feature point coordinates corresponding to the first frame feature point information in each target feature point pair, thereby obtaining multiple first frame feature point coordinates, finding the center point of the multiple first frame feature point coordinates, and using the found center point as the first center point.

[0113] In actual use, assuming the preset number is 4, there are 4 pairs of target matching feature points. The coordinates of the first center point can be expressed as (width / 2, height / 2), and the preset weight matrix can be expressed as:

[0114]

[0115] Among them, (x1, y1), (x2, y2), (x3, y3), and (x4, y4) are the coordinates of the feature points corresponding to the first frame feature point information of the four pairs of target matching feature points. (x1 * , y1 * )、(x2* , y2 * )、(x3 * , y3 * )、(x4 * ,y4 * ) are the feature point coordinates corresponding to the non-first frame feature points in the four pairs of target matching feature points obtained, and a1-a8 are the center operation weights.

[0116] If the second center point is represented as (x0, y0), then we have

[0117] x0=(a1*width / 2+a2*height / 2+a3) / (a7*width / 2+a8*height / 2)

[0118] y0=(a4*width / 2+a5*height / 2+a6) / (a7*width / 2+a8*height / 2)

[0119] Calculating the center point distance based on the first center point and the second center point may be calculating the coordinate distance between the first center point and the second center point using a distance formula between two points, and then using the obtained coordinate distance as the center point distance.

[0120] Step S404: when the feature movement distance and the center point distance meet a preset initialization condition, initializing the map system according to the multiple pairs of matching feature points.

[0121] It should be noted that in order to ensure a better initialization effect of the SLAM system, it is necessary to ensure that there is a certain difference between the first frame image and the non-first frame image, that is, the difference in the acquisition position when acquiring the first frame image and the non-first frame image needs to reach a certain value, and the distance between the image centers after acquisition also needs to reach a certain value. Therefore, before initializing the map system, you can first check whether the feature movement distance and the center point distance meet the preset initialization conditions. If so, the map system is initialized based on the multiple pairs of matching feature points.

[0122] In actual use, initializing the map system based on multiple pairs of matching feature points can be done by using epipolar constraints to screen out the correct matching feature point pairs from the multiple matching feature point pairs, then calculating the basic matrix F based on the correct matching feature point pairs, decomposing the essential matrix, and obtaining the pose parameters: rotation R and translation t; using the pose parameters, calculating the three-dimensional coordinates of the matching feature point pairs, and calculating the back-projection error. If the error is less than a set threshold, the three-dimensional coordinates are considered correct; counting the number of points with correct three-dimensional coordinates. If the number is greater than the threshold, the initialization is considered successful.

[0123] In a specific implementation, in order to reduce unnecessary operations, reduce performance overhead, and improve the initialization efficiency of the SLAM system, before step S404 in this embodiment, the following steps may also be included:

[0124] If the characteristic movement distance is greater than or equal to a preset movement determination threshold, and / or the center point distance is greater than or equal to a preset center determination threshold, it is determined that the characteristic movement distance and the center point distance meet a preset initialization condition.

[0125] It should be noted that the preset movement determination threshold and the preset center determination threshold can be set in advance by the administrator of the map system initialization device according to actual needs.

[0126] It can be understood that if the feature movement distance is greater than or equal to the preset movement determination threshold, and / or the center point distance is greater than or equal to the preset center determination threshold, it means that there is a sufficient difference between the first frame image and the non-first frame image. At this time, initializing the map system with the obtained multiple pairs of matching feature points can ensure a better initialization effect. Therefore, it can be determined that the feature movement distance and the center point distance meet the preset initialization conditions.

[0127] If the feature movement distance is less than the preset movement determination threshold, and the center point distance is less than the preset center determination threshold, it means that the difference between the first frame image and the non-first frame image is insufficient. At this time, the map system is initialized using the obtained multiple pairs of matching feature points, and the initialization effect will be poor. At this time, it can be determined that the feature movement distance and the center point distance do not meet the preset initialization conditions, and the non-first frame image is re-collected, and then step S10 is executed.

[0128] Furthermore, since there may be errors when judging by only one center point distance, in order to make accurate judgments, in this embodiment, if the feature movement distance is greater than or equal to the preset movement judgment threshold, and / or the center point distance is greater than or equal to the preset center judgment threshold, then before the step of determining whether the feature movement distance and the center point distance meet the preset initialization conditions, the center distance mean can also be calculated, and then the center distance mean is used to assist in the judgment, so as to determine whether the SLAM system can be initialized, wherein the center distance mean can be calculated by the preset center distance formula, and the preset center distance formula is as follows:

[0129]

[0130] Where k is the number of matching feature point pairs, (x i ,y i) Indicates the coordinates of the feature points corresponding to the first frame feature point information in the i-th pair of matching feature points, (x i * ,y i *) represents the coordinates of the feature points corresponding to the feature point information of the first frame in the i-th pair of matching feature points, and (width / 2, height / 2) represents the coordinates of the center point obtained from the coordinates of the feature points corresponding to the feature point information of the first frame in all matching feature point pairs.

[0131] At this time, the preset initialization conditions can be determined to be met only when the center point distance is greater than or equal to the preset center judgment threshold, the center distance mean is greater than or equal to the preset center mean threshold, and / or the feature movement distance is greater than or equal to the preset movement judgment threshold, so as to execute the subsequent steps.

[0132] This embodiment calculates the coordinate distances between feature points in each pair of matching feature points to obtain multiple feature coordinate distances; obtains the average of the multiple feature coordinate distances to obtain the feature movement distance; and calculates the center point distance based on the multiple pairs of matching feature points if the feature movement distance is greater than or equal to a preset movement threshold. When the feature movement distance and center point distance meet preset initialization conditions, the map system is initialized based on the multiple pairs of matching feature points. Because the feature movement distance and center point distance are calculated based on multiple pairs of matching feature points, the effectiveness of map system initialization using the matching feature point pairs can be determined by determining whether the feature movement distance and center point distance meet the preset initialization conditions. Map initialization is performed only when the preset initialization conditions are met, avoiding a large number of ineffective calculations and improving the efficiency of map system initialization.

[0133] In addition, an embodiment of the present invention further provides a storage medium, on which a map system initialization program is stored. When the map system initialization program is executed by a processor, the steps of the map system initialization method described above are implemented.

[0134] Reference Figure 4 , Figure 4 This is a structural block diagram of the first embodiment of the map system initialization device of the present invention.

[0135] like Figure 4 As shown, the map system initialization device proposed in the embodiment of the present invention includes:

[0136] The image acquisition module 10 is used to obtain the first frame feature point set when a non-first frame image is acquired;

[0137] A feature tracking module 20 is configured to perform feature tracking and matching on each first-frame feature point in the first-frame feature point set according to an optical flow method, and determine a region of interest corresponding to each first-frame feature point;

[0138] A feature matching module 30 is configured to extract similar feature points of interest corresponding to the feature points of each first frame from the region of interest, and obtain multiple pairs of matching feature points;

[0139] The system debugging module 40 is configured to initialize the map system according to the plurality of matching feature point pairs.

[0140] This embodiment obtains a first-frame feature point set when a non-first-frame image is captured; performs feature tracking and matching on each first-frame feature point in the first-frame feature point set based on the optical flow method to determine the region of interest corresponding to each first-frame feature point; extracts similar feature points of interest corresponding to each first-frame feature point from the region of interest to obtain multiple pairs of matching feature point pairs; and initializes the map system based on the multiple pairs of matching feature point pairs. Because each first-frame feature point is first tracked and matched when constructing the matching feature point pairs, the region of interest corresponding to each first-frame feature point is determined, and then similar feature points of interest corresponding to the first-frame feature point are extracted from the region of interest to construct the matching feature point pairs, the area to be searched is greatly reduced without affecting feature tracking, and the matching process is accelerated, thereby improving the efficiency of system initialization.

[0141] Furthermore, the feature tracking module 20 is also used to track each feature point in the first frame feature point set based on the optical flow method to obtain a corresponding non-first frame feature point set; traverse the non-first frame feature point set, and use the traversed feature points as the current non-first frame feature points; determine the area of ​​interest corresponding to the current non-first frame feature point based on the feature point coordinates of the current non-first frame feature point and a preset range threshold; at the end of the traversal, obtain the area of ​​interest corresponding to each first frame feature point.

[0142] Furthermore, the feature tracking module 20 is further configured to obtain the tracking status corresponding to each feature point in the non-first frame feature point set; and remove the feature points whose corresponding tracking status is tracking failure from the non-first frame feature point set.

[0143] Furthermore, the feature matching module 30 is also used to traverse the first-frame feature point set, and use the traversed first-frame feature points as the current first-frame feature points; extract feature points from the area of ​​interest corresponding to the current first-frame feature points to obtain multiple interest feature points; calculate the feature matching value between each interest feature point and the current first-frame feature point; use the maximum value of the feature matching values ​​as the target feature matching value; if the target feature matching value is greater than a preset matching threshold, use the interest feature point corresponding to the target feature matching value as the similar interest feature point corresponding to the current non-first-frame feature point; construct matching feature point pairs based on the similar interest feature points and the current first-frame feature points; and obtain multiple pairs of matching feature point pairs at the end of the traversal.

[0144] Furthermore, the system debugging module 40 is also used to calculate the coordinate distance between the feature points in each pair of matching feature points to obtain multiple feature coordinate distances; obtain the average value of the multiple feature coordinate distances to obtain the feature movement distance; if the feature movement distance is greater than or equal to a preset movement threshold, calculate the center point distance based on the multiple pairs of matching feature points; when the feature movement distance and the center point distance meet the preset initialization conditions, initialize the map system based on the multiple pairs of matching feature points.

[0145] Furthermore, the system debugging module 40 is also used to select a preset number of matching feature point pairs from the multiple pairs of matching feature point pairs to obtain target matching feature point pairs; determine the first center point based on the target feature point pair, and determine the center operation weight based on a preset weight matrix; determine the second center point based on the first center point and the center operation weight; and calculate the center point distance based on the first center point and the second center point.

[0146] Furthermore, the system debugging module 40 is also used to determine that the feature movement distance and the center point distance meet the preset initialization conditions if the feature movement distance is greater than or equal to the preset movement judgment threshold, and / or the center point distance is greater than or equal to the preset center judgment threshold.

[0147] It should be understood that the above is only an example and does not constitute any limitation to the technical solution of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any limitation on this.

[0148] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of the embodiment scheme, and no limitation is made here.

[0149] In addition, for technical details not fully described in this embodiment, reference can be made to the map system initialization method provided in any embodiment of the present invention, and will not be repeated here.

[0150] In addition, it should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0151] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0152] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, or of course by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0153] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for initializing a map system, characterized in that: The map system initialization method comprises the following steps: When a non-first frame image is collected, a first frame feature point set is obtained; Performing feature tracking and matching on each first-frame feature point in the first-frame feature point set according to the optical flow method to determine the region of interest corresponding to each first-frame feature point; Extract similar feature points of interest corresponding to the feature points of each first frame from the region of interest to obtain multiple pairs of matching feature points; A map system is initialized according to the multiple pairs of matching feature points.

2. The map system initialization method according to claim 1, characterized in that: The step of performing feature tracking and matching on each first-frame feature point in the first-frame feature point set according to the optical flow method to determine the region of interest corresponding to each first-frame feature point includes: Tracking each feature point in the first frame feature point set according to the optical flow method to obtain a corresponding non-first frame feature point set; Traversing the non-first frame feature point set, and using the traversed feature points as current non-first frame feature points; Determine the region of interest corresponding to the feature point of the current non-first frame according to the feature point coordinates of the feature point of the current non-first frame and a preset range threshold; At the end of the traversal, the region of interest corresponding to the feature points of each first frame is obtained.

3. The map system initialization method according to claim 2, characterized in that: Before the step of traversing the non-first frame feature point set and using the traversed feature points as the current non-first frame feature points, the method further includes: Obtaining the tracking status corresponding to each feature point in the non-first frame feature point set; The corresponding feature points whose tracking status is tracking failure are removed from the non-first frame feature point set.

4. The map system initialization method according to claim 1, wherein: The step of extracting similar interest feature points corresponding to the feature points of each first frame from the region of interest to obtain multiple pairs of matching feature point pairs includes: Traversing the first frame feature point set, and using the traversed first frame feature points as current first frame feature points; Extracting feature points from the region of interest corresponding to the feature points of the current first frame to obtain multiple feature points of interest; Calculating the feature matching value between each feature point of interest and the feature point of the current first frame; Taking the maximum value among the feature matching values ​​as the target feature matching value; If the target feature matching value is greater than a preset matching threshold, the interest feature point corresponding to the target feature matching value is used as a similar interest feature point corresponding to the feature point of the current first frame; Constructing a matching feature point pair based on the similar interest feature points and the current first frame feature points; At the end of the traversal, multiple pairs of matching feature points are obtained.

5. The map system initialization method according to any one of claims 1 to 4, characterized in that: The step of initializing the map system based on the multiple pairs of matching feature points includes: Calculate the coordinate distance between the feature points in each pair of matching feature points to obtain multiple feature coordinate distances; Obtaining an average value of the plurality of feature coordinate distances to obtain a feature movement distance; If the feature movement distance is greater than or equal to a preset movement threshold, calculating the center point distance based on the multiple pairs of matching feature points; When the feature movement distance and the center point distance meet a preset initialization condition, a map system is initialized according to the multiple pairs of matching feature points.

6. The map system initialization method according to claim 5, characterized in that: The step of calculating the center point distance based on the multiple pairs of matching feature points includes: Selecting a preset number of matching feature point pairs from the multiple matching feature point pairs to obtain target matching feature point pairs; Determining a first center point based on the target matching feature point pair, and determining a center operation weight based on a preset weight matrix; Determine a second center point according to the first center point and the center operation weight; A center point distance is calculated based on the first center point and the second center point.

7. The map system initialization method according to claim 5, characterized in that: Before the step of initializing the map system according to the plurality of matching feature point pairs when the feature movement distance and the center point distance meet a preset initialization condition, the method further includes: If the characteristic movement distance is greater than or equal to a preset movement determination threshold, and / or the center point distance is greater than or equal to a preset center determination threshold, it is determined that the characteristic movement distance and the center point distance meet a preset initialization condition.

8. A map system initialization device, characterized in that: The map system initialization device includes the following modules: The image acquisition module is used to obtain the feature point set of the first frame when a non-first frame image is acquired; A feature tracking module is used to perform feature tracking and matching on each first-frame feature point in the first-frame feature point set according to an optical flow method, and determine a region of interest corresponding to each first-frame feature point; A feature matching module is used to extract similar feature points of interest corresponding to the feature points of each first frame from the region of interest to obtain multiple pairs of matching feature points; The system debugging module is used to initialize the map system according to the multiple pairs of matching feature points.

9. A map system initialization device, characterized in that: The map system initialization device includes: a processor, a memory, and a map system initialization program stored in the memory and executable on the processor. When the map system initialization program is executed by the processor, the steps of the map system initialization method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a map system initialization program, which implements the steps of the map system initialization method according to any one of claims 1 to 7 when executed.

Images