A method, device, equipment, storage medium and product for road element tracking

Abstract

The present disclosure provides a road element tracking method, apparatus, device, storage medium and product. By performing feature learning on image feature points in multiple collected images of a real road scene with each other, global feature description information of the image feature points is obtained. The image feature points are matched by means of the global feature description information to obtain a feature point sequence. Through the optical flow trajectory information in the feature point sequence adapted to the element to be tracked, the tracking trajectory information of the road element to be tracked is obtained. In this way, the image feature points can fully integrate the global information of the image, effectively improve the reliability of the description information of the feature points, greatly reduce the noise influence of the surrounding elements of the feature points on the feature points, achieve an accurate and effective feature point matching effect, help improve the accuracy of feature point matching, ensure the effectiveness and accuracy of multi-frame road element tracking, be fast and effective, and have a high road element tracking accuracy.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular, to a method, device, equipment, storage medium and product for tracking road elements. Background Art

[0002] With the development of technology and the continuous progress of computer technology, electronic maps have become an essential application in people's daily lives. With the wide application of electronic maps, they have also become a common means for passenger car drivers to quickly find routes when traveling.

[0003] When a user uses an electronic map, the accuracy and timeliness of the electronic map are particularly important for the user. Especially when the road conditions change, the effective road elements in the electronic map need to be updated in a timely manner to ensure the accuracy of the route found by the user. For the addition and update of road elements in the electronic map, most are to collect data such as road point clouds or road images of the actual road through road information collection devices such as road collection vehicles with lidar, and then extract the road elements through manual extraction or object recognition methods, and then add them to the electronic map. However, the recognition results are limited by factors such as experience, data collection density, and interference between data information, resulting in low accuracy of road element extraction, and thus large deviations when updating the electronic map information, which is not conducive to subsequent use. Summary of the Invention

[0004] The embodiments of the present disclosure at least provide a method, device, equipment, storage medium and product for tracking road elements.

[0005] The embodiments of the present disclosure provide a method for tracking road elements, the method comprising:

[0006] Obtaining a plurality of captured images of a real road scene, and the road elements to be tracked in the plurality of captured images;

[0007] For each captured image, based on the captured image and the captured image adjacent to it in the capture order, determining global feature description information of each image feature point in the captured image;

[0008] Based on the global feature description information of each image feature point in each captured image, performing feature point matching on the image feature points in different captured images to obtain a plurality of feature point sequences, and each image feature point in each feature point sequence is a point in the captured images captured at different capture times corresponding to the same feature point;

[0009] Based on the optical flow trajectory information corresponding to at least one of the feature point sequences matched with the road elements to be tracked, determining the tracking trajectory information of the road elements to be tracked.

[0010] In an alternative embodiment, for each captured image, based on the captured image and the captured image adjacent to the captured image in the capture order, determining the global feature description information of each image feature point in the captured image includes:

[0011] For each captured image, determining a plurality of image feature points in the captured image and the description feature information of each image feature point;

[0012] For the captured image and the captured image adjacent to the captured image in the capture order, based on the description feature information of each image feature point in the two adjacent captured images, performing cross-learning processing on the plurality of image feature points in the two captured images to obtain the global feature description information of each image feature point in the image pair for the two captured images.

[0013] In an alternative embodiment, for each captured image, determining a plurality of image feature points in the captured image and the description feature information of each image feature point includes:

[0014] For each captured image, inputting the captured image into a pre-trained convolutional neural network to extract a plurality of image feature points in the captured image and the description feature information of each image feature point.

[0015] In an alternative embodiment, for the captured image and the captured image adjacent to the captured image in the capture order, based on the description feature information of each image feature point in the two adjacent captured images, performing cross-learning processing on the plurality of image feature points in the two captured images to obtain the global feature description information of each image feature point in the image pair for the two captured images includes:

[0016] For the captured image and the captured image adjacent to the captured image in the capture order, taking the description feature information of each image feature point in the two adjacent captured images as input and inputting it into a pre-trained graph neural network to obtain the global feature description information of each image feature point in the two captured images for the two captured images.

[0017] In an alternative embodiment, based on the global feature description information of each image feature point in each captured image, performing feature point matching on the image feature points in different captured images to obtain a plurality of feature point sequences, where each image feature point in each feature point sequence is a point in the captured images captured at different capture times corresponding to the same feature point, includes:

[0018] For each acquired image, determine the position information of each image feature point in the acquired image;

[0019] Based on the position information of each image feature point and the global feature description information, determine the matching feature vector of each image feature point;

[0020] For the acquired image and the acquired image adjacent to the acquired image in the acquisition order, input the matching feature vectors of the respective image feature points in the two adjacent acquired images into the trained feature point matching neural network to obtain a plurality of feature point matching results of the two acquired images, where the feature point matching result includes two image feature points, and the two image feature points are respectively located in different acquired images of the two acquired images;

[0021] Based on the plurality of feature point matching results corresponding to each pair of adjacent acquired images in the plurality of acquired images, determine a plurality of feature point sequences corresponding to the plurality of acquired images, where the feature point sequence includes a plurality of image feature points, and the plurality of image feature points are points in the acquired images acquired at different acquisition times corresponding to the same feature point.

[0022] In an alternative implementation, the feature point matching neural network is obtained through the following steps:

[0023] Obtain a plurality of sample acquired images of the target road scene and sparse point cloud data;

[0024] Based on the sparse point cloud data, generate a dense point cloud map model of the target road scene;

[0025] For each sample acquired image, determine the depth image corresponding to the sample acquired image, and the position information, pose projection information, and depth information corresponding to the depth image in the dense point cloud map model from the dense point cloud map model;

[0026] Determine a plurality of sample feature points in the sample acquired image, and the sample feature vector of each sample feature point;

[0027] Use the determined sample feature vectors as inputs, and the position information, pose projection information, and depth information corresponding to the depth image as the supervision information of the neural network to train the constructed neural network to obtain the trained feature point matching neural network.

[0028] In an alternative implementation, after determining the tracking trajectory information of the road element to be tracked based on the optical flow trajectory information corresponding to at least one of the feature point sequences matching the road element to be tracked, the method includes:

[0029] Based on the tracking trajectory information of the road element to be tracked, determine the position information and identification information of the road element to be tracked in the real road scene;

[0030] Based on the position information and the identification information, update the map identification of the road element to be tracked in the road map corresponding to the real road scene.

[0031] In an alternative embodiment, the determining the position information of the road element to be tracked in the real road scene based on the tracking trajectory information of the road element to be tracked includes:

[0032] Determine a target image that meets the image screening conditions from the multiple collected images;

[0033] Based on the tracking trajectory information and the road element to be tracked in the target image, determine the position information of the road element to be tracked in the real road scene.

[0034] In an alternative embodiment, the updating the map identification of the road element to be tracked in the road map corresponding to the real road scene based on the position information and the identification information includes:

[0035] Detect whether update information of the road element to be tracked already exists in the road element database of the real road scene, where the update information includes position and identification;

[0036] If not, use the position information and the identification information to update the information of the road element to be tracked in the road element database and the map identification of the road element to be tracked in the road map corresponding to the real road scene.

[0037] The embodiments of the present disclosure further provide a road element tracking device, and the device includes:

[0038] An information acquisition module, configured to acquire multiple collected images of a real road scene, and the road element to be tracked in the multiple collected images;

[0039] A feature determination module, configured to, for each collected image, determine the global feature description information of each image feature point in the collected image based on the collected image and the collected image adjacent to the collected image in the acquisition order;

[0040] A feature point matching module, configured to perform feature point matching on the image feature points in different collected images based on the global feature description information of each image feature point in each collected image, to obtain a plurality of feature point sequences, and each image feature point in each feature point sequence is a point in the collected images acquired at different acquisition times corresponding to the same feature point;

[0041] A trajectory tracking module, configured to determine the tracking trajectory information of the road element to be tracked based on the optical flow trajectory information corresponding to at least one sequence of feature points matching the road element to be tracked.

[0042] In an alternative embodiment, the feature determination module is specifically configured to:

[0043] For each captured image, determine a plurality of image feature points in the captured image and the description feature information of each image feature point;

[0044] For the captured image and the captured image adjacent to the captured image in the capture order, based on the description feature information of each image feature point in the two adjacent captured images, perform cross-learning processing on the plurality of image feature points in the two captured images to obtain the global feature description information of each image feature point in the image pair for the two captured images.

[0045] In an alternative embodiment, when the feature determination module is used to determine, for each captured image, a plurality of image feature points in the captured image and the description feature information of each image feature point, it is specifically configured to:

[0046] For each captured image, input the captured image into a pre-trained convolutional neural network to extract a plurality of image feature points in the captured image and the description feature information of each image feature point.

[0047] In an alternative embodiment, when the feature determination module is used to perform cross-learning processing on the plurality of image feature points in the two captured images based on the description feature information of each image feature point in the two adjacent captured images for the captured image and the captured image adjacent to the captured image in the capture order, to obtain the global feature description information of each image feature point in the image pair for the two captured images, it is specifically configured to:

[0048] For the captured image and the captured image adjacent to the captured image in the capture order, use the description feature information of each image feature point in the two adjacent captured images as input, and input it into a pre-trained graph neural network to obtain the global feature description information of each image feature point in the two captured images for the two captured images.

[0049] In an alternative embodiment, the feature point matching module is specifically configured to:

[0050] For each captured image, determine the position information of each image feature point in the captured image;

[0051] Based on the position information and the global feature description information of each image feature point, determine the matching feature vector of each image feature point;

[0052] For the acquired image and the acquired image adjacent to the acquired image in the acquisition order, input the matching feature vectors of the respective image feature points in the two adjacent acquired images into the trained feature point matching neural network to obtain multiple feature point matching results of the two acquired images, where the feature point matching result includes two image feature points, and the two image feature points are respectively located in different acquired images among the two acquired images;

[0053] Based on the multiple feature point matching results corresponding to each pair of adjacent two acquired images among the multiple acquired images, determine the multiple feature point sequences corresponding to the multiple acquired images, where the feature point sequence includes multiple image feature points, and the multiple image feature points are points in the acquired images collected at different acquisition times corresponding to the same feature point.

[0054] In an optional implementation manner, the device further includes a neural network training module, and the neural network training module is used to obtain the feature point matching neural network through the following steps:

[0055] Obtain multiple sample acquired images of the target road scene and sparse point cloud data;

[0056] Based on the sparse point cloud data, generate a dense point cloud map model of the target road scene;

[0057] For each of the sample acquired images, determine the depth image corresponding to the sample acquired image, and the position information, pose projection information, and depth information corresponding to the depth image in the dense point cloud map model from the dense point cloud map model;

[0058] Determine multiple sample feature points in the sample acquired image, and the sample feature vector of each sample feature point;

[0059] Use the determined sample feature vectors as the input, and the position information, pose projection information, and depth information corresponding to the depth image as the supervision information of the neural network to train the constructed neural network to obtain the trained feature point matching neural network.

[0060] In an optional implementation manner, the device further includes a road element update module, and the road element update module is specifically used for:

[0061] Based on the tracking trajectory information of the road element to be tracked, determine the position information and identification information of the road element to be tracked in the real road scene;

[0062] Based on the position information and the identification information, update the map identification of the road element to be tracked in the road map corresponding to the real road scene.

[0063] In an optional implementation manner, when the road element update module is used to determine the position information of the road element to be tracked in the real road scene based on the tracking trajectory information of the road element to be tracked, it is specifically used for:

[0064] Determine a target image that meets the image screening conditions from the multiple collected images;

[0065] Based on the tracking trajectory information and the road element to be tracked in the target image, determine the position information of the road element to be tracked in the real road scene.

[0066] In an optional implementation manner, when the road element update module is used to update the map identification of the road element to be tracked in the road map corresponding to the real road scene based on the position information and the identification information, it is specifically used for:

[0067] Detect whether update information of the road element to be tracked already exists in the road element database of the real road scene, where the update information includes position and identification;

[0068] If not, use the position information and the identification information to update the information of the road element to be tracked in the road element database and the map identification of the road element to be tracked in the road map corresponding to the real road scene.

[0069] An embodiment of the present disclosure further provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the above-mentioned road element tracking method are executed.

[0070] An embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of the above-mentioned road element tracking method are executed.

[0071] An embodiment of the present disclosure further provides a computer program product, including computer instructions, and when the computer instructions are executed by a processor, the steps of the above-mentioned road element tracking method are executed.

[0072] The road element tracking method, device, equipment, storage medium and product provided by the embodiments of the present disclosure obtain a plurality of captured images of a real road scene and the road elements to be tracked in the plurality of captured images; for each captured image, based on the captured image and the captured image adjacent to the captured image in the capture order, determine the global feature description information of each image feature point in the captured image; based on the global feature description information of each image feature point in each captured image, perform feature point matching on the image feature points in different captured images to obtain a plurality of feature point sequences, and each image feature point in each feature point sequence is a point in the captured images captured at different capture times corresponding to the same feature point; based on the optical flow trajectory information corresponding to at least one of the feature point sequences matched with the road element to be tracked, determine the tracking trajectory information of the road element to be tracked.

[0073] In this way, by performing feature learning on the image feature points in a plurality of captured images of a real road scene with each other to obtain the global feature description information of the image feature points, the image feature points can fully integrate the global information of the image. Furthermore, by using the global feature description information to match the image feature points to obtain the feature point sequences, the reliability of the description information of the feature points can be effectively improved, the noise influence of the elements around the feature points can be greatly reduced, and an accurate and effective feature point matching effect can be achieved, which helps to improve the accuracy of feature point matching. Further, it can help to accurately extract the optical flow tracking trajectories between multiple captured images, and thus, by combining the optical flow trajectory information in the feature point sequences adapted to the elements to be tracked in the captured images, the tracking trajectory information of the road elements to be tracked can be obtained, which can effectively realize the tracking of the road elements between multiple frames, quickly and effectively, and the accuracy of road element tracking is high.

[0074] Furthermore, through the optical flow tracking trajectories of road elements between multiple captured images, it can help to realize the recognition and tracking of road elements. Furthermore, by means of the optical flow tracking trajectories, the positioning of road elements can be accurately realized, the position information of road elements can be accurately obtained, and the positioning is accurate and effective, thereby helping to improve the accuracy of the electronic map.

[0075] To make the above objects, features, and advantages of the present disclosure more obvious and understandable, the following specific preferred embodiments are given and described in detail in conjunction with the accompanying drawings. Description of the Drawings

[0076] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the accompanying drawings required for the embodiments. The accompanying drawings here are incorporated into the specification and form a part of this specification. These accompanying drawings show embodiments that conform to the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following accompanying drawings only show some embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related accompanying drawings can also be obtained based on these accompanying drawings.

[0077] Figure 1 shows a flowchart of a road element tracking method provided by an embodiment of the present disclosure;

[0078] Figure 2 shows a distribution diagram of feature points in a road element tracking method provided by an embodiment of the present disclosure;

[0079] Figure 3 shows a matching result diagram of feature points in a road element tracking method provided by an embodiment of the present disclosure;

[0080] Figure 4 shows a flowchart of another road element tracking method provided by an embodiment of the present disclosure;

[0081] Figure 5 shows one of the schematic diagrams of a road element tracking device provided by an embodiment of the present disclosure;

[0082] Figure 6 shows another schematic diagram of a road element tracking device provided by an embodiment of the present disclosure;

[0083] Figure 7 shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure. Detailed Embodiments

[0084] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present disclosure with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only some embodiments of the present disclosure, rather than all embodiments. Usually, the components of the embodiments of the present disclosure described and shown in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the present disclosure to be protected, but only represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present disclosure.

[0085] It should be noted that like reference numerals and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

[0086] As used herein, the term "and / or" merely describes an association relationship and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. Additionally, the term "at least one" as used herein means any one of a plurality or any combination of at least two of a plurality. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0087] It has been found through research that for the addition and update of road elements in an electronic map, etc., most are to collect data such as road point clouds or road images of the actual road through road information collection devices such as road collection vehicles with lidar, and then extract road elements through manual extraction or object recognition, etc., and then add them to the electronic map. Currently, for the extraction of road elements, a commonly used method is to screen pictures through the gps azimuth difference between pictures, so as to extract road elements in the pictures. However, this method usually cannot make full use of image information and is very likely to miss many valid elements. Another commonly used method is to track road elements in the target picture through multi-object tracking technology. However, when implementing this method, due to the small size of road elements in the actual road scene, it is very difficult to track the running trajectories of the same road element contained in multiple images, and most tracking algorithms have limitations on the acquisition density of pictures. Therefore, when encountering a large-scale road scene, a large cost requirement is needed for calculation through the tracking algorithm.

[0088] Based on the above research, the present disclosure provides a road element tracking method, which can effectively improve the reliability of the description information of the extracted feature points, greatly reduce the noise influence of the elements around the feature points on the feature points, and thus can achieve an accurate and effective feature point matching effect during the feature point matching process, helping to improve the accuracy of feature point matching. Further, it can help accurately extract the optical flow tracking trajectories between multiple collected images, and thus combine the optical flow trajectory information in the feature point sequence adapted to the element to be tracked in the collected image to obtain the tracking trajectory information of the road element to be tracked, which can effectively achieve the tracking of road elements between multiple frames, quickly and effectively, and with high road element tracking accuracy.

[0089] Please refer to Figure 1 , Figure 1 which is a flowchart of a road element tracking method provided by an embodiment of the present disclosure. As Figure 1 shown, the road element tracking method provided by an embodiment of the present disclosure includes:

[0090] S101: Obtain multiple captured images of a real - road scene and the road elements to be tracked in the multiple captured images.

[0091] In this step, when it is necessary to update the electronic map to add or update the road elements in the electronic map, multiple captured images of the real - road scene can be obtained. After obtaining the multiple captured images, content recognition can be performed on the multiple captured images. For example, through image content recognition methods, or by means of the neural network of artificial intelligence to recognize the captured images, or through remote - sensing image road - information extraction methods, to convert the appearance forms of roads and backgrounds in the captured images into templates, so as to recognize the content contained in the images. Or through image digital - processing and other means, extract the problems of the road elements appearing in the images and convert the extracted problems into mathematical expressions to determine the content contained in the images, so that all the road elements included in the multiple captured images can be recognized, and then the road elements to be tracked that need to be tracked can be determined from all the road elements.

[0092] Among them, the road elements can be all the elements with obvious signs in the real - road scene that appear in the captured images, such as big trees, vehicles, electronic eyes, traffic - rule signs, etc. Among them, the big trees, vehicles, etc. can be marked as non - essential road elements, which do not play a key role in the use of the current road conditions in the electronic map. The road elements to be tracked can be the elements that can reflect the current road conditions in the real - road scene corresponding to the captured images except for the non - essential road elements, such as electronic eyes, traffic - rule signs, speed - limit signs, etc.

[0093] Among them, the multiple captured images of the real - road scene can be taken by the in - vehicle camera of the user when the user is driving, or can be automatically taken by multiple cameras pre - set in the real - road scene, or can be collected by a road - information collection vehicle equipped with collection devices such as lidar and cameras when it is necessary to update the electronic map.

[0094] Among them, each image in the multiple captured images may contain the road elements to be tracked, or only some of the multiple captured images contain the road elements to be tracked.

[0095] S102: For each captured image, based on the captured image and the captured image adjacent to it in the capture order, determine the global feature description information of each image feature point in the captured image.

[0096] In this step, after obtaining multiple captured images, multiple image feature points in each captured image can be extracted. In order to enable each image feature point to have an accurate information representation, methods such as using the neural network of artificial intelligence can be adopted to enable the image feature points to learn the global image information of the captured image, so as to obtain the global feature description information of the image feature points, that is, the global descriptor of the image feature points.

[0097] Since the intervals between the multiple captured images collected are generally not very long, although there will be some differences between the captured images, they are generally not very different as a whole. In order to enable the image feature points to fully learn the global image information of the captured image where they are located, for each captured image and the multiple image feature points identified from it, the captured image adjacent to the captured image in the acquisition order can be determined first. For these two adjacent captured images, the corresponding image feature points can be made to learn the global image information of the two captured images to obtain the global feature description information of the image feature points.

[0098] The above is described by taking the example of image feature points learning the information of adjacent captured images, but it is not limited to this. In other embodiments, on the premise of meeting the data accuracy requirements, it is also possible to make the image feature points only learn the global image information of the captured image where they are located. This can reduce the data processing volume, reduce the data processing time, and speed up the data processing progress.

[0099] Among them, the feature points can be obtained by collecting multiple pictures of the same object or scene from different angles. If the same places can be recognized as the same points or blocks, they are points analyzed by algorithms and contain rich local information. They often appear in places such as corners and sharp texture changes in the image. It should be noted that the feature points may not only be a single point, but also include a series of local information. Even in many cases, it itself is a small area with an area.

[0100] Among them, in implementation, only valid feature points can be used. For example, for a captured image, only the road elements and the feature points around them identified from the image can be extracted, and the feature points related to the sky background of the real road scene that appear in the captured image can be not used. It is also possible to extract all the feature points in the captured image. For example, the points corresponding to each pixel in the captured image can be used as feature points, so as to regard all the content in the captured image as image feature points. This can improve the comprehensiveness and accuracy of the information related to the image feature points, and help improve the integrity and accuracy of the global feature description information of the image feature points.

[0101] Exemplarily, such as Figure 2As shown, for the acquired captured images, multiple image feature points can be identified therefrom. The image feature points can be the feature points corresponding to the physical objects in the captured images and the content around the physical objects ( Figure 2 Taking the example that each physical object has three image feature points in Figure 2 , for the sake of convenience of explanation only. In practical applications, a large number of image feature points can be extracted from each physical object. For example, the number of image feature points is the same as the number of pixels corresponding to the physical object), such as Figure 2 the big tree, the car, and the electronic eye in Figure 2 can all identify their corresponding image feature points and the image feature points around them. For background content such as roads and the sky, their corresponding image feature points may not be identified, thereby reducing the amount of data processing. On the contrary, in order to improve data reliability as much as possible, each pixel point in the captured image can also be used as an image feature point. Furthermore, when each image feature point learns the global image information of the two captured images, only the feature points on the big tree, the car, and the electronic eye shown in the image can be used, and the feature points in the blank area of the image and on the zebra crossing on the road can be not used.

[0102] Among them, the feature description information is used to describe an image or an image region containing useful information.

[0103] S103: Based on the global feature description information of each image feature point in each captured image, perform feature point matching on the image feature points in different captured images to obtain multiple feature point sequences. Each image feature point in each feature point sequence is a point in the captured images collected at different capture times corresponding to the same feature point.

[0104] In this step, after obtaining the global feature description information of each image feature point in each captured image, the global feature description information of each image feature point can be used to perform feature point matching on the multiple image feature points in the multiple captured images, so as to screen out multiple image feature points corresponding to the same feature point in different captured images, thereby obtaining multiple feature point sequences. To perform feature point matching, it can be through an artificial neural network or through a corresponding point pairing algorithm. For example, the similarity of two image feature points belonging to two captured images can be calculated to see whether the two image feature points correspond to the same feature point. It can also be through an image registration algorithm based on template matching, etc., to calculate the sum of the squares of the pixel differences corresponding to two image feature points belonging to two captured images respectively, so as to obtain the feature point sequence.

[0105] Exemplarily, such as Figure 3As shown, taking the two captured images as an example, which are two adjacent images captured continuously at different times. In the two captured images, there will be certain differences in the angles and positions of the road elements that appear in their respective images. Simply relying on positions and the like is not conducive to the matching of image feature points. Therefore, the global feature description information of the image feature points can be used for matching. Figure 3 Taking the two captured images shown in Figure 3 as an example, for the image feature point 310 and the image feature point 320 that are matched and belong to the same feature point sequence, both are actually the feature points of the big tree in the upper left. That is to say, the two correspond to the same feature point, and they are just two feature points in two captured images at different capture times.

[0106] S104: Based on the optical flow trajectory information corresponding to at least one of the feature point sequences that match the road element to be tracked, determine the tracking trajectory information of the road element to be tracked.

[0107] In this step, after obtaining multiple feature point sequences, at least one feature point sequence that matches the road element to be tracked can be determined first. Specifically, when identifying the road element to be tracked from the captured images, the area or range where the element to be tracked is located can be marked by means of a bounding box or the like. Then, at least one image feature point that falls within this area or range can be matched through the area or range where the bounding box is located. Furthermore, the feature point sequence where the determined image feature point is located can be used as the feature point sequence that matches the road element to be tracked. Then, for at least one feature point sequence that matches the road element to be tracked, the optical flow trajectory of each feature point sequence can be determined, and finally the tracking trajectory information of the road element to be tracked can be determined.

[0108] Specifically, by tracking each image feature point in at least one feature point sequence that matches the road element to be tracked, the position of each feature point can be determined. Then, based on the position of each feature point, the instantaneous displacement of the feature point in the captured image, that is, its corresponding optical flow, can be obtained. Furthermore, by combining the optical flows of the image feature points in the feature point sequence, the optical flow trajectory of the feature point sequence can be obtained, thereby determining the optical flow trajectory information of the feature point sequence. Finally, based on the optical flow trajectory information of each feature point sequence that matches the road element to be tracked, the tracking trajectory information of the road element to be tracked can be determined. For example, the tracking trajectory information of the road element to be tracked can be obtained by calculating the average optical flow trajectory, etc.

[0109] The road element tracking method provided by the embodiments of the present disclosure performs feature learning on image feature points in multiple captured images of a real road scene with each other, and obtains global feature description information of the image feature points, which can enable the image feature points to fully integrate the global information of the image. Furthermore, by using the global feature description information to match the image feature points, a feature point sequence can be obtained, which can effectively improve the reliability of the description information of the feature points, greatly reduce the noise influence of the elements around the feature points on the feature points, achieve an accurate and effective feature point matching effect, help improve the accuracy of feature point matching, and further help accurately extract the optical flow tracking trajectory between multiple captured images. Therefore, by combining the optical flow trajectory information in the feature point sequence adapted to the element to be tracked in the captured image, the tracking trajectory information of the road element to be tracked can be obtained, which can effectively implement the tracking of the road element between multiple frames, quickly and effectively, and the accuracy of road element tracking is high.

[0110] Please also refer to Figure 4 , Figure 4 which is a flowchart of another road element tracking method provided by the embodiments of the present disclosure. As Figure 4 shown, the road element tracking method provided by the embodiments of the present disclosure includes:

[0111] S401: Obtain multiple captured images of a real road scene and the road element to be tracked in the multiple captured images.

[0112] S402: For each captured image, based on the captured image and the captured image adjacent to the captured image in the capture order, determine the global feature description information of each image feature point in the captured image.

[0113] S403: Based on the global feature description information of each image feature point in each captured image, perform feature point matching on the image feature points in different captured images to obtain multiple feature point sequences, and each image feature point in each feature point sequence is a point in the captured images captured at different capture times corresponding to the same feature point.

[0114] S404: Based on the optical flow trajectory information corresponding to at least one of the feature point sequences matched with the road element to be tracked, determine the tracking trajectory information of the road element to be tracked.

[0115] S405: Based on the tracking trajectory information of the road element to be tracked, determine the position information and identification information of the road element to be tracked in the real road scene.

[0116] In this step, after determining the tracking trajectory information of the road element to be tracked, the position information and identification information of the road element to be tracked in the real road scene can be determined according to the tracking trajectory information, so as to update the map identification of the road element to be tracked through the position information and identification information of the road element to be tracked subsequently.

[0117] Specifically, in some possible implementation manners, for determining the position information, it may be to determine a target image that meets the screening conditions from the multiple collected images. The screening conditions may be, for example, the image with the highest clarity and capable of completely seeing the road element to be tracked among the multiple collected images, or the image at the middle position among the multiple collected images, etc. After determining the target image, then based on the tracking trajectory information and the road element to be tracked in the target image, the position information of the road element to be tracked in the real road scene can be determined. Here, after obtaining the target image, the position of the road element to be tracked in the target image can be obtained from the target image in the tracking trajectory information corresponding to the road element to be tracked, so that through this position and the tracking trajectory information, it can be known that the road element to be tracked stays in the trajectory segment in the optical flow trajectory indicated by the tracking trajectory information, and thus the position information of the road element to be tracked in the real road scene can be determined.

[0118] Among them, the position information may be the specific position of the road element in the real road scene. In the real road scene, each road element corresponds to a specific position.

[0119] Among them, the identification information may include information such as the specific shape, color, and number of the road element in the real road scene.

[0120] S406: Update the map identification of the road element to be tracked in the road map corresponding to the real road scene based on the position information and the identification information.

[0121] In this step, after determining the position information and identification information of the road element to be tracked, the map identification of the road element to be tracked can be updated in the road map corresponding to the real road scene according to the position information and identification information.

[0122] Specifically, for updating the map identifier, based on the position information and identifier information of the to-be-tracked road element, it is detected whether the update information of the to-be-tracked road element already exists in the road element database of the real road scene, where the update information includes the position and the identifier; if not, the position information and the identifier information are used to update the information of the to-be-tracked road element in the road element database and the map identifier of the to-be-tracked road element in the road map corresponding to the real road scene.

[0123] In this way, duplicate information in the road element database can be avoided, and the data volume in the road element database can be reduced.

[0124] Among them, if the update information of the to-be-tracked road element already exists in the database of the real scene road elements, the position information and identifier information of the to-be-tracked element in the database can be directly used as the map identifier of the to-be-tracked element.

[0125] Among them, the descriptions of steps S401 to S404 can refer to the descriptions of steps S101 to S104, and the same technical effects can be achieved and the same technical problems can be solved, which will not be elaborated here.

[0126] Next, the content of each of the above steps will be further described in combination with specific embodiments.

[0127] Optionally, in some possible implementation manners, S402 includes:

[0128] For each captured image, a plurality of image feature points in the captured image and the description feature information of each image feature point are determined;

[0129] For the captured image and the captured image adjacent to the captured image in the capture order, based on the description feature information of each image feature point in the two adjacent captured images, cross-learning processing is performed on the plurality of image feature points in the two captured images to obtain the global feature description information of each image feature point in the image pair for the two captured images.

[0130] In this step, after obtaining a plurality of captured images, a plurality of image feature points in each captured image can be extracted first, and then the description feature information corresponding to each image feature point is calculated. Furthermore, for every two adjacent captured images in the capture order, according to the description feature information corresponding to each image feature point, each image feature point in the two captured images learns the image information of the two captured images to obtain the global feature description information of each image feature point.

[0131] Optionally, in some possible embodiments, for determining the descriptive feature information of the image feature points, it may be automatically extracted by means of a neural network. Specifically, for each acquired image, the acquired image may be input into a pre-trained convolutional neural network to extract multiple image feature points in the acquired image and the descriptive feature information of each image feature point.

[0132] Here, a pre-trained convolutional neural network, such as a neural network like CNN, may be obtained in advance according to training samples. By inputting the acquired image into the pre-trained convolutional neural network, multiple feature points in the acquired image are extracted, and the descriptive feature information corresponding to each feature point, that is, the descriptor of the image feature point, is obtained.

[0133] Optionally, in some possible embodiments, for determining the global feature description information of the image feature points, it may be through the following steps:

[0134] For the acquired image and the acquired image adjacent to it in the acquisition order, the descriptive feature information of each image feature point in the two adjacent acquired images is used as input and input into a pre-trained graph neural network to obtain the global feature description information of each image feature point in the two acquired images for the two acquired images.

[0135] Here, a pre-trained graph neural network, such as a graph neural network like GNN, may be obtained in advance. When constructing the graph neural network, a multi-layer network may be constructed. Multiple neuron nodes are set in each layer of the network, and each neuron node corresponds to an image feature point. Edges of the network are established between the neuron nodes and corresponding parameters are set. Thus, after obtaining the descriptive feature information of each image feature point, the descriptive feature information of each image feature point may be used as input and input into the graph neural network. After repeated learning in the graph neural network, the global feature description information of the image feature points is obtained.

[0136] Optionally, in some possible embodiments, S403 includes:

[0137] For each acquired image, determine the position information of each image feature point in the acquired image;

[0138] Based on the position information and the global feature description information of each image feature point, determine the matching feature vector of each image feature point;

[0139] For the acquired image and the acquired image adjacent to the acquired image in the acquisition order, input the matching feature vectors of each image feature point in the two adjacent acquired images into the trained feature point matching neural network to obtain multiple feature point matching results of the two acquired images. The feature point matching result includes two image feature points, and the two image feature points are respectively located in different acquired images among the two acquired images;

[0140] Based on the multiple feature point matching results corresponding to each pair of adjacent two acquired images among the multiple acquired images, determine the multiple feature point sequences corresponding to the multiple acquired images. The feature point sequence includes multiple image feature points, and the multiple image feature points are points in the acquired images acquired at different acquisition times corresponding to the same feature point.

[0141] In this step, for a single acquired image, the position information of each image feature point in the acquired image can be obtained by means of an image-related positioning algorithm or the like. Or when each pixel is used as the corresponding image feature point, the pixel position can be directly used as the position of the image feature point. Then, based on the position information and the global feature description information of each image feature point, the matching feature vector of each image feature point is determined. For example, by encoding the position information and the global feature description information, the matching feature vector, that is, the matching descriptor, is obtained. Then, with the help of the trained feature point matching neural network, the matching feature vectors of each image feature point in the two adjacent acquired images are used as inputs, and through the output of the feature point matching neural network, multiple feature point matching results of the two acquired images are obtained. The feature point matching result includes two image feature points, and the two image feature points are respectively located in different acquired images among the two acquired images. Then, through the mapping relationship between the two image feature points that match each other, multiple image feature points that match each other directly or indirectly can be grouped into a feature point sequence.

[0142] Among them, the feature point matching neural network can be the neural network corresponding to the Superglue algorithm. During the matching process, it can tentatively screen and match key points by browsing the two acquired images according to the position information of each feature point, and perform back-and-forth checks. Among them, if it is not a matching feature point, check whether there is a better matching point around until a matching point is found or there is no match, so as to enhance the feature matching performance of the feature matching vector. Subsequently, the matching degree score matrix is obtained by calculating the inner product of the feature matching vectors, and then the optimal feature assignment matrix is calculated through the algorithm to obtain the matching result of the feature points. Furthermore, multiple feature point sequences are obtained according to the matching results of each adjacent multiple feature points.

[0143] To obtain a convolutional neural network model and a graph neural network model that better conform to real road scenarios, the constructed neural network can be supervised and trained with data collected from real scenarios to obtain a trained feature point matching neural network. Specifically, the feature point matching neural network is obtained through the following steps:

[0144] Obtain multiple sample acquisition images and sparse point cloud data of the target road scenario;

[0145] Based on the sparse point cloud data, generate a dense point cloud map model of the target road scenario;

[0146] For each sample acquisition image, determine the depth image corresponding to the sample acquisition image, as well as the position information, pose projection information, and depth information corresponding to the depth image in the dense point cloud map model from the dense point cloud map model;

[0147] Determine multiple sample feature points in the sample acquisition image, and the sample feature vector of each sample feature point;

[0148] Use the determined sample feature vectors as inputs, and the position information, pose projection information, and depth information corresponding to the depth image as the supervision information of the neural network to train the constructed neural network to obtain a trained feature point matching neural network.

[0149] The road element tracking method provided by the embodiments of the present disclosure can effectively improve the reliability of the description information of feature points by matching image feature points with the help of global feature description information, greatly reduce the noise influence of elements around the feature points on the feature points, achieve an accurate and effective feature point matching effect, help improve the accuracy of feature point matching, and further, according to the tracking trajectory information of the road element to be tracked, can effectively realize the tracking of road elements between multiple frames, quickly and effectively, with high accuracy of road element tracking. Then, through the optical flow tracking trajectories of road elements between multiple acquisition images, it can help realize the recognition and tracking of road elements, and further, with the help of the optical flow tracking trajectories, the positioning of road elements can be accurately realized, and the position information of road elements can be accurately obtained, with accurate and effective positioning. Thus, based on the position information and the identification information, update the map identification of the road element to be tracked in the road map corresponding to the real road scenario, and finally help improve the accuracy of the electronic map.

[0150] Those skilled in the art can understand that in the above method of the specific implementation manner, the writing order of each step does not mean a strict execution order that constitutes any limitation to the implementation process, and the specific execution order of each step should be determined according to its function and possible internal logic.

[0151] Based on the same inventive concept, embodiments of the present disclosure also provide a road element tracking device corresponding to the road element tracking method. Since the principle of solving problems by the device in the embodiments of the present disclosure is similar to that of the above-mentioned road element tracking method in the embodiments of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be elaborated.

[0152] Please refer to Figures 5 to 6 , Figure 5 which is one of the schematic diagrams of a road element tracking device provided by an embodiment of the present disclosure. Figure 6 which is the second schematic diagram of a road element tracking device provided by an embodiment of the present disclosure. As Figure 5 shown in

[0153] an information acquisition module 510, configured to acquire a plurality of captured images of a real road scene, and the road elements to be tracked in the plurality of captured images.

[0154] a feature determination module 520, configured to, for each captured image, determine global feature description information of each image feature point in the captured image based on the captured image and the captured image adjacent to the captured image in the acquisition order.

[0155] a feature point matching module 530, configured to perform feature point matching on the image feature points in different captured images based on the global feature description information of each image feature point in each captured image, to obtain a plurality of feature point sequences, and each image feature point in each of the feature point sequences is a point in the captured images captured at different acquisition times corresponding to the same feature point.

[0156] a trajectory tracking module 540, configured to determine tracking trajectory information of the road element to be tracked based on optical flow trajectory information corresponding to at least one of the feature point sequences matched with the road element to be tracked.

[0157] In an optional implementation manner, the feature determination module 520 is specifically configured to:

[0158] for each captured image, determine a plurality of image feature points in the captured image and description feature information of each image feature point;

[0159] for the captured image and the captured image adjacent to the captured image in the acquisition order, perform cross-learning processing on the plurality of image feature points in the two captured images based on the description feature information of each image feature point in the two adjacent captured images, to obtain global feature description information of each image feature point in the image pair for the two captured images.

[0160] In an alternative embodiment, when the feature determination module 520 is used to determine multiple image feature points in the acquired image and the descriptive feature information of each image feature point for each acquired image, it is specifically configured to:

[0161] For each acquired image, input the acquired image into a pre-trained convolutional neural network to extract multiple image feature points in the acquired image and the descriptive feature information of each image feature point.

[0162] In an alternative embodiment, when the feature determination module 520 is used to perform cross-learning processing on multiple image feature points in the two acquired images based on the descriptive feature information of each image feature point in the acquired image and the acquired image adjacent to the acquired image in the acquisition order, to obtain the global feature description information of each image feature point in the image pair for the two acquired images, it is specifically configured to:

[0163] For the acquired image and the acquired image adjacent to the acquired image in the acquisition order, use the descriptive feature information of each image feature point in the two adjacent acquired images as input, and input it into a pre-trained graph neural network to obtain the global feature description information of each image feature point in the two acquired images for the two acquired images.

[0164] In an alternative embodiment, the feature point matching module 530 is specifically configured to:

[0165] For each acquired image, determine the position information of each image feature point in the acquired image;

[0166] Based on the position information and global feature description information of each image feature point, determine the matching feature vector of each image feature point;

[0167] For the acquired image and the acquired image adjacent to the acquired image in the acquisition order, input the matching feature vectors of each image feature point in the two adjacent acquired images into a trained feature point matching neural network to obtain multiple feature point matching results of the two acquired images, where the feature point matching result includes two image feature points, and the two image feature points are located in different acquired images of the two acquired images respectively;

[0168] Based on the multiple feature point matching results corresponding to each pair of adjacent acquired images in the multiple acquired images, determine the multiple feature point sequences corresponding to the multiple acquired images, where the feature point sequence includes multiple image feature points, and the multiple image feature points are points in the acquired images acquired at different acquisition times corresponding to the same feature point.

[0169] In an alternative embodiment, as Figure 6 shown in, the device further includes a neural network training module 550, and the neural network training module 550 is used to obtain the feature point matching neural network through the following steps:

[0170] Obtain a plurality of sample acquisition images and sparse point cloud data of the target road scene;

[0171] Based on the sparse point cloud data, generate a dense point cloud map model of the target road scene;

[0172] For each of the sample acquisition images, determine the depth image corresponding to the sample acquisition image, and the position information, pose projection information, and depth information corresponding to the depth image in the dense point cloud map model from the dense point cloud map model;

[0173] Determine a plurality of sample feature points in the sample acquisition image, and the sample feature vector of each sample feature point;

[0174] Use the determined sample feature vectors as inputs, and the position information, pose projection information, and depth information corresponding to the depth image as the supervision information of the neural network to train the constructed neural network to obtain the trained feature point matching neural network.

[0175] In an alternative embodiment, as Figure 6 shown in, the device further includes a road element update module 560, and the road element update module 560 is specifically used for:

[0176] Based on the tracking trajectory information of the road element to be tracked, determine the position information and identification information of the road element to be tracked in the real road scene;

[0177] Based on the position information and the identification information, update the map identification of the road element to be tracked in the road map corresponding to the real road scene.

[0178] In an alternative embodiment, when the road element update module 560 is used to determine the position information of the road element to be tracked in the real road scene based on the tracking trajectory of the road element to be tracked, it is specifically used for:

[0179] Determine a target image that meets the image screening conditions from the plurality of acquisition images;

[0180] Based on the tracking trajectory information and the road element to be tracked in the target image, determine the position information of the road element to be tracked in the real road scene.

[0181] In an alternative embodiment, when the road element update module 560 is used to update the map identifier of the road element to be tracked in the road map corresponding to the real road scene based on the position information and the identification information, it is specifically configured to:

[0182] Detect whether there is already update information of the road element to be tracked in the road element database of the real road scene, where the update information includes position and identification;

[0183] If not, use the position information and the identification information to update the information of the road element to be tracked in the road element database and the map identifier of the road element to be tracked in the road map corresponding to the real road scene.

[0184] The road element tracking device provided by the embodiments of the present disclosure can obtain the global feature description information of the image feature points by performing feature learning on the image feature points in multiple captured images of the real road scene with each other, so that the image feature points can fully integrate the global information of the image. Furthermore, by using the global feature description information to match the image feature points to obtain a feature point sequence, the reliability of the description information of the feature points can be effectively improved, the noise influence of the surrounding elements on the feature points can be greatly reduced, and an accurate and effective feature point matching effect can be achieved, which helps to improve the accuracy of feature point matching. Further, it can help to accurately extract the optical flow tracking trajectory between multiple captured images, and then combine the optical flow trajectory information in the feature point sequence adapted to the element to be tracked in the captured images to obtain the tracking trajectory information of the road element to be tracked. Then, through the optical flow tracking trajectory of the road elements between multiple captured images, it can help to realize the identification and tracking of the road elements, and further, the positioning of the road elements can be accurately realized by means of the optical flow tracking trajectory, and the position information of the road elements can be accurately obtained, and the positioning is accurate and effective. Thus, based on the position information and the identification information, the map identifier of the road element to be tracked is updated in the road map corresponding to the real road scene, and finally, it helps to improve the accuracy of the electronic map.

[0185] For the processing flow of each module in the device and the interaction flow between modules, reference can be made to the relevant descriptions in the above method embodiments, which will not be elaborated here.

[0186] Corresponding to Figure 1 and Figure 4 in the road element tracking method, the embodiments of the present disclosure further provide an electronic device 700, as Figure 7 shown, which is a schematic structural diagram of the electronic device 700 provided by the embodiments of the present disclosure, including:

[0187] A processor 710, a memory 720, and a bus 730; the memory 720 is used to store execution instructions, including an internal memory 721 and an external memory 722; here, the internal memory 721 is also called the main memory, which is used to temporarily store the operation data in the processor 710 and the data exchanged with the external memory 722 such as a hard disk. The processor 710 exchanges data with the external memory 722 through the internal memory 721. When the electronic device 700 runs, the processor 710 communicates with the memory 720 through the bus 730, so that the processor 710 executes the above Figure 1 and Figure 4 steps of the road element tracking method in the method embodiments shown.

[0188] Embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the road element tracking method in the above method embodiments. Wherein, the storage medium can be a volatile or non-volatile computer-readable storage medium.

[0189] Embodiments of the present disclosure also provide a computer program product, which carries program code. The instructions included in the program code can be used to execute the steps of the road element tracking method in the above method embodiments. For details, please refer to the above method embodiments and will not be elaborated here.

[0190] Wherein, the above computer program product can be specifically implemented in a manner of hardware, software, or a combination thereof. In an optional embodiment, the computer program product is specifically embodied as a computer storage medium. In another optional embodiment, the computer program product is specifically embodied as a software product, such as a Software Development Kit (SDK), etc.

[0191] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated here. In several embodiments provided by the present disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some communication interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0192] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0193] In addition, in each embodiment of the present disclosure, each functional unit may be integrated in a processing unit, may exist separately as individual physical units, or two or more units may be integrated in one unit.

[0194] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present disclosure, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable an electronic device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present disclosure. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0195] Finally, it should be noted that: the above-described embodiments are only specific implementation manners of the present disclosure, used to illustrate the technical solutions of the present disclosure, and are not intended to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present disclosure can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be covered by the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A method for tracking road elements, characterized in that, The method includes: obtaining a plurality of captured images of a real road scene, and road elements to be tracked in the plurality of captured images; for each captured image, determining global feature description information of each image feature point in the captured image based on the captured image and a captured image adjacent to the captured image in the capture order; performing feature point matching on image feature points in different captured images based on the global feature description information of each image feature point in each captured image, to obtain a plurality of feature point sequences, where each image feature point in each feature point sequence is a point in a captured image corresponding to the same feature point at different capture times; determining tracking trajectory information of the road elements to be tracked based on optical flow trajectory information corresponding to at least one of the feature point sequences matched with the road elements to be tracked.

2. The method according to claim 1, wherein The step of, for each captured image, determining global feature description information of each image feature point in the captured image based on the captured image and a captured image adjacent to the captured image in the capture order, includes: for each captured image, determining a plurality of image feature points in the captured image and description feature information of each image feature point; for the captured image and a captured image adjacent to the captured image in the capture order, performing cross-learning processing on the plurality of image feature points in the two captured images based on the description feature information of each image feature point in the two adjacent captured images, to obtain global feature description information of each image feature point in the image pair for the two captured images.

3. The method according to claim 2, wherein The step of, for each captured image, determining a plurality of image feature points in the captured image and description feature information of each image feature point, includes: for each captured image, inputting the captured image into a pre-trained convolutional neural network, and extracting a plurality of image feature points in the captured image and description feature information of each image feature point.

4. The method according to claim 2, characterized in that, The step of, for the captured image and a captured image adjacent to the captured image in the capture order, performing cross-learning processing on the plurality of image feature points in the two captured images based on the description feature information of each image feature point in the two adjacent captured images, to obtain global feature description information of each image feature point in the image pair for the two captured images, includes: for the captured image and a captured image adjacent to the captured image in the capture order, using the description feature information of each image feature point in the two adjacent captured images as input, and inputting the input into a pre-trained graph neural network, to obtain global feature description information of each image feature point in the two captured images for the two captured images.

5. The method according to claim 1, wherein The step of performing feature point matching on image feature points in different captured images based on the global feature description information of each image feature point in each captured image, to obtain a plurality of feature point sequences, where each image feature point in each feature point sequence is a point in a captured image corresponding to the same feature point at different capture times, includes: For each acquired image, determine the position information of each image feature point in the acquired image; Based on the position information of each image feature point and the global feature description information, determine the matching feature vector of each image feature point; For the acquired image and the acquired image adjacent to the acquired image in the acquisition order, input the matching feature vectors of the respective image feature points in the two adjacent acquired images into the trained feature point matching neural network to obtain a plurality of feature point matching results of the two acquired images, where the feature point matching result includes two image feature points, and the two image feature points are respectively located in different acquired images of the two acquired images; Based on the plurality of feature point matching results corresponding to each pair of adjacent acquired images in the plurality of acquired images, determine a plurality of feature point sequences corresponding to the plurality of acquired images, where the feature point sequence includes a plurality of image feature points, and the plurality of image feature points are points in the acquired images acquired at different acquisition times corresponding to the same feature point; 6. The method according to claim 5, wherein Obtain the feature point matching neural network through the following steps: Obtain a plurality of sample acquired images of the target road scene and sparse point cloud data; Based on the sparse point cloud data, generate a dense point cloud map model of the target road scene; For each of the sample acquired images, determine the depth image corresponding to the sample acquired image, and the position information, pose projection information, and depth information corresponding to the depth image in the dense point cloud map model from the dense point cloud map model; Determine a plurality of sample feature points in the sample acquired image, and the sample feature vector of each sample feature point; Use the determined sample feature vectors as inputs, and the position information, pose projection information, and depth information corresponding to the depth image as the supervision information of the neural network to train the constructed neural network to obtain the trained feature point matching neural network.

7. The method according to claim 1, characterized in that After determining the tracking trajectory information of the road element to be tracked based on the optical flow trajectory information corresponding to at least one of the feature point sequences matching the road element to be tracked, the method includes: Based on the tracking trajectory information of the road element to be tracked, determine the position information and identification information of the road element to be tracked in the real road scene; Based on the position information and the identification information, update the map identification of the road element to be tracked in the road map corresponding to the real road scene.

8. The method according to claim 7, characterized in that, The determining the position information of the road element to be tracked in the real road scene based on the tracking trajectory information of the road element to be tracked includes: Determine a target image that meets the image screening conditions from the plurality of acquired images; Based on the tracking trajectory information and the road element to be tracked in the target image, determine the position information of the road element to be tracked in the real road scene.

9. The method according to claim 7, wherein The updating the map identification of the road element to be tracked in the road map corresponding to the real road scene based on the position information and the identification information includes: Check whether the update information of the to-be-tracked road element already exists in the road element database of the real road scene, where the update information includes the position and identification; If not, use the position information and the identification information to update the information of the to-be-tracked road element in the road element database and the map identification of the to-be-tracked road element in the road map corresponding to the real road scene.

10. A road element tracking device, characterized in that, The device includes: An information acquisition module, configured to acquire a plurality of captured images of a real road scene, and the to-be-tracked road element in the plurality of captured images; A feature determination module, configured to, for each captured image, determine the global feature description information of each image feature point in the captured image based on the captured image and the captured image adjacent to the captured image in the acquisition order; A feature point matching module, configured to perform feature point matching on the image feature points in different captured images based on the global feature description information of each image feature point in each captured image, to obtain a plurality of feature point sequences, where each image feature point in each feature point sequence is a point in the captured images captured at different acquisition times corresponding to the same feature point; A trajectory tracking module, configured to determine the tracking trajectory information of the to-be-tracked road element based on the optical flow trajectory information corresponding to at least one of the feature point sequences matched with the to-be-tracked road element.

11. An electronic device, characterized in that, Includes: A processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the road element tracking method according to any one of claims 1 to 9 are executed.

12. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium. When the computer program is run by the processor, the steps of the road element tracking method according to any one of claims 1 to 9 are executed.

13. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, the steps of the road element tracking method according to any one of claims 1 to 9 are implemented.

Images