Vehicle positioning and tracking method and system based on machine vision and map

Abstract

The application discloses a vehicle positioning and tracking method and system based on machine vision and a map, which comprises the following steps: matching each edge pattern with each space representation pattern to form a plurality of pattern matching pairs, calculating a pattern pixel-level transformation matrix, transforming each photographed image to obtain the space position of each pixel of each photographed image, and generating a pixel space position table set; obtaining the geometric transformation relationship and the pixel correspondence relationship of each non-initial photographed image relative to an initial photographed image, and obtaining the space matrix of each non-initial photographed image; according to the current feature point set and each space matrix, if the current real-time image matches the non-initial photographed image corresponding to the space matrix, searching for the pixel space position table corresponding to the non-initial photographed image in the pixel space position table set; according to the pixel space position table, searching for the space coordinate position corresponding to each wheel point position, obtaining the space position of each vehicle at the current moment, and obtaining the trajectory information of each vehicle.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent transportation, in particular to a vehicle positioning and tracking method and system based on machine vision and a map. BACKGROUND

[0002] Traffic condition optimization is the key to prevent and alleviate traffic congestion. Vehicle trajectory can comprehensively and completely represent the traffic flow state, and contains rich traffic flow information.

[0003] However, in the traditional method, the depth information of the image is not accurate, or the visual device is installed on the pole cross arm to collect images, which is prone to shaking problem, causing errors in coordinate conversion, and thus leading to low vehicle positioning accuracy of the traditional method. SUMMARY

[0004] The purpose of the present application is to solve the technical problem of low vehicle positioning accuracy of the traditional method. To achieve the above purpose, the present application provides a vehicle positioning and tracking method and system based on machine vision and a map.

[0005] The present application provides a vehicle positioning and tracking method based on machine vision and a map, comprising:

[0006] Obtaining road map information, and obtaining a space representation graph corresponding to each map graph in each layer according to the road map information;

[0007] Obtaining a plurality of shooting images;

[0008] Edge detection and connection shaping are performed on each shooting image to obtain a plurality of edge graphs corresponding to each shooting image;

[0009] Each edge graph is matched with each space representation graph to form a plurality of graph matching pairs, a graph pixel-level transformation matrix is calculated according to the plurality of graph matching pairs, and each pixel of each shooting image is transformed according to the graph pixel-level transformation matrix to obtain the space position of each pixel of each shooting image, and a pixel space position table set is generated;

[0010] Dynamic image feature matching is performed on the initial shooting image and each non-initial shooting image in the plurality of shooting images to obtain the geometric transformation relationship and the pixel correspondence relationship of each non-initial shooting image relative to the initial shooting image, and a space matrix of each non-initial shooting image is obtained according to the geometric transformation relationship and the pixel correspondence relationship;

[0011] acquire a current feature point set of a current real-time image, and determine whether the current real-time image matches the non-initially photographed image corresponding to each of the spatial matrices according to the current feature point set and each of the spatial matrices, if so, search for a pixel spatial position table corresponding to the non-initially photographed image in the pixel spatial position table set;

[0012] acquire a plurality of wheel point positions of each vehicle in the current real-time image;

[0013] search for a spatial coordinate position corresponding to each of the wheel point positions according to the pixel spatial position table, and obtain a spatial position of each vehicle at a current time;

[0014] obtain trajectory information of each vehicle according to the spatial positions of the same vehicle at different times.

[0015] In one embodiment, after obtaining the trajectory information of the same vehicle according to the spatial positions of the same vehicle at different times, the method further comprises:

[0016] acquire cross-device trajectory information of each cross-device vehicle corresponding to cross-device;

[0017] determine whether the cross-device vehicle and the vehicle are the same target;

[0018] if the cross-device vehicle and the vehicle are the same target, fuse the cross-device trajectory information and the trajectory information according to time to form cross-device vehicle tracking information.

[0019] In one embodiment, matching each of the edge graphs with each of the spatial representation graphs to form a plurality of graph matching pairs, calculating a graph pixel-level transformation matrix according to the plurality of graph matching pairs, and transforming each of the photographed images according to the graph pixel-level transformation matrix to obtain a spatial position of each pixel of each of the photographed images, and generating a pixel spatial position table set, comprising:

[0020] triangularly splitting the edge graph and the spatial representation graph in each of the graph matching pairs to form a plurality of graph triangular pairs;

[0021] performing affine transformation on each of the graph triangular pairs to obtain the graph pixel-level transformation matrix;

[0022] performing edge pixel interpolation on the edge graph in each of the graph matching pairs to obtain a complete edge graph after pairing.

[0023] In one embodiment, the method further comprises:

[0024] triangularly split the non-paired edge graph and the non-paired space representation graph respectively to form a plurality of non-paired triangular graphs;

[0025] perform edge pixel interpolation on each of the non-paired edge graphs to obtain non-paired complete edge graphs;

[0026] add the paired complete edge graphs, the paired space representation graphs, the non-paired complete edge graphs, and the non-paired space representation graphs to obtain a complete image;

[0027] transform the complete image according to the graph pixel-level transformation matrix to obtain a spatial position of each pixel of the complete image, and generate the pixel space position table set.

[0028] In one embodiment, the dynamic image feature matching on the initial photographed image and each non-initial photographed image in the plurality of photographed images is performed to obtain a geometric transformation relationship and a pixel correspondence relationship of each non-initial photographed image relative to the initial photographed image, including:

[0029] obtaining the initial photographed image, and performing feature extraction on the initial photographed image to obtain an initial image feature point set;

[0030] obtaining a plurality of non-initial photographed images, and performing feature point extraction on each non-initial photographed image to obtain a plurality of non-initial image feature point sets;

[0031] matching each non-initial image feature point set with a feature point in the initial image feature point set to obtain a plurality of feature point matching pairs;

[0032] obtaining a geometric transformation relationship between the initial photographed image and each non-initial photographed image according to the plurality of feature point matching pairs;

[0033] performing grid division on the initial photographed image and each non-initial photographed image respectively to form an initial grid region and a non-initial grid region;

[0034] obtaining a grid-level matching relationship corresponding to the same grid position in the initial grid region and the non-initial grid region;

[0035] calculating a pixel relationship corresponding to each grid region in the grid-level matching relationship according to bilinear interpolation.

[0036] In one embodiment, the judging whether the current real-time image matches the non-initial photographed image corresponding to the space matrix according to the current feature point set and each space matrix, if yes, searching for a pixel space position table corresponding to the non-initial photographed image in the pixel space position table set, includes:

[0037] if matched, calculating a matching value of the current real-time image and the non-initial photographed image;

[0038] if the matching value is the best matching, searching for a pixel space position table corresponding to the non-initial photographed image in the pixel space position table set.

[0039] The application provides a vehicle positioning and tracking system based on machine vision and map, comprising:

[0040] a space information acquisition module, configured to acquire road map information, and obtain a space representation graph corresponding to each map graph in each layer according to the road map information;

[0041] an image acquisition module, configured to acquire a plurality of photographed images;

[0042] an edge shaping module, configured to perform edge detection and connection shaping on each photographed image to obtain a plurality of edge graphs corresponding to each photographed image;

[0043] a pixel space conversion module, configured to match each edge graph with each space representation graph to form a plurality of graph matching pairs, calculate a graph pixel-level transformation matrix according to the plurality of graph matching pairs, and transform each photographed image according to the graph pixel-level transformation matrix to obtain a space position of each pixel of each photographed image and generate a pixel space position table set;

[0044] an image feature matching module, configured to perform dynamic image feature matching on an initial photographed image and each non-initial photographed image in the plurality of photographed images to obtain a geometric transformation relationship and a pixel correspondence relationship of each non-initial photographed image relative to the initial photographed image, and obtain a space matrix of each non-initial photographed image according to the geometric transformation relationship and the pixel correspondence relationship;

[0045] a jitter correction module, configured to acquire a current feature point set of a current real-time image, and determine whether the current real-time image and the non-initial photographed image corresponding to the space matrix match according to the current feature point set and each space matrix, if matched, search for a pixel space position table corresponding to the non-initial photographed image in the pixel space position table set;

[0046] a vehicle positioning module, configured to acquire a plurality of wheel point positions of each vehicle in the current real-time image;

[0047] a space position searching module, configured to search for a space coordinate position corresponding to each wheel point position according to the pixel space position table to obtain a space position of each vehicle at a current time;

[0048] a trajectory generation module configured to obtain trajectory information of each vehicle according to the spatial position of the same vehicle at each time.

[0049] In one embodiment, the system further comprises:

[0050] a cross-device trajectory information acquisition module configured to acquire cross-device trajectory information of each cross-device vehicle corresponding to the cross-device;

[0051] a same target judgment module configured to judge whether the cross-device vehicle and the vehicle are the same target;

[0052] a cross-device fusion module configured to, if the cross-device vehicle and the vehicle are the same target, fuse the cross-device trajectory information and the trajectory information according to time to form cross-device vehicle tracking information.

[0053] In one embodiment, the pixel space conversion module comprises:

[0054] a graph triangle pair acquisition module configured to perform triangle splitting on the edge graph and the spatial representation graph in each of the graph matching pairs to form a plurality of graph triangle pairs;

[0055] an affine transformation module configured to perform affine transformation on each of the graph triangle pairs to obtain the graph pixel-level transformation matrix;

[0056] a pixel interpolation module configured to perform edge pixel interpolation on the edge graph in each of the graph matching pairs to obtain a paired complete edge graph.

[0057] In one embodiment, the system further comprises:

[0058] a non-paired graph acquisition module configured to perform triangle splitting on the non-paired edge graph and the non-paired spatial representation graph to form a plurality of non-paired triangle graphs;

[0059] a non-paired pixel interpolation module configured to perform edge pixel interpolation on each of the non-paired edge graphs to obtain a non-paired complete edge graph;

[0060] an edge correction module configured to add the paired complete edge graph, the paired spatial representation graph, the non-paired complete edge graph, and the non-paired spatial representation graph to obtain a complete image;

[0061] a pixel space position table generation module configured to transform the complete image according to the graph pixel-level transformation matrix to obtain the spatial position of each pixel of the complete image and generate the pixel space position table set.

[0062] In one embodiment, the image feature matching module comprises:

[0063] An initial feature acquisition module is configured to acquire the initial captured image, and perform feature extraction on the initial captured image to obtain an initial image feature point set.

[0064] A non-initial feature acquisition module is configured to acquire a plurality of non-initial captured images, and perform feature point extraction on each of the non-initial captured images to obtain a plurality of non-initial image feature point sets.

[0065] A feature point matching module is configured to match feature points in each of the non-initial image feature point sets with the initial image feature point set to obtain a plurality of feature point matching pairs.

[0066] A geometric transformation calculation module is configured to obtain a geometric transformation relationship between the initial captured image and each of the non-initial captured images according to the plurality of feature point matching pairs.

[0067] A grid division module is configured to perform grid division on the initial captured image and each of the non-initial captured images respectively to form initial grid regions and non-initial grid regions.

[0068] A grid matching relationship acquisition module is configured to acquire a grid level matching relationship corresponding to the same grid position in the initial grid regions and the non-initial grid regions.

[0069] A pixel relationship calculation module is configured to calculate a pixel relationship corresponding to each grid region in the grid level matching relationship according to bilinear interpolation.

[0070] In one embodiment, the shake correction module comprises:

[0071] A matching value calculation module is configured to calculate a matching value of the current real-time image and the non-initial captured image if matching.

[0072] A search module is configured to search for a pixel space position table corresponding to the non-initial captured image in the pixel space position table set if the matching value is the best matching.

[0073] The aforementioned vehicle positioning and tracking method and system based on machine vision and maps matches and maps data extracted from road map information with data extracted from captured images, establishing a mapping relationship between road map information and captured image information. This combination of road map information and captured image information allows for target location positioning from different angles, solving the problem of low positioning accuracy caused by inaccurate depth information in traditional methods. Simultaneously, a dynamic image feature matching step corrects for image jitter. Furthermore, the current real-time image is quickly and efficiently matched to the corresponding pixel spatial location table to find the spatial coordinates of the vehicle's wheel points, thus determining the spatial position of each vehicle at the current moment. Based on the spatial positions of the same vehicle at various times, the trajectory information of each vehicle is obtained. The vehicle positioning and tracking method based on machine vision and maps provided by this invention not only solves the accuracy problem of vehicle positioning but also the problem of image stabilization. This method offers high accuracy and anti-jitter properties, providing high-precision and stable vehicle trajectories, and can be widely applied in fields such as intelligent transportation, intelligent public security, intelligent traffic management, intelligent parking, and urban traffic signal control. Attached Figure Description

[0074] Figure 1 This is a flowchart illustrating the steps of the vehicle positioning and tracking method based on machine vision and maps provided by the present invention.

[0075] Figure 2 This is a schematic diagram of the road information layer provided by the present invention.

[0076] Figure 3 This is a schematic diagram of the connection of spatial points in the spatial representation diagram provided by the present invention.

[0077] Figure 4 (a) is a schematic diagram of the captured image provided by the present invention. Figure 4 (b) is a schematic diagram of the captured image after edge detection.

[0078] Figure 5 (a) is a schematic diagram of connecting edge lines into a graphic according to the present invention. Figure 5 (b) is from Figure 5 (a) shows the multiple edge graphics extracted.

[0079] Figure 6 This is a schematic diagram illustrating the one-to-one matching of the spatial representation map and the edge graphics provided by the present invention.

[0080] Figure 7(a) is a schematic diagram of the edge pattern in the pattern matching pair provided by the present application, (b) is a schematic diagram of the space representation pattern in the pattern matching pair provided by the present application, (c) is a complete edge pattern after the edge pixel interpolation provided by the present application, and (d) is a complete image after the complete edge pattern after the pattern matching, the space representation pattern after the pattern matching, the complete edge pattern without the pattern matching, and the space representation pattern without the pattern matching are added.

[0081] Figure 8 is an overlapping schematic diagram of the same vehicle position of the multiple-frame image in the overlapping area provided by the present application.

[0082] Figure 9 is a structural schematic diagram of the vehicle positioning and tracking system based on machine vision and map provided by the present application. DETAILED DESCRIPTION

[0083] The technical solutions of the present application are described in further detail below with reference to the drawings and examples.

[0084] Please refer to Figure 1 The present application provides a vehicle positioning and tracking method based on machine vision and map, comprising:

[0085] S10, acquiring road map information, and obtaining a space representation pattern corresponding to each map pattern in each layer according to the road map information;

[0086] S20, acquiring multiple photographed images;

[0087] S30, performing edge detection and connection shaping on each photographed image to obtain multiple edge patterns corresponding to each photographed image;

[0088] S40, matching each edge pattern with each space representation pattern to form multiple pattern matching pairs, calculating a pattern pixel-level transformation matrix according to the multiple pattern matching pairs, and transforming each photographed image according to the pattern pixel-level transformation matrix to obtain a space position of each pixel of each photographed image, and generating a pixel space position table set;

[0089] S50, performing dynamic image feature matching on an initial photographed image and each non-initial photographed image in the multiple photographed images to obtain a geometric transformation relationship and a pixel correspondence relationship of each non-initial photographed image relative to the initial photographed image, and obtaining a space matrix of each non-initial photographed image according to the geometric transformation relationship and the pixel correspondence relationship;

[0090] S60, acquiring a current feature point set of a current real-time image, and judging whether the current real-time image matches a non-initial photographed image corresponding to the space matrix according to the current feature point set and each space matrix, and if so, searching for a pixel space position table corresponding to the non-initial photographed image in the pixel space position table set.

[0091] S70, obtain the positions of multiple wheel points for each vehicle in the current real-time image;

[0092] S80: Based on the pixel spatial location table, find the spatial coordinates corresponding to the position of each wheel point to obtain the spatial position of each vehicle at the current moment.

[0093] S90 obtains the trajectory information of each vehicle based on the spatial location of the same vehicle at various times.

[0094] In this embodiment, the road map information comes from a high-precision map, including various road information layers such as lane boundaries, lane lines, stop lines, speed bumps, pedestrian crossing lines, pedestrian waiting area lines, bus bay lines, intersection center circle markings, no-parking zone markings, central median, traffic dividers, traffic islands, roadside parking space lines, ground markings, and turning lines. Figure 2 As shown. Each road information layer is a dataset, a polygon layer, such as a lane line layer. For a given layer, there are multiple map graphics with unique IDs, such as map graphic 1, map graphic 2, map graphic 3, ..., map graphic n. Each map graphic corresponds to a spatial description, which can be described by multiple spatial points or a planar projection. Each graphic corresponds to multiple spatial points with unique IDs, and each spatial point can be described by its longitude lng. i latitude i Information is represented, and spatial points are connected in ascending order by ID. The first and last points of the map graphic form a closed surface, thus obtaining a spatial representation map corresponding to each map graphic in each layer, such as... Figure 3 As shown.

[0095] Images are captured using devices such as cameras. These images are specifically represented in pixel space, including pixel information of various road elements such as lane boundaries, lane lines, stop lines, speed bumps, pedestrian crossings, pedestrian waiting area lines, bus bay lines, intersection center circle markings, no-stopping zone markings, central dividers, traffic islands, roadside parking space lines, ground markings, and turning lines. The Canny edge detection algorithm is used to detect the edges of these ground road elements in the captured images. The detection result is the edge lines of these ground road elements, such as... Figure 4 As shown, the edge lines are connected to form a graphic according to the actual image, resulting in multiple edge graphics corresponding to each captured image, such as... Figure 5 As shown, a captured image may contain multiple edge features. The spatial representation of the high-precision map layer is matched one-to-one with the edge features detected from the image edges, as shown below. Figure 6 As shown, it can also be expressed as:

[0096] Pair bm= {(shape HD1 , shape IMG1 ), (shape HD2 , shape IMG2 ), (shape HD3 , shape IMG3 )... (shape HD1 , shape IMGn )}.

[0097] Matching each edge graph with each space representation graph can be understood as target searching and corresponding for each edge graph in the photographed image and each space representation graph in the road map information. If the edge graph and the space representation graph correspond to the same target, it is considered that the two are matched, and then each edge graph and each space representation graph are matched to form a plurality of graph matching pairs. For each graph matching pair, there is respectively an edge graph and a space representation graph, both of which correspond to the same target, and the conversion relationship between the two-dimensional pixel coordinates and the latitude and longitude space coordinates is calculated to obtain the corresponding graph pixel-level transformation matrix. The graph pixel-level transformation matrix represents the corresponding transformation relationship of the edge graph and the space representation graph in the graph matching pair, and can be converted between each other. Thus, according to the graph pixel-level transformation matrix, each photographed image can be transformed to obtain the corresponding latitude and longitude space position of each pixel in each photographed image, and a corresponding relationship is formed one by one to generate a pixel space position table set.

[0098] Dynamic image feature matching of the initial photographed image and each non-initial photographed image in the plurality of photographed images is matching the initial image feature points in the initial photographed image with the non-initial image feature points in the non-initial photographed image, and further obtaining the corresponding geometric transformation relationship and pixel correspondence relationship on the basis of the feature point matching. The geometric transformation relationship represents the geometric transformation between the two pictures of each non-initial photographed image and the initial photographed image, such as translation, rotation, scaling and other transformations, to obtain the corresponding geometric transformation relationship. The pixel correspondence relationship represents the pixel relationship of the corresponding regions between the non-initial photographed image and the initial photographed image. On the basis of the geometric transformation relationship and the pixel correspondence relationship, the space matrix of each non-initial photographed image can be further obtained. The space matrix corresponding to the non-initial photographed image is the corrected space matrix on the basis of the initial photographed image. Dynamic image feature matching of the initial photographed image and each non-initial photographed image in the plurality of photographed images can solve the shaking problem caused by the visual device being installed on the cross arm of the rod body due to wind, vibration and other factors, and achieve shaking correction through dynamic image feature matching.

[0099] Each non-initially photographed image corresponds to a spatial matrix, and the spatial matrix represents the non-initially photographed image after the shake correction. According to the current feature point set and the spatial matrix, it can be judged whether the current real-time image matches the non-initially photographed image. If matched, it is considered that the current real-time image matches the non-initially photographed image, corresponding to the same pixel space position table. Through the pixel space position table corresponding to the non-initially photographed image, the relationship between the current real-time image corresponding pixel and the spatial position can be found.

[0100] Through the deep learning algorithm, the four-wheel finding position of all vehicles in the current real-time image in the image is detected, such as the left front wheel position, the right front wheel position, the left rear wheel position and the right rear wheel position. The two-dimensional pixel coordinates corresponding to the left front wheel position, the right front wheel position, the left rear wheel position and the right rear wheel position are respectively: location_img mj = {(x lh_mj , y lh_mj ), (x rh_mj , y rh_mj ), (x lr_mj , y lr_mj ), (x rr_mj , y rr_mj )}.

[0101] In the pixel space position table corresponding to the current real-time image, the two-dimensional pixel coordinates location_img mj corresponding to the spatial coordinate position represented by the latitude and longitude information or the projection coordinate position is found, that is, the spatial position of each vehicle at the current time is obtained. The spatial coordinate position represented by the latitude and longitude information can be represented as: location_spa mj = {(lng lh_mj , lat lh_mj ), (lng rh_mj , lat rh_mj ), (lng lr_mj , lat lr_mj ), (lng rr_mj , lat rr_mj )}.

[0102] Through the vehicle tracking algorithm, the unique identity of vehicle j at different times can be obtained. The spatial positions of the same vehicle (such as vehicle j) at different times are formed into a set according to the time sequence relationship, and the trajectory information of the same vehicle (such as vehicle j) is obtained.

[0103] The vehicle positioning and tracking method based on machine vision and map provided by the application matches the data information extracted from the road map information with the data information extracted from the photographed image, establishes the mapping relationship between the road map information and the photographed image information, combines the road map information with the photographed image information, and positions the target position from different angles, so that the problem of low positioning accuracy caused by inaccurate depth information of the image in the traditional method can be solved. At the same time, the photographed image is corrected for shaking through the dynamic image feature matching step. Further, the current real-time image is quickly matched to the corresponding pixel space position table, the spatial coordinate position of the wheel point of the vehicle is found, the spatial position of each vehicle at the current time is obtained, and thus the trajectory information of each vehicle is obtained according to the spatial position of the same vehicle at each time. Through the vehicle positioning and tracking method based on machine vision and map provided by the application, the precision problem of vehicle positioning is solved, and the anti-shaking problem is also solved. The vehicle positioning and tracking method based on machine vision and map provided by the application is high in precision and anti-shaking, can provide high-precision and stable vehicle trajectory, and can be widely applied to the fields of intelligent traffic, intelligent public security, intelligent traffic management, intelligent parking, urban traffic signal control and the like.

[0104] In one embodiment, S40, each edge graph is matched with each space representation graph to form a plurality of graph matching pairs, a graph pixel-level transformation matrix is calculated according to the plurality of graph matching pairs, and each photographed image is transformed according to the graph pixel-level transformation matrix to obtain the spatial position of each pixel of each photographed image, and a pixel space position table set is generated, including:

[0105] S410, the edge graph and the space representation graph in each graph matching pair are respectively triangularly split to form a plurality of graph triangular pairs;

[0106] S420, affine transformation is performed on each graph triangular pair to obtain a graph pixel-level transformation matrix;

[0107] S430, edge pixel interpolation is performed on the edge graph in each graph matching pair to obtain a complete paired edge graph.

[0108] In this embodiment, the edge graph is derived from the captured image, and the space representation graph is derived from the road map information. The edge graph in the matched pair of graphs is triangulated, and the space representation graph in the matched pair of graphs is triangulated, forming a corresponding pair of graph triangles. In one embodiment, the triangulation of the matched pair of edge graph and space representation graph is performed by the Delaunay Triangulation method. For each pair of graph triangles, the transformation relationship is calculated by the affine transformation, obtaining the graph pixel level transformation relationship between the triangular part of the edge graph and the corresponding triangular part of the space representation graph, forming the graph pixel level transformation matrix. The graph pixel level transformation matrix represents the corresponding transformation relationship between the edge graph and the space representation graph in the matched pair of graphs, and can be converted between each other. The edge pixel interpolation is performed on the edge graph in each matched pair of graphs to complete the edge graph.

[0109] In one embodiment, the vehicle positioning and tracking method based on machine vision and map further comprises:

[0110] S440, the non-matched edge graph and the non-matched space representation graph are respectively triangulated to form a plurality of non-matched triangular graphs;

[0111] S450, the edge pixel interpolation is performed on the non-matched edge graph to obtain a non-matched complete edge graph;

[0112] S460, the matched complete edge graph, the matched space representation graph, the non-matched complete edge graph and the non-matched space representation graph are added to obtain a complete image;

[0113] S470, the complete image is transformed according to the graph pixel level transformation matrix to obtain the spatial position of each pixel of the complete image, and a pixel space position table set is generated.

[0114] In this embodiment, the triangulation method of the non-matched edge graph is the same as that in step S410. The method of performing edge pixel interpolation on the non-matched triangular graph is the same as that in step S430. The matched complete edge graph, the matched space representation graph, the non-matched complete edge graph and the non-matched space representation graph are added to obtain a complete image, as shown in Figure 7 .

[0115] The graph pixel-level transformation matrix represents the corresponding transformation relationship between the edge graph and the spatial representation graph in the graph matching pair, and can be converted between each other. According to the graph pixel-level transformation matrix, the spatial position of each pixel in the corresponding complete image can be obtained, and then the latitude and longitude spatial information in the map is corresponded, so that the spatial position of each pixel in the entire image can be obtained, and a pixel spatial position table set is generated.

[0116] In one embodiment, S50, dynamic image feature matching is performed on the initial photographed image and each non-initial photographed image in the plurality of photographed images, to obtain the geometric transformation relationship and the pixel correspondence relationship of each non-initial photographed image relative to the initial photographed image, including:

[0117] S510, obtaining the initial photographed image and performing feature extraction on the initial photographed image to obtain an initial image feature point set;

[0118] S520, obtaining a plurality of non-initial photographed images and performing feature point extraction on each non-initial photographed image to obtain a plurality of non-initial image feature point sets;

[0119] S530, matching the feature points in each non-initial image feature point set and the initial image feature point set to obtain a plurality of feature point matching pairs;

[0120] S540, obtaining the geometric transformation relationship between the initial photographed image and each non-initial photographed image according to the plurality of feature point matching pairs;

[0121] S550, respectively performing grid division on the initial photographed image and each non-initial photographed image to form an initial grid area and a non-initial grid area;

[0122] S560, obtaining a grid-level matching relationship corresponding to the same grid position in the initial grid area and the non-initial grid area;

[0123] S570, calculating the pixel relationship corresponding to each grid area in the grid-level matching relationship according to the bilinear interpolation.

[0124] In this embodiment, the feature points of the initial shot image are extracted by the SURF algorithm to obtain an initial image feature point set. The plurality of non-initial shot images are a plurality of actual pictures in a period of time. The plurality of actual pictures are imaging results under different shaking conditions. The feature points of each non-initial shot image are extracted by the SURF algorithm to obtain a non-initial image feature point set corresponding to each non-initial shot image. The feature point matching pairs between each non-initial image feature point set and the feature points in the initial image feature point set are calculated by the NNDR (Nearest Neighbor Distance Ratio) algorithm to form a plurality of feature point matching pairs, thereby achieving dynamic image feature matching between the initial shot image and each non-initial shot image in the plurality of shot images.

[0125] According to the relationship of the plurality of feature point matching pairs, the RANSAC (Random Sample Consensus) algorithm is used to estimate the geometric transformation, such as translation, rotation, scaling, etc., between the two pictures of each non-initial shot image and the initial shot image to obtain the corresponding geometric transformation relationship.

[0126] The initial shot image is divided into grids, and each non-initial shot image is also divided into grids. In an embodiment, the grid can be a square grid (16x16) specification. Each grid corresponds to a small block area of the initial shot image and a small block area of the non-initial shot image, forming an initial grid area and a non-initial grid area. When performing area matching, the SSD (sum of Squared Difference) method is used to find the most matching corresponding relationship between the initial grid area and the non-initial grid area in each grid, that is, to obtain the grid-level matching relationship. The SSD (sum of Squared Difference) method is a method for measuring the difference between two images, and the calculation formula is: SSD = Σ (x1-x2) 2 where x1 and x2 represent the pixel values between two images, the two images are adjusted to the same size, the gray values (or color channel values) of the two images are compared pixel by pixel, the square of the difference between the gray values of each pixel is calculated, and the sum of the squares of the differences of all pixels is calculated. The smaller the SSD value is, the more similar the two images are, and the more matched they are. If the SSD is 0, it means that the two images are completely the same. By using the SSD (sum of Squared Difference) method, the grid-level matching relationship can be quickly calculated and obtained, which is easy to implement. According to the bilinear interpolation, the pixel relationship in each grid area in the grid-level matching relationship is calculated. According to the pixel relationship, the relationship between the initial shot image and the spatial matrix of each non-initial shot image can be further obtained.

[0127] Therefore, by performing dynamic image feature matching on the initial photographed image and each non-initial photographed image in the plurality of photographed images in this embodiment, the problem of shaking due to the installation of the visual device on the horizontal arm of the rod body and the shaking caused by wind, vibration and other factors can be solved, and shaking correction through dynamic image feature matching is achieved.

[0128] In one embodiment, S60, according to the current feature point set and each spatial matrix, it is judged whether the current real-time image matches the non-initial photographed image corresponding to the spatial matrix. If matched, the pixel space position table corresponding to the non-initial photographed image is searched in the pixel space position table set, including:

[0129] S610, if matched, the matching value of the current real-time image and the non-initial photographed image is calculated;

[0130] S620, if the matching value is the best matching, the pixel space position table corresponding to the non-initial photographed image is searched in the pixel space position table set.

[0131] In this embodiment, for each current real-time image, the corresponding current feature point set is obtained by FREAK (Fast Retina Keypoint) method. According to the current feature point set and each spatial matrix, it is judged whether the current real-time image matches the non-initial photographed image corresponding to the spatial matrix by FNNS (Fast Nearest Neighbor Search) algorithm, and then the matching relationship between the current real-time image and the non-initial photographed image is calculated. If matched, the matching value of the current real-time image and the non-initial photographed image is calculated by RANSAC (Random Sample Consensus) algorithm. When the matching value is the best matching, it is considered that it belongs to the high matching value, has high matching degree, and further searching in the pixel space position table set is performed to obtain the pixel space position table corresponding to the non-initial photographed image most matched with the current real-time image as the pixel space position table corresponding to the current real-time image.

[0132] In one embodiment, by using a deep learning algorithm, the four-wheel finding position of all vehicles in the current real-time image in the image is detected, such as the left front wheel position, the right front wheel position, the left rear wheel position and the right rear wheel position. The two-dimensional pixel coordinates corresponding to the left front wheel position, the right front wheel position, the left rear wheel position and the right rear wheel position are respectively: location_img mj = {(x lh_mj , y lh_mj ), (x rh_mj , y rh_mj ), (x lr_mj , y lr_mj ), (x rr_mj , y rr_mj )}.

[0133] In the pixel space position table corresponding to the current real-time image, the two-dimensional pixel coordinates location_img mj corresponding to the spatial coordinate position represented by the longitude and latitude information or the projection coordinate position. The spatial coordinate position represented by the longitude and latitude information can be represented as: location_spa mj = {(lng lh_mj , lat lh_mj ), (lng rh_mj , lat rh_mj ), (lng lr_mj , lat lr_mj ), (lng rr_mj , lat rr_mj )}.

[0134] In one embodiment, S90, after obtaining the trajectory information of the same vehicle according to the spatial position of the same vehicle at each time, the method further comprises:

[0135] S100, obtaining the cross-device trajectory information of each cross-device vehicle corresponding to the cross-device;

[0136] S110, determining whether the cross-device vehicle and the vehicle are the same target;

[0137] S120, if the cross-device vehicle and the vehicle are the same target, then fuse the cross-device trajectory information and the trajectory information according to time to form the cross-device vehicle tracking information.

[0138] In this embodiment, the cross-device can be understood as a cross-regional device, and the trajectory information is fused between multiple devices. A larger coverage range is obtained through the relay mode of multiple cross-devices. The trajectory information of the cross-device is obtained through steps S10 to S90, and then the trajectory information of each vehicle corresponding to the device can be obtained. In one embodiment, multiple cross-devices at least ensure an overlapping area of more than one vehicle length in the regional coverage range. The multiple cross-devices are synchronized by NTP, and frame synchronization is ensured by trigger capture, so that the cross-device data fusion is more accurate.

[0139] When the same vehicle is determined between the vehicles of multiple cross-devices, in the overlapping area of the devices, the vehicles positioned by different devices at the same time are matched through the spatial position formed by the wheel landing points of each vehicle, and it is determined whether the overlapping area of the spatial position formed by the wheel landing points is greater than a threshold value, if greater, it is determined that the two vehicles are the same target, for example Figure 8The space position formed by the wheel landing points of the vehicle can be a quadrangle formed by the wheel landing points of the vehicle in one embodiment. Further, it is determined whether the overlapping area of the two quadrangles is greater than a threshold value. If the overlapping area is greater than the threshold value, the two vehicles are determined to be the same target.

[0140] When the cross-device vehicles are the same target, the multiple sets of data obtained under different devices can be fused to form cross-device vehicle tracking information. For example, vehicle j in device 1 and vehicle i in device 2 are determined to be the same target vehicle. The trajectory information of the vehicle obtained by device 1 is:

[0141] trace 1j ={location_spa 1j_t1 ,location_spa 1j_t2 location_spa 1j_t3 ,...location_spa 1j_tn}。

[0142] The trajectory information of the vehicle obtained by device 2 is:

[0143] trace 2i ={location_spa 2i_t(n-3) ,location_spa 2i_t(n-2) ,location_spa 2i_t(n-1) ,…location_spa 2i_t(2n-4)}。

[0144] The trajectory information of the vehicle corresponding to device 1 is fused with the trajectory information of the vehicle corresponding to device 2 based on time to form cross-device vehicle tracking information, as shown below:

[0145] trace uni ={location_spa 1j_t1 ,,location_spa 1j_t2 location_spa 1j_t3 ,...location_spa 1j_tn ,location_spa 2i_t(n+1) ,location_spa 2i_t(n+2) location_spa 2i_t(n+3) ,…location_spa 2i_t(2n-4)}。

[0146] The trajectory information of the vehicle obtained by the device 1 at a time t1 to a time tn is connected and fused with the trajectory information of the vehicle obtained by the device 2 at a time t (n+1) to a time t (2n-4) to form cross-device vehicle tracking information.

[0147] The vehicle positioning and tracking method based on machine vision and map provided by the application includes single-device vehicle high-precision positioning and tracking and cross-device vehicle high-precision positioning and tracking, the long trajectory of the same target is spliced by the same target determination method in the overlapping area, the long-distance tracking problem of the vehicle is solved, and the positioning and tracking of the same vehicle across devices are realized. The vehicle positioning and tracking method based on machine vision and map provided by the application has the characteristics of high precision, anti-shaking and good continuity, can provide high-precision and stable vehicle trajectory, can provide high-precision and stable data for security, parking, traffic order, traffic management, traffic signal control and the like, and realizes accurate digital management target.

[0148] Please refer to Figure 9 The application provides a vehicle positioning and tracking system 100 based on machine vision and map. The vehicle positioning and tracking system 100 based on machine vision and map includes a spatial information acquisition module 10, an image acquisition module 20, an edge shaping module 30, a pixel space conversion module 40, an image feature matching module 50, a shaking correction module 60, a vehicle positioning module 70, a spatial position searching module 80 and a trajectory generation module 90. The spatial information acquisition module 10 is used to acquire road map information, and obtain a spatial representation graph corresponding to each map graph in each layer according to the road map information. The image acquisition module 20 is used to acquire a plurality of photographed images.

[0149] The edge shaping module 30 is used to perform edge detection and connection shaping on each photographed image to obtain a plurality of edge graphs corresponding to each photographed image. The pixel space conversion module 40 is used to match each edge graph with each spatial representation graph to form a plurality of graph matching pairs, calculate a graph pixel-level transformation matrix according to the plurality of graph matching pairs, and transform each photographed image according to the graph pixel-level transformation matrix to obtain the spatial position of each pixel of each photographed image and generate a pixel spatial position table set. The image feature matching module 50 is used to perform dynamic image feature matching on an initial photographed image and each non-initial photographed image in the plurality of photographed images to obtain the geometric transformation relationship and the pixel correspondence relationship of each non-initial photographed image relative to the initial photographed image, and obtain the spatial matrix of each non-initial photographed image according to the geometric transformation relationship and the pixel correspondence relationship.

[0150] The jitter correction module 60 is configured to acquire a current feature point set of a current real-time image, and determine whether the current real-time image matches a non-initially photographed image corresponding to each spatial matrix according to the current feature point set and each spatial matrix. If the current real-time image matches the non-initially photographed image, the jitter correction module 60 is configured to search for a pixel spatial position table corresponding to the non-initially photographed image in the pixel spatial position table set. The vehicle positioning module 70 is configured to acquire a plurality of wheel point positions of each vehicle in the current real-time image. The spatial position searching module 80 is configured to search for a spatial coordinate position corresponding to each wheel point position according to the pixel spatial position table, and obtain a spatial position of each vehicle at a current time. The trajectory generation module 90 is configured to obtain trajectory information of each vehicle according to the spatial positions of the same vehicle at different times.

[0151] In the embodiment, the related description of the spatial information acquisition module 10 can refer to the related description of S10 in the above embodiment. The related description of the image acquisition module 20 can refer to the related description of S20 in the above embodiment. The related description of the edge shaping module 30 can refer to the related description of S30 in the above embodiment. The related description of the pixel spatial conversion module 40 can refer to the related description of S40 in the above embodiment. The related description of the image feature matching module 50 can refer to the related description of S50 in the above embodiment. The related description of the jitter correction module 60 can refer to the related description of S60 in the above embodiment. The related description of the vehicle positioning module 70 can refer to the related description of S70 in the above embodiment. The related description of the spatial position searching module 80 can refer to the related description of S80 in the above embodiment. The related description of the trajectory generation module 90 can refer to the related description of S90 in the above embodiment.

[0152] In one embodiment, the vehicle positioning and tracking system 100 based on machine vision and map further comprises a cross-device trajectory information acquisition module, a same target judgment module, and a cross-device fusion module. The cross-device trajectory information acquisition module is configured to acquire cross-device trajectory information of each cross-device vehicle corresponding to the cross-device. The same target judgment module is configured to determine whether the cross-device vehicle and the vehicle are the same target. The cross-device fusion module is configured to, if the cross-device vehicle and the vehicle are the same target, fuse the cross-device trajectory information and the trajectory information according to time to form cross-device vehicle tracking information.

[0153] In the embodiment, the related description of the cross-device trajectory information acquisition module can refer to the related description of S100 in the above embodiment. The related description of the same target judgment module can refer to the related description of S110 in the above embodiment. The related description of the cross-device fusion module can refer to the related description of S120 in the above embodiment.

[0154] In one embodiment, the pixel space conversion module includes a graphic triangle pair acquisition module, an affine transformation module, and a pixel interpolation module. The graphic triangle pair acquisition module is configured to perform triangle decomposition on the edge graphic and the space representation graphic in each graphic matching pair, respectively, to form a plurality of graphic triangle pairs. The affine transformation module is configured to perform affine transformation on each graphic triangle pair to obtain a graphic pixel-level transformation matrix. The pixel interpolation module is configured to perform edge pixel interpolation on the edge graphic in each graphic matching pair to obtain a complete edge graphic after pairing.

[0155] In this embodiment, the related description of the graphic triangle pair acquisition module can refer to the related description of S410 in the above embodiment. The related description of the affine transformation module can refer to the related description of S420 in the above embodiment. The related description of the pixel interpolation module can refer to the related description of S430 in the above embodiment.

[0156] In one embodiment, the vehicle positioning and tracking system 100 based on machine vision and map further includes a non-paired graphic acquisition module, a non-paired pixel interpolation module, an edge correction module, and a pixel space position table generation module. The non-paired graphic acquisition module is configured to perform triangle decomposition on the non-paired edge graphic and the non-paired space representation graphic, respectively, to form a plurality of non-paired triangle graphics. The non-paired pixel interpolation module is configured to perform edge pixel interpolation on each non-paired edge graphic to obtain a complete non-paired edge graphic. The edge correction module is configured to add the complete edge graphic after pairing, the complete space representation graphic after pairing, the complete non-paired edge graphic, and the complete non-paired space representation graphic to obtain a complete image. The pixel space position table generation module is configured to transform the complete image according to the graphic pixel-level transformation matrix to obtain the space position of each pixel of the complete image, and generate a pixel space position table set.

[0157] In this embodiment, the related description of the non-paired graphic acquisition module can refer to the related description of S440 in the above embodiment. The related description of the non-paired pixel interpolation module can refer to the related description of S450 in the above embodiment. The related description of the edge correction module can refer to the related description of S460 in the above embodiment. The related description of the pixel space position table generation module can refer to the related description of S470 in the above embodiment.

[0158] In one embodiment, the image feature matching module comprises an initial feature acquisition module, a non-initial feature acquisition module, a feature point matching module, a geometric transformation calculation module, a grid division module, a grid matching relationship acquisition module, and a pixel relationship calculation module. The initial feature acquisition module is configured to acquire an initial captured image, and perform feature extraction on the initial captured image to obtain an initial image feature point set. The non-initial feature acquisition module is configured to acquire a plurality of non-initial captured images, and perform feature point extraction on each non-initial captured image to obtain a plurality of non-initial image feature point sets. The feature point matching module is configured to match the feature points in each non-initial image feature point set with the initial image feature point set to obtain a plurality of feature point matching pairs.

[0159] The geometric transformation calculation module is configured to obtain a geometric transformation relationship between the initial captured image and each non-initial captured image according to the plurality of feature point matching pairs. The grid division module is configured to perform grid division on the initial captured image and each non-initial captured image respectively to form initial grid regions and non-initial grid regions. The grid matching relationship acquisition module is configured to acquire a grid level matching relationship corresponding to the same grid positions in the initial grid regions and the non-initial grid regions. The pixel relationship calculation module is configured to calculate a pixel relationship corresponding to each grid region in the grid level matching relationship according to bilinear interpolation.

[0160] In this embodiment, the related description of the initial feature acquisition module can refer to the related description of S510 in the above embodiment. The related description of the non-initial feature acquisition module can refer to the related description of S520 in the above embodiment. The related description of the feature point matching module can refer to the related description of S530 in the above embodiment. The related description of the geometric transformation calculation module can refer to the related description of S540 in the above embodiment. The related description of the grid division module can refer to the related description of S550 in the above embodiment. The related description of the grid matching relationship acquisition module can refer to the related description of S560 in the above embodiment. The related description of the pixel relationship calculation module can refer to the related description of S570 in the above embodiment.

[0161] In one embodiment, the shake correction module 60 comprises a matching value calculation module and a lookup module. The matching value calculation module is configured to calculate a matching value of the current real-time image and the non-initial captured image if the matching is successful. The lookup module is configured to look up the pixel space position table corresponding to the non-initial captured image in the pixel space position table set if the matching value is the best matching.

[0162] In this embodiment, the related description of the matching value calculation module can refer to the related description of S610 in the above embodiment. The related description of the lookup module can refer to the related description of S620 in the above embodiment.

[0163] In the various embodiments described above, the particular order or hierarchy of steps in processes disclosed should not be understood to represent a limitation but shall be understood to be an example of an illustrative order. Based upon design preferences, it is understood that the particular order or hierarchy of steps in processes can be rearranged while remaining within the scope of the present disclosure. The accompanying method claims present elements of the various steps in exemplary order and are not meant to be limited to the specific order or hierarchy presented.

[0164] Those of skill would further appreciate that the various illustrative logical blocks, modules, and steps described in connection with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.

[0165] The various illustrative logical blocks, modules, and steps described in connection with the embodiments disclosed herein can be implemented or performed by a general purpose processor, a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor can be a microprocessor, but in the alternative, the general purpose processor can be any conventional processor, controller, microcontroller, or state machine. A processor can also be implemented as a combination of computing devices, e.g., a combination of a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other such configuration.

[0166] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium can be coupled to the processor, such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The processor and the storage medium can reside in an ASIC. The ASIC can reside in a user terminal. In the alternative, the processor and the storage medium can reside as discrete components in a user terminal. The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. The various methods described herein can be implemented on a computer using software having suitable instructions or programming code applied as needed. The embodiments described herein are not inherently related to any particular computer or other apparatus. Various hardware and software

[0167] The above detailed description of the specific embodiments of the present application is provided for the purpose of further explaining the objects, technical solutions and advantages of the present application, and it should be understood that the above description is only a specific embodiment of the present application and is not used to limit the protection scope of the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A vehicle localization and tracking method based on machine vision and mapping, characterized in that, include: Obtain road map information, and based on the road map information, obtain a spatial representation map corresponding to each map graphic in each layer; Acquire multiple captured images; Each captured image is subjected to edge detection and connected to form multiple edge patterns corresponding to each captured image; Each edge graphic is matched with each spatial representation map to form multiple graphic matching pairs. Based on the multiple graphic matching pairs, a graphic pixel-level transformation matrix is ​​calculated, and each captured image is transformed according to the graphic pixel-level transformation matrix to obtain the spatial position of each pixel in each captured image, thereby generating a pixel spatial position table set. Dynamic image feature matching is performed on the initial captured image and each non-initial captured image among the plurality of captured images to obtain the geometric transformation relationship and pixel correspondence relationship of each non-initial captured image relative to the initial captured image. Based on the geometric transformation relationship and the pixel correspondence relationship, the spatial matrix of each non-initial captured image is obtained. Obtain the current feature point set of the current real-time image, and determine whether the current real-time image matches the non-initial captured image corresponding to the spatial matrix based on the current feature point set and each spatial matrix. If they match, search the pixel spatial location table corresponding to the non-initial captured image in the pixel spatial location table set. Obtain the positions of multiple wheel points for each vehicle in the current real-time image; Based on the pixel spatial location table, find the spatial coordinate position corresponding to each wheel point position to obtain the spatial position of each vehicle at the current moment. Based on the spatial location of the same vehicle at various times, the trajectory information of each vehicle is obtained.

2. The vehicle positioning and tracking method based on machine vision and mapping according to claim 1, characterized in that, After obtaining the trajectory information of the same vehicle based on its spatial location at various times, the method further includes: Obtain the cross-device trajectory information for each cross-device vehicle corresponding to the cross-device; Determine whether the cross-device vehicle and the vehicle are the same target; If the cross-device vehicle and the vehicle are the same target, then the cross-device trajectory information and the trajectory information are fused based on time to form cross-device vehicle tracking information.

3. The vehicle positioning and tracking method based on machine vision and mapping according to claim 1, characterized in that, The process involves matching each edge graphic with each spatial representation map to form multiple graphic matching pairs, calculating a pixel-level transformation matrix based on the multiple graphic matching pairs, and transforming each captured image according to the pixel-level transformation matrix to obtain the spatial position of each pixel in each captured image, generating a pixel spatial position table set, including: For each of the graphic matching pairs, the edge graphic and the spatial representation map are triangulated to form multiple graphic triangle pairs; Perform an affine transformation on each pair of triangles in the graphic to obtain the pixel-level transformation matrix of the graphic. For each of the graphic matching pairs, edge pixel interpolation is performed on the edge graphics to obtain the paired complete edge graphics.

4. The vehicle positioning and tracking method based on machine vision and mapping according to claim 3, characterized in that, The method further includes: The unpaired edge graphics and the unpaired spatial representation graphics are triangularly split to form multiple unpaired triangular graphics. Perform edge pixel interpolation on each of the unpaired edge graphics to obtain unpaired complete edge graphics; The paired complete edge graphics, the paired spatial representation map, the unpaired complete edge graphics, and the unpaired spatial representation map are added together to obtain a complete image. The complete image is transformed according to the pixel-level transformation matrix to obtain the complete spatial position of each pixel, and the pixel spatial position table is generated.

5. The vehicle positioning and tracking method based on machine vision and mapping according to claim 1, characterized in that, The step of performing dynamic image feature matching between the initial captured image and each non-initial captured image among the plurality of captured images to obtain the geometric transformation relationship and pixel correspondence relationship of each non-initial captured image relative to the initial captured image includes: The initial captured image is acquired, and features are extracted from the initial captured image to obtain an initial image feature point set; Multiple non-initial images are acquired, and feature points are extracted from each non-initial image to obtain multiple non-initial image feature point sets; Match each of the non-initial image feature point sets with the feature points in the initial image feature point set to obtain multiple feature point matching pairs; Based on the multiple feature point matching pairs, the geometric transformation relationship between the initial captured image and each of the non-initial captured images is obtained; The initial captured image and each of the non-initial captured images are respectively divided into grid regions to form initial grid regions and non-initial grid regions; Obtain the grid-level matching relationship between the initial grid region and the same grid position in the non-initial grid region; The pixel relationships in each grid region are calculated based on bilinear interpolation.

6. The vehicle positioning and tracking method based on machine vision and mapping according to claim 1, characterized in that, The step of determining whether the current real-time image matches the non-initial captured image corresponding to the spatial matrix based on the current feature point set and each spatial matrix, and if they match, searching the pixel spatial location table corresponding to the non-initial captured image in the pixel spatial location table set, includes: If a match is found, the matching value between the current real-time image and the non-initial captured image is calculated. If the matching value is the best match, then the pixel spatial location table corresponding to the non-initial captured image is searched in the pixel spatial location table set.

7. A vehicle positioning and tracking system based on machine vision and mapping, characterized in that, include: The spatial information acquisition module is used to acquire road map information and, based on the road map information, obtain a spatial representation map corresponding to each map graphic in each layer; The image acquisition module is used to acquire multiple captured images; An edge shaping module is used to perform edge detection and edge connection shaping on each captured image to obtain multiple edge graphics corresponding to each captured image; The pixel space transformation module is used to match each edge graphic with each spatial representation map to form multiple graphic matching pairs, calculate the graphic pixel-level transformation matrix based on the multiple graphic matching pairs, and transform each captured image based on the graphic pixel-level transformation matrix to obtain the spatial position of each pixel of each captured image and generate a pixel spatial position table set. The image feature matching module is used to perform dynamic image feature matching between the initial captured image and each non-initial captured image among the plurality of captured images, to obtain the geometric transformation relationship and pixel correspondence relationship of each non-initial captured image relative to the initial captured image, and to obtain the spatial matrix of each non-initial captured image based on the geometric transformation relationship and the pixel correspondence relationship. The jitter correction module is used to obtain the current feature point set of the current real-time image, and determine whether the current real-time image matches the non-initial captured image corresponding to the spatial matrix based on the current feature point set and each spatial matrix. If they match, the module searches the pixel spatial location table corresponding to the non-initial captured image in the pixel spatial location table set. The vehicle positioning module is used to obtain the positions of multiple wheel points of each vehicle in the current real-time image; The spatial location lookup module is used to look up the spatial coordinates corresponding to each wheel point position according to the pixel spatial location table, so as to obtain the spatial location of each vehicle at the current moment. The trajectory generation module is used to obtain the trajectory information of each vehicle based on the spatial position of the same vehicle at various times.

8. The vehicle positioning and tracking system based on machine vision and mapping according to claim 7, characterized in that, The system also includes: The cross-device trajectory information acquisition module is used to acquire the cross-device trajectory information of each cross-device vehicle corresponding to the cross-device. The same target determination module is used to determine whether the cross-device vehicle and the vehicle are the same target; The cross-device fusion module is used to fuse the cross-device trajectory information with the trajectory information based on time if the cross-device vehicle and the vehicle are the same target, thereby forming cross-device vehicle tracking information.

9. The vehicle positioning and tracking system based on machine vision and mapping according to claim 7, characterized in that, The pixel space conversion module includes: The graphic triangle pair acquisition module is used to perform triangulation on the edge graphic and the spatial representation map in each graphic matching pair to form multiple graphic triangle pairs; An affine transformation module is used to perform affine transformations on each pair of triangles in the graphic to obtain the pixel-level transformation matrix of the graphic. The pixel interpolation module is used to perform edge pixel interpolation on the edge graphics of each of the graphic matching pairs to obtain the paired complete edge graphics.

10. The vehicle positioning and tracking system based on machine vision and mapping according to claim 9, characterized in that, The system also includes: The unpaired graphic acquisition module is used to triangulate the unpaired edge graphic and the unpaired spatial representation map into multiple unpaired triangular graphics. The unpaired pixel interpolation module is used to perform edge pixel interpolation on each of the unpaired edge graphics to obtain unpaired complete edge graphics; An edge correction module is used to add the paired complete edge graphics, the paired spatial representation map, the unpaired complete edge graphics, and the unpaired spatial representation map together to obtain a complete image; The pixel spatial location table generation module is used to transform the complete image according to the graphic pixel-level transformation matrix to obtain the spatial location of each pixel in the complete image and generate the pixel spatial location table set.

11. The vehicle positioning and tracking system based on machine vision and mapping according to claim 7, characterized in that, The image feature matching module includes: An initial feature acquisition module is used to acquire the initial captured image and extract features from the initial captured image to obtain an initial image feature point set; The non-initial feature acquisition module is used to acquire multiple non-initial captured images and extract feature points from each non-initial captured image to obtain multiple non-initial image feature point sets. The feature point matching module is used to match each of the non-initial image feature point sets with the feature points in the initial image feature point set to obtain multiple feature point matching pairs; The geometric transformation calculation module is used to obtain the geometric transformation relationship between the initial captured image and each of the non-initial captured images based on the multiple feature point matching pairs; The grid division module is used to divide the initial captured image and each of the non-initial captured images into grids to form an initial grid region and a non-initial grid region. The grid matching relationship acquisition module is used to acquire the grid-level matching relationship between the same grid position in the initial grid region and the non-initial grid region; The pixel relationship calculation module is used to calculate the pixel relationship corresponding to each grid region in the grid-level matching relationship based on bilinear interpolation.

12. The vehicle positioning and tracking system based on machine vision and mapping according to claim 7, characterized in that, The jitter correction module includes: The matching value calculation module is used to calculate the matching value between the current real-time image and the non-initial captured image if a match is found. The search module is used to search the pixel spatial location table corresponding to the non-initial captured image in the pixel spatial location table set if the matching value is the best match.

Images