A method and device for constructing aerial rectangular elements of a map

Abstract

The present application provides a method and apparatus for constructing an aerial rectangular element of a map. By collecting a vehicle to obtain target data, calculating the gradient for each frame of video data, extracting the edges of the target rectangle, and performing triangulation to calculate the initial value of the spatial rectangle. Then, an optimization algorithm is used to optimize and adjust the target spatial rectangle and the image gradient. Finally, an accurate spatial rectangular object is constructed. On the one hand, it utilizes the edge features of the rectangular-like object for optimization and uses the representation of the same whole, making it easier to successfully construct aerial elements with smaller errors and applicable to the construction of rectangular-like objects with rounded corners without clear corner points. On the other hand, the application directly targets the rectangular object, represents it as a whole, and enables it to participate in the optimization and construction process to directly construct the vector representation of the aerial element. In addition, the present application is based on consumer-grade visual sensor data, uses a single consumer-grade visual sensor, and is applicable to low-cost crowdsourcing map construction.

Description

Technical Field

[0001] This application belongs to the field of intelligent unmanned vehicle autonomous driving, and more specifically relates to the field of map construction technology for driverless vehicles. Background Art

[0002] In the field of autonomous driving, maps and positioning are important basic modules in the autonomous driving system. As one of the key capabilities to achieve autonomous driving, high-precision maps serve as an effective supplement to existing sensors for autonomous vehicles, providing more reliable perception. And aerial map elements are an important part of high-precision maps. Aerial map elements refer to non-ground elements that high-precision maps are concerned with, such as aerial signs, traffic lights, and height limit bars.

[0003] The following existing technologies disclose some methods for representing and constructing aerial map elements, but there are still certain defects. For example, the patent application document with the document number CN 114972121A discloses a method for processing the corners of a square road sign. By restoring the spatial positions of the four corners of the road sign, this method proposes to construct a square road sign by using three of the points to construct a plane and projecting the fourth point onto the plane. This method requires stable extraction of the four corners of the square road sign. However, in image vision, due to perspective reasons, aerial rectangular elements often blend in front of complex and irregular backgrounds, and their corners are often difficult to extract, and the extraction of corners is very unstable, resulting in a large error in the constructed road sign.

[0004] Another example is that the patent document with the document number CN 114170366A discloses a three-dimensional reconstruction method based on the fusion of point and line features. This method integrates the point triangulation technology in traditional SLAM into the line feature triangulation technology to three-dimensionally reconstruct the point and line features in space. However, this method constructs low-level features (points, lines) in space, and further work is required to extract corresponding objects from these features.

[0005] For another example, the patent document with the document number CN 114413881A discloses a method for constructing a high-precision vector map. This method uses lidar and cameras in cooperation to construct a high-precision vector map, which has relatively high requirements for the configured equipment. Summary of the Invention

[0006] Aiming at the deficiencies of the existing technology, this application proposes a method and device for constructing aerial rectangular elements of a high-precision map.

[0007] The technical solution of this application is as follows:

[0008] In the first aspect, this application proposes a method for constructing aerial rectangular elements of a map. The method includes the following major steps:

[0009] Obtain target data: including monocular front-view camera video data and vehicle driving trajectory data.

[0010] Calculate the initial value of the spatial rectangle: By calculating the gradient for each frame of the video data, extracting the edges of the target rectangle, and performing triangulation to calculate the initial value of the spatial rectangle.

[0011] Construct a spatial rectangular object: Use an optimization algorithm to optimize and adjust the target spatial rectangle and image gradient, construct an accurate spatial rectangular object, and represent the spatial rectangle as a six-dimensional vector, which includes the angle θ between the projection line of the spatial rectangle on the ground plane and the x-axis of the global coordinate system, the coordinates (x, y, z) of the center point of the rectangle, the half-width w of the rectangle, and the half-height h of the rectangle.

[0012] Here, the optimization algorithm is to construct a non-linear optimization problem based on the state variables represented in this way, adjust the numerical values of the state variables starting from the initial values, and minimize the designed objective function to obtain more accurate numerical values of the state variables.

[0013] According to an embodiment of the present application, the obtaining of the target data includes monocular front-view camera video data, vehicle driving trajectory data, and the time stamp corresponding to the data acquisition.

[0014] According to an embodiment of the present application, the calculation of the initial value of the spatial rectangle specifically includes the following steps:

[0015] Undistort the target area on the image, calculate the canny edges, and extract the four edge lines corresponding to the top, bottom, left, and right of the rectangle.

[0016] Perform inter-frame target tracking on the successive frames on the time line. Common inter-frame target tracking includes Kalman filtering, Hungarian matching, and optical flow method.

[0017] Select two frames with the largest perspective change from the frames on the time line that are successfully tracked, and triangulate the edge lines to obtain four straight lines in space.

[0018] Finally, calculate the initial value of the spatial rectangle using the four straight lines.

[0019] According to an embodiment of the present application, the optical flow method is used for tracking the successive frames on the time line. If the points on the straight line in the previous frame are tracked by optical flow and are within the set threshold distance from the corresponding straight line in the next image, they are inliers. If the proportion of inliers among the total tracked points meets the requirements, it is considered that this straight line is successfully tracked.

[0020] According to an embodiment of the present application, the triangulation of the edge lines is to calculate the corresponding observation planes P1 and P2 respectively using the straight lines on the two images and the position of the camera optical center. The two planes intersect to obtain the corresponding spatial straight line L([n, d]). The calculation formula is:

[0021]

[0022] According to an embodiment of the present application, when calculating the initial value of the spatial rectangle using four straight lines, the least-squares solutions of the intersections of two adjacent straight lines among the four straight lines are calculated respectively, and an equation is constructed:

[0023]

[0024] The least-squares solutions are obtained by QR decomposition to get four points p1, p2, p3, p4. Then the rectangle position is (p1 + p2 + p3 + p4) / 4, the rectangle direction θ is (atan((y2 - y1) / (x2 - x1)) + atan((y4 - y3) / (x4 - x3))) / 2, the rectangle half-width is (||x2 - x1|| + ||x4 - x3||) / 2, and the rectangle half-height is (||y2 - y1|| + ||y4 - y3||) / 2.

[0025] According to an embodiment of the present application, before undistorting the target area on the image, time synchronization is also included, that is, using the data timestamp to calculate the vehicle position when each frame of the video is captured, including:

[0026] Find the vehicle driving trajectory data with the closest time difference before and after image capture respectively;

[0027] Interpolate the two pieces of data to calculate the position and attitude at the time of image capture

[0028] p = p1*(t2 - t) / (t2 - t1) + p2*(t - t1) / (t2 - t1)

[0029] q = (q2*q1^-1)^((t - t1) / (t2 - t1))*q1.

[0030] According to an embodiment of the present application, the construction of the spatial rectangular object is to accurately adjust the corresponding rectangle initial value by constructing a graph optimization problem, including:

[0031] Perform DT transformation on the canny edge of the image to obtain the distance map DistanceMap of the canny edge;

[0032] Uniformly sample each side of the rectangle;

[0033] Construct a graph optimization problem, where the residual of the target to be optimized is the numerical value of the sampling point projected onto the corresponding distance map DistanceMap of the image using the camera position and attitude p, q.

[0034] In a second aspect of the present application, there is also provided a device for representing and constructing an aerial rectangular element of a map, which is characterized by including:

[0035] Target data acquisition module.

[0036] Spatial rectangle initial value calculation module, which is used to calculate the gradient for each frame of video data, extract the edges of the target rectangle, and perform triangulation to calculate the initial value of the spatial rectangle.

[0037] Spatial rectangle object construction module, which is used to optimize and adjust the target spatial rectangle and image gradient using an optimization algorithm, construct an accurate spatial rectangle object, and represent the spatial rectangle as a six-dimensional vector, including the angle θ between the projection line of the spatial rectangle on the ground plane and the x-axis of the global coordinate system, the coordinates (x, y, z) of the center point of the rectangle, the half-width w of the rectangle, and the half-height h of the rectangle.

[0038] The device uses the above modules to implement the steps of the method for representing and constructing the aerial rectangle elements of the map.

[0039] As can be seen from the above technical solutions, in this application, target data is acquired by collecting vehicles, the gradient is calculated for each frame of video data, the edges of the target rectangle are extracted, and triangulation is performed to calculate the initial value of the spatial rectangle. Then, an optimization algorithm is used to optimize and adjust the target spatial rectangle and image gradient, and finally an accurate spatial rectangle object is constructed. On the one hand, it uses the edge features of the rectangular object for optimization and uses the same overall representation, making it easier to successfully construct aerial elements, with smaller errors, and is applicable to the construction of rounded rectangular objects without clear corner points. On the other hand, this application directly targets the rectangular object, represents it as a whole, and enables it to participate in the optimization and construction process to directly construct the vector representation of the aerial elements. In addition, this application is based on consumer-grade visual sensor data, uses a single consumer-grade visual sensor, and is applicable to low-cost crowdsourced map construction.

[0040] It can be seen that this application has the advantages of easier construction, smaller errors, and lower costs. Description of the Drawings

[0041] Figure 1 Is the flowchart of the construction method of this application;

[0042] Figure 2 Is the canny edge and distanmap;

[0043] Figure 3 Is the schematic diagram of the spatial rectangle back-projected onto the image;

[0044] Figure 4 Is the visualization effect diagram of the spatial rectangle in the map. Detailed Implementation Manner

[0045] To enable those skilled in the art to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described herein based on the accompanying drawings. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific implementation manners, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0046] It should be noted that the illustrations provided in the following embodiments only schematically illustrate the basic concept of this application. Therefore, only the modules related to this application are shown in the drawings, rather than being drawn according to the number, shape, and size of the modules in actual implementation. The type, quantity, and proportion of each module in actual implementation can be arbitrarily changed, and the module layout type may also be more complex.

[0047] See Figure 1 , in one embodiment of this application, a method for constructing a map aerial rectangular element is provided, and its specific process steps are described in detail through the following content:

[0048] Step 1: Obtain target data:

[0049] In this embodiment, the target data includes monocular front-view camera video data, vehicle driving trajectory data, and the time stamp corresponding to the data acquisition. The trajectory data is the position and orientation of the vehicle at different times. The monocular front-view camera is a consumer-grade vision sensor, which reduces the implementation cost of this method.

[0050] Step 2: Time synchronization:

[0051] For the above target data collected, first, use the data time stamp to calculate the vehicle position when each frame of the video is captured. The specific method is as follows:

[0052] 1) Find the vehicle driving trajectory data with the closest time difference before and after the image capture respectively.

[0053] 2) Interpolate the two vehicle driving trajectory data to calculate the position and attitude when the image is captured

[0054] p = p1 * (t2 - t) / (t2 - t1) + p2 * (t - t1) / (t2 - t1)

[0055] q = (q2 * q1^-1)^((t - t1) / (t2 - t1)) * q1.

[0056] In the above formula, p is the position (3D vector) in the trajectory data, q is the direction (quaternion) in the trajectory data, and t is the timestamp of the collected data. Those with subscript 1 are the previous data, subscript 2 are the subsequent data, and those without subscript are the data at the current time (interpolated data).

[0057] Step 3: Calculate the edges:

[0058] Undistort the target area on the image, calculate the Canny edges, and extract the four edge lines corresponding to the top, bottom, left, and right of the rectangle, as Figure 2 shown.

[0059] Here, the target area is determined by the semantic map output by the perception module, and the perception module usually uses deep learning semantic segmentation for calculation.

[0060] Here, the Canny edges are the pixel gradient map of the image calculated using the Canny edge detection algorithm in the open-source computer vision library opencv( Figure 2 shown). The basic principle of the Canny edge detection algorithm is to calculate the pixel gradient at each pixel position one by one by taking the difference, and the pixels with gradients exceeding the threshold may be on the object edge line. Among them, according to the calculation method, it can be divided into the pixel gradient in the x direction (taking the difference between adjacent pixels in the x direction) and the pixel gradient in the y direction (taking the difference between adjacent pixels in the y direction).

[0061] Step 4: Perform tracking

[0062] Track the frames in sequence in time. In this embodiment, the optical flow method is used for tracking. If the points on the straight line in the previous frame are tracked by optical flow and are within a set threshold (for example, this threshold is set to 5 pixels) of the corresponding straight line in the subsequent image, they are inliers. If the proportion of inliers among the total tracked points meets the requirement (generally greater than 80%), it is considered that the straight line tracking is successful.

[0063] Step 5: Triangulate the edge lines

[0064] Select the two frames with the largest change in viewing angle from the frames with successful tracking on the time line, and triangulate the four edge lines respectively to obtain four straight lines in space.

[0065] In one embodiment, the method for triangulating the edge lines can be: calculate the corresponding observation planes P1 and P2 respectively using the straight lines on the two frames of images and the position of the camera optical center, and the two planes intersect to obtain the corresponding space straight line L([n, d]). The calculation formula is:

[0066]

[0067] where: L is the representation of the straight line in 3D space (Plücker matrix, 4×4 matrix)

[0068] n is the normal vector of the straight line that forms a plane with the origin, a 3D vector

[0069] d is the direction vector of the straight line, a 3D vector

[0070] P1 and P2 (P in capital) are two planes, 4D vectors.

[0071] Step 6: Calculate the initial value of the spatial rectangle using four straight lines.

[0072] Specifically, calculate the least squares solution of the intersection points of two adjacent straight lines among the four straight lines respectively, and construct the equation:

[0073]

[0074] Where:

[0075] d1 and d2 are the direction vectors of the two straight lines respectively;

[0076] n1 and n2 are the normal vectors of the plane formed by each straight line and the origin respectively;

[0077] p (p in lowercase) is the point coordinate to be solved (3D vector).

[0078] Solve the least squares solution using QR decomposition to obtain four points p1, p2, p3, p4. Then the position of the rectangle is (p1 + p2 + p3 + p4) / 4, the direction θ of the rectangle is (atan((y2 - y1) / (x2 - x1)) + atan((y4 - y3) / (x4 - x3))) / 2, the half width of the rectangle is (||x2 - x1|| + ||x4 - x3||) / 2, and the half height of the rectangle is (||y2 - y1|| + ||y4 - y3||) / 2.

[0079] Where, p1, p2, p3, p4 (p in lowercase) are the coordinates of the four vertices of the rectangle (3D vectors, x, y, z) respectively, and the order of the four vertices is top left, top right, bottom left, bottom right. x is the abscissa of the rectangle vertex, and its subscript corresponds to the vertex subscript. y is the ordinate of the rectangle vertex, and its subscript corresponds to the vertex subscript

[0080] Step 7: Construct a graph optimization problem to precisely adjust the corresponding rectangle initial value. Specifically include:

[0081] 1) Perform DT transformation on the canny edge of the image to obtain the DistanceMap graph of the canny edge. See Figure 2 In the figure, the left 1 is the target area of the original image, the left 2 is the pixel gradient graph in the x-axis direction, the left 3 is the pixel gradient graph in the y-axis direction, the left 4 is the sum of the pixel gradients in the x and y axes, and the right 1 is the result after performing DT transformation on the edge graph (the gray level of each pixel represents the distance from the nearest edge).

[0082] 2) Uniformly sample each side of the rectangle, with a certain number of sampling points on each side. For example, 20 points on each side. Other numbers can be used. The more points, the greater the computational amount and the more accurate the result.

[0083] 3) Construct a graph optimization problem, where the residual of the target to be optimized is the numerical value of the sampling points projected onto the corresponding DistanceMap graph in the image using the camera position and attitude p, q.

[0084] In an embodiment of the present application, the specific optimization algorithm is that for each frame of the image capturing the target, the projection of the rectangle on the image is calculated from the six-dimensional vector of the rectangle using the camera imaging principle; the objective function is the sum of the squares of the distances from the sampling points on each frame of the image to the projected sides of the rectangle, and the distance is the pixel value at the corresponding position on the Distanced1Map graph.

[0085] The goal of the optimization algorithm is to adjust the state quantity value of the rectangle to minimize the objective function value, and the specific optimization method is the Gauss-Newton method.

[0086] Specific calculation method:

[0087] The rectangle is: [θ, x, y, z, w / 2, h / 2]

[0088] Then the straight lines of the four sides in the rectangle coordinate system are respectively:

[0089] Top: [n = [0, 0, h / 2], d = [1, 0, 0]]

[0090] Bottom: [n = [0, 0, h / 2], d = [-1, 0, 0]]

[0091] Left: [n = [0, 0, w / 2], d = [0, -1, 0]]

[0092] Right: [n = [0, 0, w / 2], d = [0, 1, 0]]

[0093] The calculation method for converting the straight line coordinate system is as follows:

[0094]

[0095] Among them

[0096]

[0097] In the formula:

[0098] n and d are the normal vector and direction vector in the straight line representation respectively, and the subscript r represents the straight line in the rect coordinate system (with the center point of the rectangle as the origin)

[0099] L wis a spatial straight line, a 6D vector, and the subscript w indicates that the straight line is in the world coordinate system

[0100] T wr is the transformation matrix from the world coordinate system to the rect coordinate system, a 6×6 matrix

[0101] R wr is the rotation matrix from the world coordinate system to the rect coordinate system, a 3×3 matrix

[0102] t wr is the rotation matrix from the world coordinate system to the rect coordinate system, a 3D vector

[0103] θ is the θ in the rectangle parameters

[0104] x, y, and z are the x, y, and z coordinates of the center point of the rectangle parameters respectively.

[0105] The calculation method of projecting the straight line onto the image is as follows:

[0106]

[0107] Among them, fx, fy, cx, and cy are the internal parameters of the camera, and the above l is the straight line equation on the image.

[0108] Above, the whole process of constructing the aerial rectangular element of the map is completed. The obtained spatial rectangle is represented as a 6D vector, which includes the angle θ between the projection line of the spatial rectangle on the ground plane and the x-axis of the global coordinate system, the center point coordinates (x, y, z) of the rectangle, the half-width w of the rectangle, and the half-height h of the rectangle.

[0109] The back-projection of this spatial rectangle onto the image is as Figure 3 shown, and the visualization effect of this spatial rectangle in the map is as Figure 4 shown.

[0110] Specifically, according to the optimized values, using the aforementioned projection method (starting from the six-dimensional parameters of the rectangle, calculating the straight lines of the four sides of the top, bottom, left, and right, performing coordinate transformation on the straight lines, projecting the straight lines onto the image, and drawing the corresponding straight lines onto the original image to obtain Figure 3 , and thus the accuracy of the optimized six-dimensional values can be visually estimated ( Figure 3 is an example of one frame).

[0111] And the four sides of the rectangle in the map coordinate system obtained by calculating the optimized values (starting from the six-dimensional parameters of the rectangle, calculating the straight lines of the four sides of the top, bottom, left, and right, performing coordinate transformation, transforming to the map coordinate system, and thus obtaining the rectangle in the map, and observing with the visualization tool to obtain Figure 4 , Figure 4 The top left, top right, and bottom left are the effects of the rectangle on the map from different perspectives, Figure 4The picture of the traffic lights taken by the camera is shown in the lower right corner.

[0112] It should be understood that for those of ordinary skill in the art, various forms of processes shown above can be used, steps can be reordered, added or deleted. For example, the steps described in this application can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this application can be achieved, and this application does not impose any restrictions here.

[0113] The above-described embodiments are only specific implementation manners of this application, which are used to illustrate the technical solutions of this application, rather than to limit it. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: Any person skilled in the art within the technical scope disclosed in this application can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or make equivalent replacements for some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A method for constructing an aerial rectangular element of a map, characterized in that, The method includes: Obtaining target data: including video data and vehicle driving trajectory data; Calculating the initial value of the spatial rectangle: calculating the gradient for each frame of the video data, extracting the edges of the target rectangle, and performing triangulation to calculate the initial value of the spatial rectangle; specifically including: undistorting the target area on the image, calculating the canny edges, and extracting the four edge lines corresponding to the top, bottom, left, and right of the rectangle; performing inter-frame target tracking on the frames in sequence on the time line, and the inter-frame target tracking adopts Kalman filtering, Hungarian matching or optical flow method; selecting the two frames with the largest perspective change from the frames successfully tracked on the time line, and triangulating the edge lines to obtain four straight lines in space; finally, calculating the initial value of the spatial rectangle using the four straight lines; Constructing a spatial rectangular object: optimizing and adjusting the target spatial rectangle and the image gradient to construct an accurate spatial rectangular object, and representing the spatial rectangle as a six-dimensional vector, which includes the angle θ between the projection line of the spatial rectangle on the ground plane and the x-axis of the global coordinate system, the center point coordinates (x, y, z) of the rectangle, the half-width w of the rectangle, and the half-height h of the rectangle; Specifically, an optimization algorithm is used to optimize and adjust the target spatial rectangle and the image gradient to construct an accurate spatial rectangular object; the optimization algorithm is that for each frame of the image capturing the target, the projection of the rectangle on the image is calculated from the six-dimensional vector of the rectangle using the camera imaging principle; the objective function is the sum of the squares of the distances from the sampled points on each frame of the image to the sides of the rectangle projection, and the distance is the pixel value at the corresponding position on the DistanceMap; the goal of the optimization algorithm is to adjust the state quantity value of the rectangle to minimize the objective function value.

2. The method for constructing a map aerial rectangular element according to claim 1, characterized in that, The obtaining of the target data includes monocular front-view camera video data, vehicle driving trajectory data, and the time stamps corresponding to the data acquisition.

3. The method for constructing a map aerial rectangular element according to claim 1, wherein The inter-frame target tracking adopts the optical flow method. If the points on the straight line in the previous frame of the image are tracked by optical flow and are within the set threshold of the distance from the corresponding straight line in the next image, they are inliers. If the proportion of the inliers in the total tracked points meets the requirements, it is considered that this straight line is successfully tracked.

4. The method for constructing the aerial rectangular element of the map according to claim 1, wherein, For the triangulation of the edge lines, the corresponding observation planes P1 and P2 are calculated respectively using the straight lines on the two images and the position of the camera optical center. The two planes intersect to obtain the corresponding spatial straight line L([n, d]), and the calculation formula: ， where: L is the representation of the straight line in 3D space; n is the normal vector of the plane formed by the straight line and the origin, a 3D vector; d is the direction vector of a straight line, a 3D vector; P1 and P2 are two observation planes, 4D vectors.

5. The method for constructing a map aerial rectangular element according to claim 1, characterized in that, The calculation of the initial value of the spatial rectangle using the four straight lines is to calculate the least squares solution of the intersection of two adjacent straight lines among the four straight lines respectively, and construct the equation: ， Wherein: d 1、 d 2 are the direction vectors of the two straight lines respectively; n 1、 n2 are the normal vectors of the planes formed by each straight line and the origin respectively; p is the point coordinate to be solved; The least squares solution is obtained by QR decomposition to get four points p1, p2, p3, p4. Then the rectangle position is (p1 + p2 + p3 + p4) / 4, the rectangle direction θ is (atan((y2 - y1) / (x2 - x1)) + atan((y4 - y3) / (x4 - x3))) / 2, the rectangle half-width is (||x2 - x1|| + ||x4 - x3||) / 2, and the rectangle half-height is (||y2 - y1|| + ||y4 - y3||) / 2.

6. The method for constructing a rectangular element in the air of a map according to claim 1, characterized in that Before undistorting the target area on the image, it also includes time synchronization, that is, using the data timestamp to calculate the vehicle position when each frame of the video is captured, including: Find the vehicle driving trajectory data with the closest time difference before and after image capture respectively; Interpolate the two pieces of data to calculate the position and attitude at the time of image capture; p = p1*(t2 - t) / (t2 - t1)+p2*(t - t1) / (t2 - t1); q=(q2*q1^-1)^((t - t1) / (t2 - t1))*q1; Where p is the position in the trajectory data, q is the direction in the trajectory data, t is the timestamp of the captured data, the one with subscript 1 is the previous data, the one with subscript 2 is the later data, and the one without subscript is the data at the current time, that is, the interpolated data.

7. The method for constructing a map aerial rectangular element according to claim 1, characterized in that The construction of the spatial rectangular object is to precisely adjust the corresponding rectangular initial value by constructing a graph optimization problem, including: Perform a DT transformation on the canny edge of the image to obtain the distance map DistanceMap of the canny edge; the DT transformation is to calculate the distance between each point and the closest point within the region, and here the region is set to the edge line of the image; Uniformly sample each side of the rectangle; Construct a graph optimization problem, where the residual of the target to be optimized is the numerical value of the sampling point projected onto the corresponding distance map DistanceMap of the image using the camera position and attitude p, q.

8. An apparatus for representing and constructing aerial rectangular elements of a map, characterized in that, Including: Target data acquisition module; Spatial rectangle initial value calculation module, which is used to calculate the gradient for each frame of the video data, extract the edges of the target rectangle, and perform triangulation to calculate the initial value of the spatial rectangle; specifically including: undistorting the target area on the image, calculating the canny edge, and extracting the four edge lines corresponding to the rectangle up, down, left, and right; performing inter-frame target tracking on the frames before and after on the time line, and the inter-frame target tracking uses Kalman filtering, Hungarian matching or optical flow method; selecting two frames with the largest view angle change from the frames with successful tracking on the time line, and triangulating the edge lines to obtain four straight lines in space; finally, calculating the initial value of the spatial rectangle using the four straight lines; Spatial rectangle object construction module, which is used to optimize and adjust the target spatial rectangle and the image gradient using an optimization algorithm to construct an accurate spatial rectangle object, and represent the spatial rectangle as a six-dimensional vector, which includes the angle θ between the projection line of the spatial rectangle on the ground plane and the x-axis of the global coordinate system, the center point coordinates (x, y, z) of the rectangle, the half-width w of the rectangle, and the half-height h of the rectangle; specifically, it is to optimize and adjust the target spatial rectangle and the image gradient using an optimization algorithm to construct an accurate spatial rectangle object; the optimization algorithm is for each frame of the image capturing the target, calculating the projection of the rectangle on the image from the six-dimensional vector of the rectangle using the camera imaging principle; the objective function is the sum of the squares of the distances from the sampling points on each frame of the image to the sides of the rectangle projection, and the distance is the pixel value at the corresponding position on the DistanceMap; the goal of the optimization algorithm is to adjust the state quantity value of the rectangle to make the objective function value reach the minimum; The device uses the above modules to implement the steps of the method for constructing the map aerial rectangle element described in any one of claims 1-7.

Images