A contour fitting method, electronic device, vehicle and storage medium

Abstract

Embodiments of the present application provide a contour fitting method, an electronic device, a vehicle and a storage medium, comprising: determining a set of projection points in a coordinate system according to a set of grounding points of an object in an image domain; determining a starting point and an ending point of a target object in the set of projection points; determining a reference point according to the starting point and the ending point; and determining contour fitting information of the target object according to the starting point, the ending point and the reference point. Through the embodiments of the present application, the contour of the target object can be accurately fitted.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a contour fitting method, an electronic device, a vehicle, and a computer-readable storage medium. Background Technology

[0002] In the field of intelligent driving, for solutions utilizing visual sensors, the accuracy and timeliness of visual perception are crucial for subsequent planning, decision-making, and control. Related technologies involve semantic analysis of objects such as obstacles in the image domain to determine their bounding boxes, which are then processed to define the outlines of the obstacles. However, this approach focuses primarily on object location recognition and has poor perception capabilities for the location and orientation of obstacles, leading to reduced accuracy in fitting the object's contour. Summary of the Invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a contour fitting method, an electronic device, a vehicle, and a computer-readable storage medium that overcome or at least partially solve the above problems.

[0004] To address the above problems, embodiments of the present invention disclose a contour fitting method, comprising:

[0005] The set of projection points in the coordinate system is determined based on the set of ground points of objects in the image domain.

[0006] Determine the start and end points of the target object within the set of projection points;

[0007] A reference point is determined based on the starting point and the ending point;

[0008] The contour fitting information of the target object is determined based on the starting point, the ending point, and the reference point.

[0009] Optionally, determining the reference point based on the starting point and the ending point includes:

[0010] A reference circumference is determined based on the starting point, the ending point, and the straight-line distance between the starting point and the ending point, wherein the straight-line distance between the starting point and the ending point is the diameter of the reference circumference.

[0011] A reference point is determined on the reference circumference.

[0012] Optionally, determining the reference point on the reference circumference includes:

[0013] Determine the target far instance point from the set of projection points;

[0014] Based on the target far-field point and the reference circumference, determine the target far-field location point;

[0015] Determine the target arc based on the target's distant location point;

[0016] Determine a reference point on the target arc.

[0017] Optionally, determining the target far instance point in the set of projection points includes:

[0018] For any candidate instance point in the set of projection points, determine the first straight-line distance between the candidate instance point and the starting point;

[0019] Determine the second straight-line distance between the candidate instance point and the termination point;

[0020] By combining the first straight-line distance and the second straight-line distance, the target distant instance point is determined from the candidate instance points.

[0021] Optionally, determining the target's distant location point based on the target's far-instance point and the reference circumference includes:

[0022] Connect the center of the reference circle with the target farthest point to determine the reference straight line;

[0023] The intersection of the reference line and the reference circle is determined as the target's far-distance location point.

[0024] Optionally, determining the target arc based on the target's distant location point includes:

[0025] On the reference circumference, the arc within a preset range of the target distant position point is determined as the target arc.

[0026] Optionally, determining the reference point on the target arc includes:

[0027] On the target arc, at least one candidate position point is determined based on a preset step size;

[0028] A reference point is selected from the at least one candidate location point.

[0029] Optionally, selecting a reference point from the at least one candidate location point includes:

[0030] For any one of the at least one candidate location points, connect the candidate location point to the starting point to determine the first straight line segment;

[0031] Connect the candidate location points with the termination point to determine the second straight line segment;

[0032] Calculate the distance between the instance points in the projection point set and the first point-line distance between the first line segment and the second line segment;

[0033] A reference point is determined based on the distance value between the first point and the line.

[0034] Optionally, calculating the distance between an instance point in the projection point set and the first point-line distance between the first line segment and the second line segment includes:

[0035] Calculate the first sub-distance value between the instance point in the projection point set and the first line segment;

[0036] Calculate the second sub-distance value between the instance point in the projection point set and the second line segment;

[0037] The first sub-distance value and the second sub-distance value are combined to form the first point-line distance value.

[0038] Optionally, calculating the first sub-distance value between the instance point in the projection point set and the first line segment includes:

[0039] The minimum distance between an instance point in the projection point set and the first line segment is calculated as the first sub-distance value; correspondingly,

[0040] The calculation of the second sub-distance value between the instance points in the projection point set and the second line segment includes:

[0041] The minimum distance between an instance point in the projection point set and the second line segment is calculated as the second sub-distance value.

[0042] Optionally, determining the reference point based on the first point-line distance value includes:

[0043] Determine the minimum distance value from the first point-line distance value;

[0044] The candidate location point corresponding to the minimum distance value is determined as the reference point.

[0045] Optionally, the method further includes:

[0046] Determine the second point-to-line distance value between the target's distant location point and the reference line;

[0047] When the distance value between the second point and the line is not greater than the preset distance threshold, a line fitting is performed based on the starting point and the ending point to determine the contour fitting information of the target object.

[0048] Optionally, determining the contour fitting information of the target object based on the starting point, the ending point, and the reference point includes:

[0049] A right-angled triangle is formed by enclosing the starting point, the ending point, and the reference point.

[0050] The contour fitting information of the target object is determined based on the right-angled triangular frame.

[0051] Optionally, determining the contour fitting information of the target object based on the right-angled triangle bounding box includes:

[0052] Based on the right-angled triangle frame, determine the rectangular frame;

[0053] The rectangle is defined as the contour fitting information of the target object.

[0054] Optionally, determining the set of projection points in the coordinate system based on the set of grounding points of objects in the image domain includes:

[0055] Transform the set of grounding points of objects in the image domain to the vehicle coordinate system to determine the set of projection points in the vehicle coordinate system.

[0056] Optionally, determining the start and end points of the target object in the set of projection points includes:

[0057] Obtain the vehicle's forward direction;

[0058] The first instance point of the target object in the projection point set in the forward direction is determined as the starting point;

[0059] The last instance point of the target object in the set of grounding points in the forward direction is determined as the termination point.

[0060] An electronic device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the contour fitting method as described above.

[0061] A vehicle including the electronic equipment described above.

[0062] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the contour fitting method as described above.

[0063] The embodiments of the present invention have the following advantages:

[0064] This invention, in its embodiments, determines a set of projection points in a coordinate system based on a set of grounding points of objects in the image domain; determines the start and end points of the target object within the set of projection points; determines a reference point based on the start and end points; and determines the contour fitting information of the target object based on the start, end, and reference points. The contour fitting information of the target object is composed of the start, end, and reference points in the set of projection points. This three-point fitting more closely matches the actual contour characteristics, and the orientation of the fitted contour can be determined by the orientation of the three points, enabling adaptation to various different target objects and improving the accuracy of the fitted target object contour. Attached Figure Description

[0065] Figure 1 This is a flowchart illustrating the steps of an embodiment of the contour fitting method of the present invention;

[0066] Figure 2 This is a flowchart illustrating the steps of another embodiment of the contour fitting method of the present invention;

[0067] Figure 3 This is a flowchart illustrating the steps involved in determining the set of grounding points according to the present invention.

[0068] Figure 4 This is a schematic diagram of an example of the grounding point set of the present invention;

[0069] Figure 5 This is a flowchart illustrating the coordinate transformation process of the present invention.

[0070] Figure 6 This is a schematic diagram of the target reference point search of the present invention;

[0071] Figure 7 This is a schematic diagram of the right-angled triangular frame of the present invention;

[0072] Figure 8 This is a schematic diagram of the rectangular frame of the present invention;

[0073] Figure 9 This is a flowchart illustrating the steps of an example contour fitting method of the present invention;

[0074] Figure 10 This is a flowchart of the contour fitting process of the present invention. Detailed Implementation

[0075] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0076] In the field of intelligent driving, the accuracy and timeliness of obstacle visual perception are crucial for subsequent planning, decision-making, and control. Currently, for monocular cameras, related technologies use deep learning models to obtain obstacle detection boxes. Then, based on a series of assumptions such as a level ground surface, the pinhole camera principle is used, combined with camera calibration parameters and odometer information, to calculate the obstacle's position, orientation, and other information. However, this approach has a drawback: the perception results for nearby obstacles, especially the position and orientation of vehicles, are poor. For vehicle obstacles, due to their large size and the limited field of view of panoramic cameras, nearby vehicles are easily truncated, leading to inaccurate detection boxes.

[0077] The accuracy of determining the position and orientation of nearby obstacles is crucial for intelligent driving applications such as automated parking. Therefore, solutions such as LiDAR and BEV (Bird's Eye View) have been explored to address this issue. Taking LiDAR as an example, the vehicle acquires point cloud data of the target obstacle, clusters it, projects it onto the ground, and obtains a two-dimensional plane ground point set under the VCS (Vertical Coordinate System). Based on this point set, the ground projection contour of the nearby vehicle under the VCS is fitted. Among point set-based vehicle contour estimation algorithms, the L-shaped contour fitting algorithm is the most common. The goal of the L-shaped contour fitting algorithm is to obtain the rectangle that best fits the points of each target point cluster. A classic criterion for evaluating the fitting performance is the least squares method, which aims to find two straight lines that minimize the sum of the smaller distances from all points to these two lines. This can be expressed as:

[0078]

[0079] Where, P∪Q={1,2,...,m},c1,c2∈R,0°≤θ<90°,

[0080] xcosθ+ysinθ=c1,

[0081] -xsinθ+ycosθ=c2,

[0082] θ, c1, and c2 represent the relevant parameters of the two perpendicular lines. Due to the combinatorial complexity of the partitioning problem, the above optimization problem is difficult to solve. Therefore, L-shaped contour fitting algorithms generally rely on search-based algorithms to find the most suitable rectangle. The basic idea is to traverse all possible directions of the rectangle. For each direction, a rectangle facing that direction and containing all points of the point set can be found. Then, the distances from all points to the four sides of the rectangle can be obtained, and the points are divided into sets P and Q based on the distances. Then, the sum of errors is calculated. After iterating through all directions and calculating the sum of errors, the direction with the minimum sum of errors can be found, and the corresponding rectangle is taken as the fitting rectangle. It should be noted that although a rectangle is fitted, only two adjacent sides of the rectangle are actually used, so it is still called an L-shaped contour fitting algorithm.

[0083] There are several standards for calculating the sum of errors, the main ones being minimizing the area of ​​the rectangle, maximizing the proximity between a point and an edge, and minimizing the squared error between a point and an edge. Maximizing the proximity between a point and an edge means minimizing the distance between the point and the edge, while minimizing the squared error between a point and an edge means minimizing the square of the distance between the point and the edge.

[0084] However, obtaining the vehicle bounding box based on the above method has the following drawbacks:

[0085] 1. It relies on LiDAR, making it unsuitable for purely vision-based solutions;

[0086] 2. The data points cannot distinguish between vehicle body and non-vehicle body attributes, resulting in a 90-degree uncertainty in orientation;

[0087] 3. The fitted rectangle approximates the minimum bounding rectangle of the point set, rather than the actual outline of the obstacle;

[0088] 4. Angle-based search is inefficient and cannot meet real-time requirements on platforms with demanding computational resources.

[0089] To at least partially solve the above problems, embodiments of the present invention are proposed.

[0090] Reference Figure 1 The diagram illustrates a flowchart of an embodiment of the contour fitting method of the present invention, which specifically includes the following steps:

[0091] Step 101: Determine the set of projection points in the coordinate system based on the set of grounding points of objects in the image domain.

[0092] Images of the vehicle's external environment can be acquired using a visual sensor. The contact points between various objects and the ground in these images, i.e., grounding points, can be identified. The grounding points of all objects can be collected to obtain the set of grounding points of objects in the image domain.

[0093] The set of projection points in the coordinate system can be determined from the set of ground points of objects in the image domain.

[0094] Step 102: Determine the starting point and ending point of the target object in the set of projection points;

[0095] The starting and ending points of the target object's projection on the ground can be determined from the set of grounding points. The starting point is the first grounding projection point of the target object in the set of projection points, and the ending point is the last grounding projection point of the target object in the set of projection points.

[0096] Step 103: Determine a reference point based on the starting point and the ending point;

[0097] The target object can be fitted based on the starting point and the ending point to determine the reference point.

[0098] Step 104: Determine the contour fitting information of the target object based on the starting point, the ending point, and the reference point.

[0099] Connect the starting point, ending point, and reference point to form the outer edge of the contour, and determine the contour fitting information of the target object.

[0100] This invention, in its embodiments, determines a set of projection points in a coordinate system based on a set of grounding points of objects in the image domain; determines the start and end points of the target object within the set of projection points; determines a reference point based on the start and end points; and determines the contour fitting information of the target object based on the start, end, and reference points. The contour fitting information of the target object is composed of the start, end, and reference points in the set of projection points. This three-point fitting more closely matches the actual contour characteristics, and the orientation of the fitted contour can be determined by the orientation of the three points, enabling adaptation to various different target objects and improving the accuracy of the fitted target object contour.

[0101] Reference Figure 2 The diagram illustrates a flowchart of another embodiment of the contour fitting method of the present invention, which specifically includes the following steps:

[0102] Step 201: Obtain the image to be recognized;

[0103] Real-time images can be acquired from a visual sensor as the image to be identified. This visual sensor can be a monocular camera. Types of monocular cameras include, but are not limited to, fisheye cameras, fixed-focus cameras, zoom cameras, and panoramic cameras. A fisheye camera uses a fisheye lens and provides an ultra-wide field of view, typically capturing images within a 180-degree or larger field of view. A fixed-focus camera refers to a camera with a fixed, non-adjustable focal length. A zoom camera is a camera whose zoom effect can be changed by adjusting the focal length. A panoramic camera is a camera capable of capturing a 360-degree panoramic image or video. In one example of this invention, the image to be identified is an image captured by a fisheye camera; that is, the image to be identified is a fisheye image.

[0104] Step 202: Determine the set of projection points in the coordinate system based on the set of grounding points of objects in the image domain;

[0105] Identify the grounding point corresponding to each object in the image domain of the image to be identified, and determine the set of grounding points. Then, convert the grounding points in the set of grounding points into a set of projection points in the corresponding coordinate system.

[0106] The process of identifying a set of grounding points may include: identifying instance points from the image to be identified; segmenting the instance points to determine the set of grounding points.

[0107] The image to be recognized can be preprocessed, and then edge detection can be performed on each object in the image. Feature extraction is performed on the edge parts to identify the instance points of the objects and their projected contact points. Then, these instance points are clustered based on the objects to segment them and determine the set of grounding points for each object.

[0108] For example, when other vehicles are captured during vehicle use, the process of segmenting other vehicles as objects into instance points is as follows: For discontinuous vehicles, segmentation can be achieved by changing the ground wire category. For continuous vehicles, significant non-differentiable changes will occur at the transition points. When distinguishing them, the segmentation of continuous vehicles can be achieved by the change value of the vertical coordinate in the image domain exceeding a set threshold and the distance in the VCS domain exceeding a set threshold.

[0109] Furthermore, since the edges of images captured by different cameras may be affected by factors such as focal length and distortion, instance points can be filtered. The process of identifying the set of grounding points from the image to be identified also includes: acquiring visual angle data; filtering the instance points based on the visual angle data to determine target instance points; and performing segmentation of the instance points based on the target instance points to determine the set of grounding points.

[0110] Visual angle data from a visual sensor, such as the FOV (Field of View) range of a fisheye camera, can be acquired. This visual angle data can be determined based on different visual sensors, and this embodiment of the invention does not impose specific limitations on it. The visual angle data is then used to filter instance points, selecting those within the specified range as target instance points. Subsequent processing uses only these target instance points, performing segmentation of the instance points to determine the grounding point set. Filtering instance points using visual angle data ensures the accurate position of each instance point in the identified grounding point set, thereby guaranteeing the accuracy of the grounding point set. This improves the accuracy of contour recognition when the original data is accurate.

[0111] For example, to ensure the usability of grounding point instances in fisheye images, only instances within a specific range are extracted. Since the installation locations of different fisheye cameras vary, the restricted range differs for different cameras. Considering the imaging characteristics of fisheye cameras, the mapping error of detection points at the edges of the fisheye image to the VCS coordinate system is larger compared to detection points closer to the center of the image. Therefore, this invention uses VCS filtering based on the camera's installation location and the point of contact. The filtering rule is that the angle between the line connecting the VCS and the camera position is within the camera's reliable FOV range, and the distance is less than a set threshold. In practical use, the FOV is set to 150 degrees, and the distance threshold is set to 10 meters.

[0112] To enable those skilled in the art to clearly understand the process of determining the set of grounding points, refer to Figure 3 To illustrate, let's take an example:

[0113] Traverse the visual grounding points along the horizontal axis of the image. Determine if the distance or rate of change between adjacent pixels is greater than a set threshold. If the distance or rate of change between adjacent pixels is greater than the set threshold, obtain a new obstacle instance segmentation subset and mark the start and end points of this instance. If the distance or rate of change between adjacent pixels is not greater than the set threshold, continue traversing the visual grounding points along the horizontal axis of the image. Repeat the above process for all grounding points. After all grounding points have been processed, obtain a set of grounding point subsets after instance segmentation. Use this set as the grounding point set.

[0114] Furthermore, when using a fisheye camera as a visual sensor, the inherent distortion can be eliminated. When the image to be identified is a fisheye image, distortion correction processing is performed on the fisheye image; the step of identifying the set of grounding points from the image to be identified is then performed using the distortion-corrected image. That is, when a fisheye image from a fisheye camera is acquired as the image to be identified, distortion correction processing can be performed to eliminate the distortion, and then the step of identifying the set of grounding points from the image to be identified is performed using the distortion-corrected image. This reduces the positional error of the grounding points in the set of grounding points, improving the accuracy of contour recognition.

[0115] The specific process for distortion correction of fisheye images can be as follows: First, determine the distortion model, such as the pinhole camera model, hyperbolic fisheye model, or perspective projection model. The distortion model is based on the non-linear relationship between image coordinates (pixel coordinates) and actual spatial coordinates. Then, perform inverse mapping based on the distortion correction model. By deriving the true position of each distorted point after distortion correction, inverse mapping can map each pixel in the distorted image to its ideal pixel position. During interpolation, since blank areas may appear in the mapped image, these areas need to be filled using interpolation algorithms (such as bilinear interpolation or cubic spline interpolation) to maintain a smooth transition in the image. After these processes, the distortion-corrected fisheye image is obtained.

[0116] The coordinate system can be the actual vehicle coordinate system being processed. Determining the set of projection points in the coordinate system based on the set of ground points of objects in the image domain includes: transforming the set of ground points of objects in the image domain to the vehicle coordinate system, and then determining the set of projection points in the vehicle coordinate system.

[0117] The coordinates of the grounding points in the grounding point set can be transformed. The pixel coordinates of the grounding point set are transformed into the vehicle coordinate system to determine the set of projection points in the vehicle coordinate system; this transformation is then applied to the actual processing coordinate system, unifying the coordinate system for processing at a single scale, reducing errors and improving accuracy.

[0118] For example, refer to Figure 4 The coordinates of the grounding point set can be transformed from the pixel coordinate system to the actual vehicle coordinate system, and displayed synchronously with the parking space detection results.

[0119] Step 203: Determine the starting point and ending point of the target object in the set of projection points;

[0120] The starting and ending points can be determined from the instance points of the target object in the set of projection points.

[0121] In an optional embodiment of the present invention, determining the starting point and ending point of the target object in the projection point set includes: obtaining the forward direction; determining the first instance point of the grounding point set in the forward direction as the starting point; and determining the last instance point of the grounding point set in the forward direction as the ending point.

[0122] Based on the direction of travel during data collection, the first instance point of the grounding point set in the direction of travel is taken as the starting point, and the last instance point of the grounding point set in the direction of travel is taken as the ending point, thereby determining the direction of the instance points in the grounding point set.

[0123] To enable those skilled in the art to clearly understand the process of determining the set of grounding points, refer to Figure 5 To illustrate, let's take an example:

[0124] In the vehicle coordinate system, the validity of the start and end points is verified. Points with distances greater than a threshold from multiple adjacent points are considered outliers. If a point is invalid, the validity of the start or end point is determined. If the start or end point is invalid, the next adjacent point in the subset of grounding points is obtained as a candidate point, and the validity of the start and end points is further evaluated. If the start or end point is valid, the number of valid points in the subset of grounding points is calculated, and points within a specified range are considered valid. It is then determined whether there are enough valid grounding points. If there are not enough valid grounding points, the instance is discarded. If there are enough valid grounding points, the circumference to be searched is calculated using the start and end points for use in the L-shaped fitting algorithm, i.e., subsequent processing is performed.

[0125] Step 204: Determine a reference circumference based on the starting point, the ending point, and the straight-line distance between the starting point and the ending point, wherein the straight-line distance between the starting point and the ending point is the diameter of the reference circumference;

[0126] A reference circle can be determined based on a starting point and an ending point. This reference circle can be a complete circle or a partial circle; this embodiment of the invention does not impose specific limitations. The straight-line distance between the starting point and the ending point is the diameter of the reference circle; that is, the starting point and the ending point must lie on the reference circle.

[0127] In an optional embodiment of the present invention, determining the reference circumference based on the starting point, the ending point, and the straight-line distance between the starting point and the ending point includes: determining the straight-line distance between the starting point and the ending point; and determining the reference circumference based on the straight-line distance, the starting point, and the ending point.

[0128] First, determine the straight-line distance between the starting and ending points. This distance can be calculated using the formula for the distance between two points. Then, combine this straight-line distance with the starting and ending points to determine the fitted reference circle.

[0129] Specifically, determining the reference circumference based on the straight-line distance, the starting point, and the ending point includes: determining the straight-line distance as a diameter; determining the center coordinates based on the diameter, the starting point, and the ending point; and determining the reference circumference based on the center coordinates and the diameter.

[0130] The straight-line distance can be defined as the diameter of the circle to which the reference circumference belongs. Based on the diameter, the starting point, and the ending point, the coordinates of the circle's center can be determined. Then, using the center coordinates and the diameter, the reference circumference passing through the starting and ending points can be determined.

[0131] Furthermore, determining the center coordinates based on the diameter, the starting point, and the ending point includes: determining the radius based on the diameter; and determining the center coordinates based on a preset center formula, combined with the radius, the starting point, and the ending point.

[0132] Half the diameter can be used as the radius. Then, based on a pre-defined center formula, the center coordinates are determined using the radius, starting point, and ending point. Finally, a reference arc is determined based on the center coordinates and the radius. That is, the corresponding calculation formula can be expressed as: radius r is:

[0133]

[0134] The expression for circle C, to which the reference arc belongs, is:

[0135] (x-x0) 2 +(y-y0) 2 =r 2

[0136] x0=(x begin +x end ) / 2

[0137] y0=(y begin +y end ) / 2

[0138] Among them, (x begin ,y begin (x) represents the x and y coordinates of the starting point. end ,y end (x0, y0) represents the x and y coordinates of the endpoint, r represents the radius of the circle, and (x0, y0) represents the coordinates of the center of the circle.

[0139] Step 205: Determine a reference point on the reference arc;

[0140] You can search on the reference arc to determine the best corner point as the reference point.

[0141] In an optional embodiment of the present invention, determining the target reference point on the reference arc includes:

[0142] Sub-step S2051: Determine the target far instance point from the set of projection points;

[0143] First, the instance point that is farthest from the starting point and the ending point in the set of projection points is determined as the target far instance point.

[0144] The step of determining the target far instance point in the set of projection points includes: for any candidate instance point in the set of projection points, determining a first straight-line distance between the candidate instance point and the starting point; determining a second straight-line distance between the candidate instance point and the ending point; and combining the first straight-line distance and the second straight-line distance to determine the target far instance point from the candidate instance points.

[0145] Any candidate instance point in the set of projection points can be used as the calculation location point. The first straight-line distance between the candidate instance point and the starting point can be determined based on the distance calculation formula between two points. The first straight-line distance is the straight-line distance between the candidate instance point and the starting point. Then, a corresponding calculation method is used to determine the second straight-line distance between the candidate instance point and the ending point. The second straight-line distance is the straight-line distance between the candidate instance point and the ending point. Based on the above calculation method, the first and second straight-line distances for each candidate instance point are calculated.

[0146] Then, the first straight-line distance and the second straight-line distance are combined. For example, the first straight-line distance and the second straight-line distance can be added together, and the candidate instance point with the largest sum of distances is determined as the target far instance point.

[0147] Sub-step S2052: Determine the target's far-distance location point based on the target's far-distance instance point and the reference circumference;

[0148] Determine the target's far distance location point on the reference circle, corresponding to the target's farthest point.

[0149] The step of determining the target far-distance location point based on the target far-distance instance point and the reference circumference includes: connecting the center of the reference circumference with the target far-distance instance point to determine a reference straight line; and determining the intersection of the reference straight line and the reference circumference as the target far-distance location point.

[0150] Connect the center of the reference circle to the farthest point of the target to form a reference line. Then extend the reference line and determine the intersection of the extended reference line and the reference circle as the farthest point of the target.

[0151] Sub-step S2053: Determine the target arc based on the target's far-distance location point;

[0152] Since the optimal reference point is located near the target's distant location, in order to improve processing efficiency, the search range can be narrowed down on the reference circumference based on the target's distant location to form a target arc. This avoids searching the entire circumference during subsequent searches, improving search efficiency and enabling the rapid determination of the optimal reference point.

[0153] Specifically, determining the target arc based on the target far-distance position point includes: determining an arc within a preset range of the target far-distance position point as the target arc on the reference circumference.

[0154] The target arc can be defined as a certain range near the target's distant location, that is, a certain range of the target's distant location on the reference circle.

[0155] In an optional embodiment of the present invention, determining an arc within a preset range of the target distant location point as the target arc on the reference circumference includes: obtaining a preset neighborhood range value; and determining the target arc on the reference circumference, using the target distant location point as the midpoint and combining it with the preset neighborhood range value. The preset neighborhood range value can be obtained, and the search range near the target distant location point can be determined based on this value. The target distant location point can be used as the midpoint, and the target arc can be determined by combining it with the neighborhood range value to narrow the search range and improve processing efficiency.

[0156] Sub-step S2054: Determine a reference point on the target arc.

[0157] The target reference point can be determined on the target arc based on neighborhood search, which can handle the occurrence of compatible noise and improve the robustness of the search.

[0158] In an optional embodiment of the present invention, determining the reference point on the target arc includes: determining at least one candidate position point on the target arc based on a preset step size; and selecting the reference point from the at least one candidate position point.

[0159] On the target arc, the position points on the arc are traversed based on a preset step size. At regular preset step intervals on the target arc, a current reference point is determined. By traversing the entire target arc, at least one candidate position point can be obtained. The optimal reference point is then selected from these candidate position points. The size of the preset step size can be determined according to actual conditions, and this embodiment of the invention does not impose a specific limitation on it.

[0160] In an optional embodiment of the present invention, the step of selecting a reference point from the at least one candidate location point includes: for any one of the at least one candidate location points, connecting the candidate location point to the starting point to determine a first straight line segment; connecting the candidate location point to the ending point to determine a second straight line segment; calculating the first point-to-line distance value between the instance point in the projection point set and the first straight line segment and the second straight line segment; and determining a reference point based on the first point-to-line distance value.

[0161] We can use any one of the at least one candidate location points as the currently calculated corner point and determine whether it is a reference point. We can connect the currently calculated candidate location point to the starting point to determine the first straight line segment, which serves as one edge of the fitted contour. We can connect the currently calculated candidate location point to the ending point to determine the second straight line segment, which serves as the other edge of the fitted contour. Then, we calculate the point-to-line distance values ​​(i.e., the first point-to-line distance values) between each instance point in the projection point set and the first and second straight line segments. Based on the above steps, we calculate the first point-to-line distance values ​​for all candidate location points. Based on the first point-to-line distance value for each currently calculated candidate location point, we determine the target reference point from the candidate location points.

[0162] The step of calculating the first point-to-line distance value between the instance point in the projection point set and the first line segment and the second line segment includes: calculating the first sub-distance value between the instance point in the projection point set and the first line segment; calculating the second sub-distance value between the instance point in the projection point set and the second line segment; and combining the first sub-distance value and the second sub-distance value to calculate the first point-to-line distance value.

[0163] First, calculate the point-to-line distance between the instance points in the projection point set and the first line segment, i.e., the first sub-distance value. Then, calculate the point-to-line distance between the instance points in the projection point set and the second line segment, i.e., the second sub-distance value. The point-to-line distance can be calculated using the geometric formula for point-to-line distance, which will not be elaborated upon here. Finally, combine the first and second sub-distance values ​​to determine the first point-to-line distance value.

[0164] The calculation process for the first sub-distance value, namely, calculating the first sub-distance value between the instance point in the projection point set and the first line segment, includes: calculating the minimum distance between the instance point in the projection point set and the first line segment as the first sub-distance value.

[0165] In practical applications, the minimum distance between the instance point in the projection point set and the first line segment can be used as the first sub-distance value, which can more accurately reflect the relationship between the instance point and the first line segment.

[0166] The calculation process for the second sub-distance value, namely, calculating the second sub-distance value between the instance point in the projection point set and the second straight line segment, includes: calculating the minimum distance between the instance point in the projection point set and the second straight line segment as the second sub-distance value.

[0167] In practical applications, the minimum distance between the instance point in the projection point set and the second line segment can be used as the second sub-distance value, which can more accurately reflect the relationship between the instance point and the second line segment.

[0168] In an optional embodiment of the invention, determining the reference point based on the first point-line distance value includes: determining the minimum distance value from the first point-line distance value; and determining the candidate position point corresponding to the minimum distance value as the reference point.

[0169] The minimum distance value can be determined from the first point-to-line distance values, and the current reference point corresponding to the minimum distance value is determined as the target reference point. That is, among the first point-to-line distance values ​​corresponding to different current reference points, the minimum first point-to-line distance value is determined, and the candidate position point corresponding to the minimum first point-to-line distance value is determined as the target reference point.

[0170] In summary, you can refer to Figure 6 The search is performed on the target arc with a certain step size, and the sum of the closest distances between all instance points and the two right-angled sides of the first and second line segments is calculated as the first point-line distance value:

[0171]

[0172] in,

[0173]

[0174] L1:A1x+B1y+C1=0

[0175] L2:A2x+B2y+C2=0

[0176] In the formula, (x p ,y p (x) represents the coordinates of the current candidate location point found in the search. i ,y i ) represents the coordinates of the i-th instance point, and L1 is determined by the current candidate position point (x). p ,y p ) and starting point (x) begin ,y beginL2 is composed of the current candidate location point (x) p ,y p ) and termination point (x) end ,y end The equation is composed of (A1,B1,C1) and (A2,B2,C2), which are the parameters of the corresponding line equations, and N represents the number of instance points.

[0177] The candidate point with the smallest distance value between the first point and the line is used as the reference point, and the starting point, ending point, and target reference point are used as the corner points of the fitted contour.

[0178] Step 206: Determine the contour fitting information of the target object based on the starting point, the ending point, and the reference point.

[0179] The starting point, ending point, and reference point can be bounded together to form a closed figure, and the contour fitting information of the target object can be determined based on this closed figure.

[0180] In an optional embodiment of the present invention, determining the contour fitting information of the target object based on the starting point, the ending point, and the reference point includes: enclosing the starting point, the ending point, and the reference point to form a right-angled triangle frame; and determining the contour fitting information of the target object based on the right-angled triangle frame.

[0181] You can refer to Figure 7 The starting point, ending point, and reference point can be used to form a right-angled triangle, which defines the right-angled triangle frame. Then, based on this right-angled triangle frame, the contour fitting information of the specific target object can be determined.

[0182] The step of determining the contour fitting information of the target object based on the right-angled triangle frame includes: determining a rectangular frame based on the right-angled triangle frame; and determining the rectangular frame as the contour fitting information of the target object.

[0183] You can refer to Figure 8 By axially symmetrically transforming the hypotenuse of the right-angled triangle into a rectangle, the contour fitting information of the target object can be determined based on this rectangle.

[0184] Furthermore, to improve the accuracy and processing efficiency of the fitting, the distance to the target object can be determined based on the distance to the target's distant location point, and then different fitting methods can be adopted. A second point-to-line distance value between the target's distant location point and the reference line can be determined; when the second point-to-line distance value is not greater than a preset distance threshold, a line fitting is performed based on the starting point and the ending point to determine the contour fitting information of the target object.

[0185] The distance between the target's distant location and the reference line, i.e., the second point-to-line distance, can be calculated using the point-to-line distance formula. When the second point-to-line distance is not greater than a preset distance threshold, it indicates that the target object's outline is close to a straight line. In this case, a straight-line fitting can be directly performed on the starting and ending points, connecting them to determine the target outline information. When the second point-to-line distance is greater than the preset distance threshold, indicating a certain depth, the step of determining the target object's outline fitting information based on the starting point, ending point, and reference point can be executed. This involves fitting the starting point, ending point, and reference point using the steps described in the above embodiments to determine the target outline information. The size of the preset distance threshold can be determined according to actual needs; this embodiment of the invention does not impose a specific limitation on it.

[0186] This invention uses a starting point, an ending point, and a reference point to form contour fitting information. Three-point fitting better matches the actual contour characteristics, and the orientation of the target object corresponding to the fitted contour can be determined by the orientation of the three points, adapting to various different target objects. By searching on the reference arc, the search range can be predetermined, and the search can be performed from the target arc, reducing search data overhead and quickly determining the reference point. Based on the reference point, a specific rectangular box is fitted as the target contour fitting information, accurately providing the orientation of vehicles and other target objects, as well as the position of individual corner points. This provides important reference for optimizing the position and orientation of vehicles and other target objects, improving the accuracy of the target reference point, and thus reducing fitting errors at the reference point position; thereby improving the accuracy of fitting the contour of target objects.

[0187] To enable those skilled in the art to clearly understand the implementation process of the embodiments of the present invention, please refer to... Figure 9 Here is a complete example:

[0188] 1. Obtain the vehicle's visual grounding point based on the images captured by the surround-view fisheye camera.

[0189] 2. The docking location is segmented into instances to obtain multiple subsets of different vehicle grounding points, and the start and end points of each subset are obtained.

[0190] 3. Based on the calibration parameters, transform the visual grounding point to the vehicle coordinate system and verify the validity of the starting and ending points.

[0191] 4. Based on the circle formed by the starting and ending points and the visual grounding point, the vehicle outline is obtained by fitting.

[0192] The fitting process can be referenced. Figure 10Based on the subset of circumference and ground points to be searched, the farthest point is selected as a candidate contour corner point, and this distance is denoted as d. It is then determined whether this distance d is greater than a given threshold. If the distance d is not greater than the threshold, it degenerates into "Type 1" contour fitting, and the least squares method is used to output the fitted line. If the distance d is greater than the threshold, a neighborhood traversal is used to search all circumference points near the candidate contour point, and the point that minimizes the loss function is calculated. Based on the contour corner point, coordinate axes, and vehicle position, virtual points are used to fill in the remaining points, obtaining a rectangular box for unified output format and visualization.

[0193] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0194] This invention also discloses an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the steps of the contour fitting method as described above.

[0195] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0196] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0197] This invention also discloses a vehicle including the electronic equipment described above.

[0198] This invention also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the contour fitting method described above.

[0199] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0200] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0201] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0202] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0203] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0204] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0205] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0206] The foregoing has provided a detailed description of a contour fitting method, an electronic device, a vehicle, and a computer-readable storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A contour fitting method, characterized in that, include: The set of projection points in the coordinate system is determined based on the set of grounding points of objects in the image domain; Determine the start and end points of the target object within the set of projection points; A reference point is determined based on the starting point and the ending point; The contour fitting information of the target object is determined based on the starting point, the ending point, and the reference point.

2. The method according to claim 1, characterized in that, Determining the reference point based on the starting point and the ending point includes: A reference circumference is determined based on the starting point, the ending point, and the straight-line distance between the starting point and the ending point, wherein the straight-line distance between the starting point and the ending point is the diameter of the reference circumference. A reference point is determined on the reference circumference.

3. The method according to claim 2, characterized in that, Determining a reference point on the reference circumference includes: Determine the target far instance point from the set of projection points; Based on the target far-field point and the reference circumference, determine the target far-field location point; Determine the target arc based on the target's distant location point; Determine a reference point on the target arc.

4. The method according to claim 3, characterized in that, Determining the target far instance point in the set of projection points includes: For any candidate instance point in the set of projection points, determine the first straight-line distance between the candidate instance point and the starting point; Determine the second straight-line distance between the candidate instance point and the termination point; By combining the first straight-line distance and the second straight-line distance, the target distant instance point is determined from the candidate instance points.

5. The method according to claim 3, characterized in that, Determining the target's distant location point based on the target's far-instance point and the reference circumference includes: Connect the center of the reference circle with the target farthest point to determine the reference straight line; The intersection of the reference line and the reference circle is determined as the target's far-distance location point.

6. The method according to claim 3, characterized in that, Determining the target arc based on the target's distant location point includes: On the reference circumference, the arc within a preset range of the target distant position point is determined as the target arc.

7. The method according to claim 3, characterized in that, Determining the reference point on the target arc includes: On the target arc, at least one candidate position point is determined based on a preset step size; A reference point is selected from the at least one candidate location point.

8. The method according to claim 7, characterized in that, The step of selecting reference points from the at least one candidate location points includes: For any one of the at least one candidate location points, connect the candidate location point to the starting point to determine the first straight line segment; Connect the candidate location points with the termination point to determine the second straight line segment; Calculate the distance between the instance points in the projection point set and the first point-line distance between the first line segment and the second line segment; A reference point is determined based on the distance value between the first point and the line.

9. The method according to claim 8, characterized in that, The calculation of the distance between the instance points in the projection point set and the first point-line distance between the first line segment and the second line segment includes: Calculate the first sub-distance value between the instance point in the projection point set and the first line segment; Calculate the second sub-distance value between the instance point in the projection point set and the second line segment; The first sub-distance value and the second sub-distance value are combined to form the first point-line distance value.

10. The method according to claim 9, characterized in that, The calculation of the first sub-distance value between the instance points in the projection point set and the first line segment includes: The minimum distance between an instance point in the projection point set and the first line segment is calculated as the first sub-distance value; correspondingly, The calculation of the second sub-distance value between the instance points in the projection point set and the second line segment includes: The minimum distance between an instance point in the projection point set and the second line segment is calculated as the second sub-distance value.

11. The method according to claim 8, characterized in that, Determining the reference point based on the first point-line distance value includes: Determine the minimum distance value from the first point-line distance value; The candidate location point corresponding to the minimum distance value is determined as the reference point.

12. The method according to claim 3, characterized in that, The method further includes: Determine the second point-to-line distance value between the target's distant location point and the reference line; When the distance value between the second point and the line is not greater than the preset distance threshold, a line fitting is performed based on the starting point and the ending point to determine the contour fitting information of the target object.

13. The method according to claim 1, characterized in that, Determining the contour fitting information of the target object based on the starting point, the ending point, and the reference point includes: A right-angled triangle is formed by enclosing the starting point, the ending point, and the reference point. The contour fitting information of the target object is determined based on the right-angled triangular frame.

14. The method according to claim 13, characterized in that, The step of determining the contour fitting information of the target object based on the right-angled triangle bounding box includes: Based on the right-angled triangle frame, determine the rectangular frame; The rectangle is defined as the contour fitting information of the target object.

15. The method according to claim 1, characterized in that, Determining the set of projection points in the coordinate system based on the set of grounding points of objects in the image domain includes: Transform the set of grounding points of objects in the image domain to the vehicle coordinate system to determine the set of projection points in the vehicle coordinate system.

16. The method according to claim 15, characterized in that, Determining the start and end points of the target object in the set of projection points includes: Obtain the vehicle's forward direction; The first instance point of the target object in the projection point set in the forward direction is determined as the starting point; The last instance point of the target object in the set of grounding points in the forward direction is determined as the termination point.

17. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the contour fitting method as described in any one of claims 1 to 16.

18. A vehicle, characterized in that, Including the electronic device as described in claim 17.

19. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the steps of the contour fitting method as described in any one of claims 1 to 16.

Images