Multi-camera based size estimation method, apparatus, device, and storage medium

Abstract

The application relates to a multi-camera-based size estimation method, device, equipment and storage medium. The size estimation method comprises the following steps: detecting at least one image of a pseudo three-dimensional bounding box including an obstacle, and obtaining a plurality of observation points, wherein the plurality of observation points are used to identify positions of side bottom edges of the obstacle; fitting a fitting line for representing the side bottom edges of the obstacle in a world coordinate system according to the plurality of observation points, and projecting the fitting line into each image respectively, so that the projection of the fitting line intersects with the vertical lines of the bottom edges of the pseudo three-dimensional bounding box in the images at a plurality of intersection points; determining two end points of the side bottom edges of the obstacle from the plurality of intersection points, and determining the length of the obstacle according to the two end points. The size estimation method provided by the application can improve the problem that the size estimation accuracy of the obstacle in the prior art is not high.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a size estimation method and device based on multiple cameras, equipment and storage medium. BACKGROUND

[0002] Obstacle perception is an important branch of automatic driving technology. An automatic driving vehicle needs to identify and detect obstacles such as vehicles in the surrounding environment to achieve a more intelligent and safe automatic driving process. When detecting obstacles, if the size and other information of the obstacles can be accurately detected, the automatic driving system can make more reasonable and effective path planning.

[0003] Currently, the position of an obstacle on an image captured by a single camera can be labeled using a pseudo three-dimensional bounding box, and the size of the obstacle can be estimated according to the pseudo three-dimensional bounding box. However, the accuracy of this method needs to be improved because the single pseudo three-dimensional bounding box provides limited observation points, especially when the obstacle is truncated by the image, it is difficult to output the accurate length of the obstacle. Therefore, the existing technology still has the problem of low accuracy of obstacle size estimation. SUMMARY

[0004] Therefore, the present application provides a size estimation method and device based on multiple cameras, equipment and storage medium to improve the problem of low accuracy of obstacle size estimation in the prior art.

[0005] In a first aspect, the present application provides a size estimation method based on multiple cameras, which comprises: detecting at least one image including a pseudo three-dimensional bounding box of an obstacle to obtain a plurality of observation points, wherein the plurality of observation points are used to identify the position of the side bottom edge of the obstacle; fitting a fitting line representing the side bottom edge of the obstacle in a world coordinate system according to the plurality of observation points, and projecting the fitting line into each image respectively, so that the projection of the fitting line intersects with the bottom edge perpendicular of the pseudo three-dimensional bounding box in a plurality of intersection points; determining two end points of the side bottom edge of the obstacle from the plurality of intersection points, and determining the length of the obstacle according to the two end points.

[0006] In combination with the first aspect, in a first implementation manner of the first aspect, the step of fitting the fitting line representing the side bottom edge of the obstacle in the world coordinate system according to the plurality of observation points comprises: respectively converting the pixel coordinates of the plurality of observation points into the world coordinates of the observation points; determining the confidence of each observation point according to the confidence rule, wherein the observation points include the tire contact point and the end point of the side bottom edge of the pseudo three-dimensional bounding box, and the confidence of the tire contact point is greater than the confidence of the end point of the side bottom edge of the pseudo three-dimensional bounding box; fitting the fitting line according to the world coordinates and the confidence of the plurality of observation points respectively.

[0007] In a second implementation form of the first aspect, according to the world coordinates and the confidence levels of the plurality of observation points, the step of fitting the plurality of observation points to obtain the fitting line comprises: calculating a total error of fitting the plurality of observation points and an error of each observation point according to the world coordinates and the confidence levels of the plurality of observation points; fitting the plurality of observation points to obtain the fitting line according to the world coordinates and the confidence levels of the plurality of observation points in a case that the total error is less than or equal to an error threshold; and selecting the plurality of observation points according to the error of each observation point in a case that the total error is greater than the error threshold, fitting the selected plurality of observation points to obtain the fitting line according to the world coordinates and the confidence levels of the selected plurality of observation points, and wherein the total error of fitting the selected plurality of observation points is less than or equal to the error threshold.

[0008] In a third implementation form of the first aspect, the step of determining the two end points of the side bottom edge of the obstacle from the plurality of intersection points comprises: converting the pixel coordinates of each intersection point to obtain the world coordinates of each intersection point; calculating the offset of each intersection point on the fitting line according to the world coordinates of each intersection point; and taking the intersection point with the largest offset and the intersection point with the smallest offset from the plurality of intersection points as the two end points of the side bottom edge of the obstacle.

[0009] In a fourth implementation form of the first aspect, after the step of determining the two end points of the side bottom edge of the obstacle from the plurality of intersection points, the method further comprises: taking one of the two end points as a base point, and constructing a first ray and a second ray based on the base point in the world coordinate system, wherein the first ray and the second ray are respectively used to point to the height and the width of the obstacle; projecting the first ray and the second ray onto at least one image, so that the projections of the first ray and the second ray intersect with the top edge and the side edge of the two-dimensional bounding box on each image at a first intersection point and a second intersection point, respectively; and determining the width, the height and / or the center point of the obstacle according to the two end points, the first intersection point and the second intersection point on each image.

[0010] In a fifth implementation form of the first aspect, according to the fourth implementation form of the first aspect, the step of constructing the first ray and the second ray based on the base point in the world coordinate system comprises: determining the heading of the obstacle according to the line connecting the two end points; converting the pixel coordinates of the base point to obtain the world coordinates of the base point; and constructing the first ray and the second ray with the world coordinates of the base point as the starting point, wherein the first ray is perpendicular to the ground on which the obstacle is located and points to the top of the obstacle, and the second ray is perpendicular to the heading and the first ray and points to the side of the obstacle.

[0011] In a sixth implementation form of the first aspect, in the fourth implementation form of the first aspect, the step of determining the width, the height and / or the center point of the obstacle according to the two end points, the first intersection point and the second intersection point on each image comprises: determining a height set of the obstacle according to the base point and the first intersection point on each image, wherein the height set comprises at least one height of the obstacle; determining a width set of the obstacle according to the base point and the second intersection point on each image, wherein the width set comprises at least one width of the obstacle; determining a center point set of the obstacle according to the two end points, the first intersection point and the second intersection point on each image, wherein the center point set comprises at least one center point of the obstacle; and averaging the height set, the width set and the center point set respectively to obtain the height, the width and the center point of the obstacle.

[0012] In a second aspect, the present application provides a size estimation device, comprising: a detection unit configured to detect at least one image comprising a pseudo three-dimensional bounding box of an obstacle to obtain a plurality of observation points, wherein the plurality of observation points are used to identify positions of side edges of the obstacle; a fitting unit configured to fit a fitting line representing the side edges of the obstacle in a world coordinate system according to the plurality of observation points; a projection unit configured to project the fitting line into each image respectively, so that the projection of the fitting line intersects with a vertical line of a bottom edge of the pseudo three-dimensional bounding box in a plurality of intersection points; and a determination unit configured to determine two end points of the side edges of the obstacle from the plurality of intersection points, and determine a length of the obstacle according to the two end points.

[0013] In a first implementation form of the second aspect, the fitting unit is specifically configured to: convert the pixel coordinates of the plurality of observation points into world coordinates of the observation points respectively; determine a confidence degree corresponding to each observation point according to a confidence rule, wherein the observation points comprise tire contact points and end points of the side edges of the pseudo three-dimensional bounding box, and the confidence degree of the tire contact points is greater than the confidence degree of the end points of the side edges of the pseudo three-dimensional bounding box; and fit the fitting line according to the world coordinates and the confidence degrees of the plurality of observation points respectively.

[0014] In a second implementation manner of the second aspect, in the second implementation manner of the second aspect, the fitting unit is further configured to: calculate a total error of fitting the multiple observation points and an error of each observation point according to the world coordinates and the confidence levels of the multiple observation points; in a case where the total error is less than or equal to an error threshold, fit the multiple observation points to obtain the fitting line according to the world coordinates and the confidence levels of the multiple observation points; and in a case where the total error is greater than the error threshold, screen the multiple observation points according to the error of each observation point, so that the total error of fitting the screened multiple observation points is less than or equal to the error threshold, and fit the screened multiple observation points to obtain the fitting line according to the world coordinates and the confidence levels of the screened multiple observation points.

[0015] In a third implementation manner of the second aspect, in the third implementation manner of the second aspect, the determining unit is specifically configured to: convert the pixel coordinates of each intersection point to obtain the world coordinates of each intersection point; calculate the offset of each intersection point on the fitting line according to the world coordinates of each intersection point; and take the intersection point with the largest offset and the intersection point with the smallest offset in the multiple intersection points as two end points of the side edge of the side surface of the obstacle.

[0016] In a fourth implementation manner of the second aspect, in the fourth implementation manner of the second aspect, the size estimation apparatus further includes a constructing unit, the constructing unit being configured to take one of the two end points as a base point, and construct a first ray and a second ray based on the base point in the world coordinate system, where the first ray and the second ray are respectively used to point to the height and the width of the obstacle; the projecting unit is further configured to project the first ray and the second ray onto at least one image, so that the projections of the first ray and the second ray intersect with the top edge and the side edge of the two-dimensional surrounding frame on each image at a first intersection point and a second intersection point; and the determining unit is further configured to determine the width, the height and / or the center point of the obstacle according to the two end points, the first intersection point and the second intersection point on each image.

[0017] In a fifth implementation manner of the second aspect, in the fourth implementation manner of the second aspect, the constructing unit is specifically configured to: determine a heading direction of the obstacle according to the line connecting the two end points; convert the pixel coordinates of the base point to obtain the world coordinates of the base point; and construct the first ray and the second ray with the world coordinates of the base point as a starting point, where the first ray is perpendicular to the ground on which the obstacle is located and points to the top of the obstacle, and the second ray is perpendicular to the heading direction and the first ray and points to the side surface of the obstacle.

[0018] In a sixth implementation of the second aspect, in combination with the fourth implementation of the second aspect, the determining unit is specifically configured to: determine a height set of the obstacle according to the base point and the first intersection point on each image, wherein the height set comprises at least one height of the obstacle; determine a width set of the obstacle according to the base point and the second intersection point on each image, wherein the width set comprises at least one width of the obstacle; determine a center point set of the obstacle according to the two end points, the first intersection point and the second intersection point on each image, wherein the center point set comprises at least one center point of the obstacle; and obtain the height, the width and the center point of the obstacle by averaging the height set, the width set and the center point set respectively.

[0019] In a third aspect, the present application further provides a size estimation device, comprising a processor and a memory connected through a bus; the processor is configured to execute a plurality of instructions; and the memory is configured to store the plurality of instructions, which are adapted to be loaded and executed by the processor to perform the size estimation method of the first aspect or any one of the implementation manners of the first aspect.

[0020] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a plurality of instructions, which are adapted to be loaded and executed by a processor to perform the size estimation method of the first aspect or any one of the implementation manners of the first aspect.

[0021] In summary, the present application provides a multi-camera-based size estimation method, device, equipment and storage medium, wherein the size estimation method comprises: a size estimation device detects at least one image comprising a pseudo three-dimensional bounding box of an obstacle to obtain a plurality of observation points, and fits a fitting line representing a side bottom edge of the obstacle in a world coordinate system according to the plurality of observation points, and then projects the fitting line into each image, so that the projection of the fitting line intersects with the bottom edge perpendicular of the pseudo three-dimensional bounding box in each image to obtain a plurality of intersection points, and determines two end points of the side bottom edge of the obstacle from the plurality of intersection points, and determines the length of the obstacle according to the two end points. It can be seen that the present application uses at least one image collected by a plurality of cameras to obtain a plurality of observation points of the obstacle, and restores the real length of the obstacle according to the plurality of observation points, thereby breaking the limitation of insufficient observation points and possible image truncation in the prior art, and improving the problem of low size estimation accuracy of the obstacle. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 A schematic diagram of an image comprising a pseudo three-dimensional bounding box of an obstacle in one embodiment;

[0023] Figure 2 A schematic diagram of a case where the head of an obstacle is truncated by an image in one embodiment;

[0024] Figure 3 A schematic diagram for the case that the tail of the obstacle is truncated by the image in one embodiment;

[0025] Figure 4 A schematic diagram for the fitting step in one embodiment;

[0026] Figure 5 A schematic diagram for the multiple intersection points of the projection of the fitted line and the perpendicular of the bottom side in one embodiment;

[0027] Figure 6 A schematic diagram for the step of constructing the first ray and the second ray in one embodiment;

[0028] Figure 7 A schematic diagram for the step of determining the first intersection point and the second intersection point in one embodiment;

[0029] Figure 8 A schematic diagram for the step of determining the height of the obstacle in one embodiment;

[0030] Figure 9 A schematic diagram for the step of constructing the three-dimensional bounding box of the obstacle in one embodiment;

[0031] Figure 10 A schematic diagram for the flow of the size estimation method in one embodiment;

[0032] Figure 11 A schematic block diagram of a size estimation apparatus provided in the present application;

[0033] Figure 12 A structural block diagram of a size estimation device provided in the present application. DETAILED DESCRIPTION

[0034] In order to make the objects, technical solutions and advantages of the present application clearer, further detailed description will be given to the present application in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0035] Since the embodiments of the present application involve relatively more professional terms, in order to facilitate understanding, the related terms and concepts that may be involved in the embodiments of the present application will be introduced first.

[0036] 1. Two-dimensional (2Dimensions, 2D) bounding box

[0037] The two-dimensional bounding box is also called rectangular frame label. The two-dimensional bounding box includes a rectangular frame and an oblique rectangular frame. The pseudo three-dimensional bounding box in the present application refers to the rectangular frame, for example Figure 7 as shown, the two-dimensional bounding box is IJKL.

[0038] 2. Pseudo-3D bounding box

[0039] The pseudo-3D bounding box is also referred to as a 2.5D bounding box. The pseudo-3D bounding box is a bounding box in a "pseudo-cuboid shape" for an obstacle in an image. Generally, the pseudo-3D bounding box includes a front-side 2-view cuboid and a front-side 3-view cuboid, etc. In this application, the pseudo-3D bounding box refers to the front-side 2-view cuboid. The front-side 2-view cuboid includes a 2D bounding box for representing a side surface of the obstacle, and / or a 2D bounding box for representing a front surface (back surface) of the obstacle. For example Figure 1 As shown, the pseudo-3D bounding box of the obstacle includes a 2D bounding box ABCD for representing a side surface of the obstacle, and a 2D bounding box ADEF for representing a back surface of the obstacle. The pseudo-3D bounding box includes a side surface bottom edge (line segment AB).

[0040] 3. 3D bounding box

[0041] The 3D bounding box refers to a six-surface cuboid for identifying a real 3D boundary of an obstacle. When the 3D bounding box is displayed on an image, it is a six-surface cuboid for framing a space where the car is located as shown in Figure 9 .

[0042] It should be noted that the size estimation device and the size estimation apparatus involved in the present application can include but are not limited to a dedicated size estimation device, a terminal device, a computer, a processor, etc. and can be a device integrated in a car or a detachable independent device on the car. The size estimation device and the size estimation apparatus can interact with a camera on the car to obtain an image including an obstacle, etc. The processor can include but is not limited to a central processing unit (CPU), a general-purpose processor, a co-processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The processor can implement the described methods of the present application, such as detecting an image including a pseudo-3D bounding box and a 2D bounding box of an obstacle, etc. The present application will not be repeated here.

[0043] It should be noted that the diagrams provided in the embodiments only schematically illustrate the basic concepts of the present application, and only the components related to the present application are shown in the diagrams, rather than the number, shape and size of the components when actually implemented. The shapes, number and proportions of the components when actually implemented can be arbitrarily changed, and the layout of the components can be more complex. The structures, proportions and sizes shown in the diagrams attached to the present application are only used to illustrate the content disclosed in the present application, so as to be understood and read by those skilled in the art, and do not limit the conditions under which the present application can be implemented, and therefore do not have technical significance. Any modification of the structure, change of the proportional relationship or adjustment of the size, without affecting the effects and purposes that can be achieved by the present application, should still fall within the scope of the technical content disclosed by the present application. At the same time, the directions or positional relationships such as "upper", "lower", "left", "right", "middle", "vertical", "horizontal", "horizontal", "inner", "outer", "radial", "circumferential" and the like referred to in the present specification are based on the directions or positional relationships shown in the drawings, and are only used to simplify the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, structure and operation. The change or adjustment of the relative relationship, without substantially changing the technical content, is also considered as the scope in which the present application can be implemented. Therefore, it should not be understood as a limitation on the present application. In addition, the terms "first" and "second" are only for descriptive purposes, and should not be understood as indicating or implying relative importance.

[0044] Since the accuracy of the size estimation method based on a single pseudo three-dimensional bounding box is not high, the present application proposes a size estimation method based on multiple cameras. Next, the present application will take a size estimation device as the execution subject to illustrate the size estimation method of the present application. Specifically:

[0045] First, the size estimation device acquires at least one image photographed by multiple cameras, which includes an obstacle and a pseudo three-dimensional bounding box used to identify the position of the obstacle. The image of the pseudo three-dimensional bounding box including the obstacle is shown in Figure 1 ABCD and EF are the pseudo three-dimensional bounding boxes of the obstacle, and the line segment AB is the side bottom edge of the pseudo three-dimensional bounding box. After acquiring at least one image of the obstacle, the at least one image is detected to obtain a plurality of observation points, which can include Figure 1end point A and end point B of the side bottom edge AB, and the tire contact point H and the point G. It is to be noted that the multi-camera includes multiple cameras, so that each camera of the multi-camera can take a picture of the obstacle at different orientations at the same time to obtain at least one image including the obstacle, and the image not including the obstacle is discarded, so that the number of images including the obstacle taken by the multi-camera is less than or equal to the number of cameras, and is greater than or equal to one. It is to be further noted that the multiple observation points can be from different images, for example, in the case that the head of the obstacle in Figure 2 is truncated by the image, the point A and the point G at the tail of the obstacle are obtained as the observation points, and for example, in the case that the tail of the obstacle in Figure 3 is truncated by the image, the point B and the point H at the head of the obstacle are obtained as the observation points.

[0046] Secondly, after obtaining the multiple observation points, the size estimation device projects the point A, the point G, the point H and the point B into the world coordinate system respectively. Since the observation points A, B, G and H are all points on the ground where the obstacle is located, it is considered that the observation points A, B, G and H are in the same plane when projected into the world coordinate system, and the corresponding points of the observation points A, B, G and H in the world coordinate system can be simply represented as the points a, b, g and h as shown in Figure 4 The size estimation device determines the confidence degrees corresponding to the points A, B, G and H respectively according to a confidence rule, and the confidence rule records the confidence degrees corresponding to the observation points respectively, for example, the confidence degree of the tire contact point is a preset first confidence degree, and the confidence degree of the end point of the side bottom edge of the pseudo three-dimensional bounding box is a preset second confidence degree, and the first confidence degree is greater than the second confidence degree.

[0047] Thirdly, after determining the confidence degrees, the size estimation device fits according to the world coordinates and the confidence degrees corresponding to the points a, b, g and h respectively, and calculates the total error of fitting the four points, if the total error is greater than an error threshold, the point with the largest error is filtered out, and the fitting is performed again and the total error of fitting is calculated, if the total error is still greater than the error threshold, the point with the largest error is continuously filtered out from the remaining points, until the total error of fitting is less than or equal to the error threshold. Assuming that the total error of fitting the points a, b, g and h is less than or equal to the error threshold, the fitting line obtained by fitting the points a, b, g and h is taken as the side bottom edge of the obstacle, and the direction indicated by the fitting line is the heading of the obstacle, wherein the fitting method includes but is not limited to Deming regression algorithm. After obtaining the fitting line, the size estimation device projects the fitting line into each image respectively, so that the projection of the fitting line intersects with the vertical line of the bottom edge of the pseudo three-dimensional bounding box in multiple intersection points, and the multiple intersection points can be from different images. As shown in Figure 5As shown, point P is the intersection between the projection of the fitting line and the perpendicular CB of the bottom side, and point Q is the intersection between the projection of the fitting line and the perpendicular AD of the bottom side.

[0048] Finally, the size estimation device converts the plurality of intersection points to the world coordinate system to determine the offset of each intersection point on the fitting line, and takes the intersection point with the largest offset and the intersection point with the smallest offset in the plurality of intersection points as the two end points of the side bottom of the obstacle, and determines the length of the obstacle according to the two end points. For example Figure 5 As shown, assuming that point P and point Q are the intersection point with the largest offset and the intersection point with the smallest offset in the plurality of intersection points, respectively, then taking point P and point Q as the two end points on the side bottom of the obstacle, and calculating the distance between point P and point Q in the world coordinate system by using the Pythagorean theorem to obtain the length of the obstacle.

[0049] In summary, in view of the problem that the size estimation method based on a single pseudo three-dimensional bounding box in the prior art has low accuracy, the size estimation device of the present application first acquires a plurality of observation points of the obstacle through at least one image collected by a plurality of cameras, then projects the plurality of observation points to the world coordinate system to fit the side bottom of the obstacle, and finally determines the real length of the obstacle based on the side bottom. It can be seen that the present application breaks through the limitations of insufficient observation points and possible image truncation in the prior art, and improves the problem of low size estimation accuracy of the obstacle.

[0050] In an embodiment, as shown in Figure 10 The present application provides a size estimation method based on a plurality of cameras. Next, the present application will take the size estimation device as the execution subject to describe the method provided in the present embodiment. Specifically:

[0051] 1001: detecting at least one image including a pseudo three-dimensional bounding box of an obstacle to obtain a plurality of observation points.

[0052] Among them, the plurality of observation points are used to identify the position of the side bottom of the obstacle, and the observation points can include the end points A and B of the side bottom AB, and the tire grounding point H and point G as shown in Figure 1

[0053] 1002: fitting a fitting line for representing the side bottom of the obstacle in the world coordinate system according to the plurality of observation points.

[0054] Among them, the size estimation device first converts the pixel coordinates of the plurality of observation points to corresponding world coordinates respectively, and uses the Deming regression algorithm to fit the world coordinates of the plurality of observation points to obtain the fitting line. The observation points A, B, G and H in the world coordinate system can be represented as Figure 4 ​The point a, the point b, the point g and the point h, and the straight line between the point a, the point b, the point g and the point h, i.e. the fitting line.

[0055] 1003: Projecting the fitting line into each image respectively, so that the projection of the fitting line intersects with the vertical line of the bottom side of the pseudo three-dimensional bounding box in each image at a plurality of intersection points.

[0056] The size estimation device projects the fitting line into each image in the at least one image respectively, so that the fitting line intersects with the vertical line of the bottom side of the pseudo three-dimensional bounding box in each image at a plurality of intersection points. Figure 5 As shown, the point P is the intersection point between the projection of the fitting line and the vertical line CB of the bottom side, and the point Q is the intersection point between the projection of the fitting line and the vertical line AD of the bottom side.

[0057] 1004: Determining two end points of the side bottom side of the obstacle from the plurality of intersection points, and determining the length of the obstacle according to the two end points.

[0058] The size estimation device takes the two points on the edge of the fitting line in the plurality of intersection points as the two end points of the side bottom side of the obstacle, and converts the pixel coordinates of the two end points to obtain the corresponding world coordinates, and calculates the distance between the world coordinates of the two end points to obtain the length of the obstacle. For example Figure 5 , assuming that the plurality of intersection points are projected into the world coordinate system, and the intersection point P and the intersection point Q in the plurality of intersection points are determined to be the two end points on the edge of the fitting line, the intersection point P and the intersection point Q are taken as the two end points of the side bottom side of the obstacle, and the distance between the world coordinates of the intersection point P and the intersection point Q is calculated to obtain the length of the obstacle, for example Figure 6 The line segment qp is the length of the obstacle.

[0059] In an implementable manner, the step of fitting the fitting line for representing the side bottom side of the obstacle in the world coordinate system according to the plurality of observation points includes: converting the pixel coordinates of the plurality of observation points to obtain the world coordinates of each observation point respectively; determining the confidence degree corresponding to each observation point according to the confidence degree rule, wherein the observation points include the tire contact points and the end points of the side bottom side of the pseudo three-dimensional bounding box, and the confidence degree of the tire contact point is greater than the confidence degree of the end point of the side bottom side of the pseudo three-dimensional bounding box; and fitting the fitting line according to the world coordinates and the confidence degrees corresponding to the plurality of observation points respectively.

[0060] In order to fit a fitting line that can more accurately represent the side bottom edge of the obstacle, the application can also improve the accuracy of the fitting line by setting different confidence levels for the observation points. This is because different observation points can express the side bottom edge of the obstacle to different degrees, for example, the tire contact point of the car can be closer to the side bottom edge of the car than the pseudo three-dimensional enclosing frame. In this regard, when performing the fitting step, the size estimation device first determines the pixel coordinates of each observation point on the corresponding image according to the position of each observation point on the corresponding image, and converts the pixel coordinates to obtain the corresponding world coordinates. In addition, the size estimation device also determines the confidence level of each observation point according to the confidence rule, and the confidence levels of different types of observation points are different, for example, the confidence level of the tire contact point is the first confidence level, and the confidence level of the end point of the side bottom edge of the pseudo three-dimensional enclosing frame is the second confidence level, and the first confidence level is greater than the second confidence level. After determining the world coordinates and confidence levels of each observation point, the size estimation device determines the regression straight line between the plurality of observation points using the Deming regression algorithm, so that the sum of the weighted distances of the plurality of observation points to the regression straight line is minimized, and the regression straight line is taken as the fitting line of the plurality of observation points, wherein the weight corresponding to each observation point, i.e. the confidence level of each observation point.

[0061] In an implementable manner, the step of fitting the fitting line according to the world coordinates and confidence levels corresponding to the plurality of observation points includes: calculating the total error of fitting the plurality of observation points and the error corresponding to each observation point according to the world coordinates and confidence levels corresponding to the plurality of observation points; in the case that the total error is less than or equal to the error threshold, fitting the plurality of observation points according to the world coordinates and confidence levels corresponding to the plurality of observation points to obtain the fitting line; in the case that the total error is greater than the error threshold, screening the plurality of observation points according to the error corresponding to each observation point, so that the total error of fitting the screened plurality of observation points is less than or equal to the error threshold, and fitting the screened plurality of observation points according to the world coordinates and confidence levels corresponding to the screened plurality of observation points to obtain the fitting line.

[0062] In order to further fit a fitting line that can more accurately represent the side bottom edge of the obstacle, the application can also improve the accuracy of the fitting line by screening the observation points. Specifically, assuming that the size estimation device obtains W observation points in the at least one image, the W observation points are first fitted and the total error of the fitting is calculated. If the total error is greater than the error threshold, the observation point with the largest error in the W observation points is discarded to obtain (W-1) observation points. Then the (W-1) observation points are fitted and the total error of the fitting is calculated. If the total error is less than or equal to the error threshold, the regression straight line corresponding to the (W-1) observation points is taken as the fitting line. If the total error is greater than the error threshold, the above screening process is repeated until (W-Z) observation points are screened, and the total error of the fitting of the (W-Z) observation points is less than or equal to the error threshold. The regression straight line corresponding to the (W-Z) observation points is taken as the fitting line. Wherein, W and Z are positive integers and W is greater than Z, and the error threshold is a preset constant value.

[0063] In an implementable manner, the step of determining two end points of the side bottom edge of the obstacle from the plurality of intersection points includes: converting the pixel coordinates of each intersection point to obtain the world coordinates of each intersection point; calculating the offset of each intersection point on the fitting line according to the world coordinates of each intersection point; and taking the intersection point with the largest offset and the intersection point with the smallest offset in the plurality of intersection points as the two end points of the side bottom edge of the obstacle.

[0064] In the above embodiment, the implementable manner determines the end points of the side bottom edge of the obstacle from the plurality of intersection points by determining the offset of each intersection point on the fitting line. Specifically, the size estimation device converts the pixel coordinates corresponding to each intersection point into world coordinates to project each intersection point onto the fitting line. Any point on one side of the fitting line where all intersection points are located is taken as the origin, and the distance from the world coordinates of each intersection point to the origin is calculated as the offset corresponding to each intersection point. Since the end points of the side bottom edge of the obstacle are the most marginal points among all intersection points, the points with the largest and smallest offsets in the plurality of intersection points are taken as the end points of the side bottom edge of the obstacle. The heading of the obstacle is determined according to the two end points, and the distance between the world coordinates of the two end points is calculated using the Pythagorean theorem to obtain the length of the obstacle.

[0065] In an implementable manner, after determining the two end points of the side bottom edge of the obstacle from the plurality of intersection points, the size estimation device can further: take one of the two end points as a base point, and construct a first ray and a second ray based on the base point in the world coordinate system, wherein the first ray and the second ray are respectively used to point to the height and width of the obstacle; project the first ray and the second ray onto at least one image, so that the projections of the first ray and the second ray intersect with the top edge and the side edge of the two-dimensional bounding box in each image at a first intersection point and a second intersection point respectively; and determine the width, height and / or center point of the obstacle according to the two end points, the first intersection point and the second intersection point in each image.

[0066] wherein, after determining the two end points of the side bottom edge of the obstacle, the size estimation device takes one of the two end points as a base point, for example Figure 6 the side bottom edge of the obstacle in FIG. 6 is a line segment PQ, and the two end points of the side bottom edge are point P and point Q. The size estimation device takes point Q as the base point. Then the size estimation device converts the base point to the world coordinate system, and constructs a first ray pointing to the height of the obstacle and a second ray pointing to the width of the obstacle with the world coordinate of the base point as the starting point, for example Figure 6 as shown in FIG. 6, the corresponding point of the base point Q in the world coordinate system is point q, the first ray is β, and the second ray is γ. Then the size estimation device projects the first ray and the second ray onto each image, so that the projection of the first ray intersects with the top edge of the two-dimensional bounding box in each image at a second intersection point, and the projection of the second ray intersects with the side edge of the two-dimensional bounding box in each image at a second intersection point, for example Figure 7 the projection of the first ray β in FIG. 6 onto the image is β', and the projection of the second ray γ onto the image is γ'. The projection β' of the first ray and the projection γ' of the second ray intersect with the top edge JK and the side edge KL of the two-dimensional bounding box IJKL at a first intersection point M and a second intersection point N respectively. Finally, since the two end points, the first intersection point and the second intersection point can describe four vertices of a three-dimensional bounding box of the obstacle, the size estimation device can respectively convert the pixel coordinates of the two end points, the first intersection point and the second intersection point to obtain the world coordinates of the two end points, the first intersection point and the second intersection point, and determine the size information of the obstacle according to the world coordinates of the two end points, the first intersection point and the second intersection point, the size information including at least one of the width, the height and / or the center point. For example, as shown in Figure 7 the size estimation device first converts the pixel coordinates of point P, point Q, point M and point N in the world coordinate system to obtain the world coordinates of point P, point Q, point M and point N respectively, then calculates the distance between the world coordinates of point Q and point M to obtain the height of the obstacle according to the Pythagorean theorem, calculates the distance between the world coordinates of point Q and point N to obtain the width of the obstacle, and takes the midpoint of the world coordinates of point P, point Q, point M and point N to obtain the center point of the obstacle.

[0067] In an implementable manner, the step of constructing the first ray and the second ray based on the base point in the world coordinate system comprises: determining a heading of the obstacle according to the line connecting the two end points; converting the pixel coordinates of the base point to obtain the world coordinates of the base point; and constructing the first ray and the second ray with the world coordinates of the base point as the starting point, wherein the first ray is perpendicular to the ground where the obstacle is located and points to the top of the obstacle, and the second ray is perpendicular to the heading and the first ray and points to the side of the obstacle.

[0068] In an implementable manner, the step of constructing the first ray and the second ray based on the base point in the world coordinate system comprises: determining a heading of the obstacle according to the line connecting the two end points; converting the pixel coordinates of the base point to obtain the world coordinates of the base point; and constructing the first ray and the second ray with the world coordinates of the base point as the starting point, wherein the first ray is perpendicular to the ground where the obstacle is located and points to the top of the obstacle, and the second ray is perpendicular to the heading and the first ray and points to the side of the obstacle. Figure 6 As shown, the size estimation device first converts the pixel coordinates (r1, c1) of the base point Q on the side bottom in the image to obtain the world coordinates (x1, y1, z1) of the base point Q in the world coordinate system, i.e., point q, and converts the pixel coordinates (r2, c2) of the other end point P in the image to obtain the world coordinates (x2, y2, z2) of point P in the world coordinate system, i.e., point p, and then establishes a target vector pointing to point q with point q as the starting point, and takes the target vector as the heading a of the obstacle. Then, the first ray b perpendicular to the ground and pointing to the top of the obstacle is established with point q as the starting point, and the second ray g perpendicular to the first ray b and the heading a of the obstacle and pointing to the side of the obstacle is established.

[0069] In an implementable manner, the step of determining the width, height and / or center point of the obstacle according to the two end points, the first intersection point and the second intersection point on each image comprises: determining a height set of the obstacle according to the base point and the first intersection point on each image, wherein the height set includes at least one height of the obstacle; determining a width set of the obstacle according to the base point and the second intersection point on each image, wherein the width set includes at least one width of the obstacle; determining a center point set of the obstacle according to the two end points, the first intersection point and the second intersection point on each image, wherein the center point set includes at least one center point of the obstacle; and taking the average of the height set, the width set and the center point set respectively to obtain the height, the width and the center point of the obstacle.

[0070] In an implementable manner, the step of determining the width, height and / or center point of the obstacle according to the two end points, the first intersection point and the second intersection point on each image comprises: determining a height set of the obstacle according to the base point and the first intersection point on each image, wherein the height set includes at least one height of the obstacle; determining a width set of the obstacle according to the base point and the second intersection point on each image, wherein the width set includes at least one width of the obstacle; determining a center point set of the obstacle according to the two end points, the first intersection point and the second intersection point on each image, wherein the center point set includes at least one center point of the obstacle; and taking the average of the height set, the width set and the center point set respectively to obtain the height, the width and the center point of the obstacle. Figure 1The image of the complete display of the obstacle shown has a first intersection point and a second intersection point, for example Figure 2 The image of the partial display of the obstacle shown has a first intersection point and a second intersection point, for example Figure 3 The image of the partial display of the obstacle shown has a first intersection point. Therefore, based on any one image including two end points, a first intersection point and a second intersection point, the size estimation device can determine the size information of the obstacle according to the two end points, the first intersection point and the second intersection point on the image. However, in order to further improve the accuracy, the size estimation device first obtains the height, width and center point of the obstacle according to at least one image respectively, so as to obtain a height set including at least one height, a width set including at least one width and a center point set including at least one center point, and then the size estimation device takes the average value of all heights in the height set as the height of the obstacle, takes the average value of all widths in the width set as the width of the obstacle, and takes the average value of the coordinates of all center points in the center point set as the center point of the obstacle.

[0071] In an implementable manner, the step of determining the height of the obstacle according to the two end points, the first intersection point and the second intersection point on the image includes: determining the height of the obstacle according to the pixel coordinates of the base point, the world coordinates of the base point, the pixel coordinates of the first intersection point and the unit vector of the first ray, wherein the mathematical expression of the step of determining the height of the obstacle according to the pixel coordinates of the base point, the world coordinates of the base point, the pixel coordinates of the first intersection point and the unit vector of the first ray is:

[0072] Or,

[0073] Wherein, L is the height of the obstacle, (r1, c1) is the pixel coordinates of the base point, (x1, y1, z1) is the world coordinates of the base point, (r3, c3) is the pixel coordinates of the first intersection point, (u x , v y , w z ) is the unit vector of the first ray, f x and f y are the focal length parameters of the camera parameters. It should be noted that the present embodiment provides a simple calculation method, after the first ray is mapped to the image and the first intersection point is obtained, the pixel coordinates of the first intersection point are directly used to calculate the height of the obstacle, without first converting the world coordinates of the first intersection point from the pixel coordinates of the first intersection point, and then using the world coordinates of the first intersection point to calculate the height of the obstacle, thereby further reducing the complexity of the calculation and improving the efficiency of the size estimation.

[0074] For example, as Figure 8As shown, the pixel coordinates of the base point Q (r1, c1), the world coordinates of the base point Q (x1, y1, z1), the pixel coordinates of the first intersection point M (r3, c3), the unit vector of the first ray (u x ,u y ,u z ), the camera intrinsic focal length parameters f x and f y are all known, the size estimation device can directly bring the several known parameters into the above mathematical expression to obtain the height of the obstacle, i.e., the length of the line segment QM. Similarly, the width of the obstacle can also be determined according to the pixel coordinates of the base point, the world coordinates of the base point, the pixel coordinates of the second intersection point, and the unit vector of the second ray in the above calculation manner, and the specific mathematical expression can refer to the mathematical expression in the calculation of the length of the obstacle, which will not be described here. The unit vector of the first ray can be (0, 0, 1).

[0075] In an implementable manner, the size estimation device can also construct a three-dimensional bounding box according to the size information of the obstacle obtained in the foregoing steps to reflect the real three-dimensional boundary of the obstacle. Specifically, after detecting the side bottom edge of the pseudo three-dimensional bounding box, the step of determining the size information of the obstacle according to the two end points, the first intersection point and the second intersection point further comprises: connecting the base point with the other end point to obtain a first line segment for representing the length of the obstacle; connecting the base point with the first intersection point to obtain a second line segment for representing the height of the obstacle; connecting the base point with the second intersection point to obtain a third line segment for representing the width of the obstacle; and restoring a three-dimensional bounding box of the obstacle according to the first line segment, the second line segment and the third line segment.

[0076] For example Figure 9 As shown, the size estimation device first connects the base point Q with the other end point P to obtain a first line segment QP, connects the base point Q with the first intersection point M to obtain a second line segment QM, and connects the base point Q with the second intersection point N to obtain a third line segment QN, and then restores the other line segments of the three-dimensional bounding box according to the line segments QP, QM and QN, thereby restoring the three-dimensional bounding box of the obstacle.

[0077] In summary, the present application can restore the real length of the obstacle based on at least one image of the obstacle, thereby improving the problem of low size estimation accuracy of the obstacle in the prior art.

[0078] In one embodiment, the present application also provides a size estimation device, which is shown in Figure 11The embodiments of the present application can divide the functional units of the device according to the above method examples. For example, each functional unit can be divided according to each function, or two or more functions can be integrated in one unit. The integrated unit can be realized in the form of hardware or software functional unit. It should be noted that the division of the unit in the embodiments of the present application is illustrative, and is only a logical function division. In actual implementation, another division mode can be used. Figure 11 As shown in FIG. 12, the obstacle detection device includes a detection unit 1110, a fitting unit 1120, a projection unit 1130, and a determination unit 1140. Specifically, the detection unit 1110 is configured to detect at least one image of a pseudo three-dimensional bounding box including an obstacle to obtain a plurality of observation points, wherein the plurality of observation points are used to identify positions of side edges of the obstacle; the fitting unit 1120 is configured to fit a fitting line used to represent the side edges of the obstacle in a world coordinate system according to the plurality of observation points; the projection unit 1130 is configured to project the fitting line into each image respectively, so that the projection of the fitting line intersects with the vertical line of the bottom edge of the pseudo three-dimensional bounding box in a plurality of intersection points; and the determination unit 1140 is configured to determine two end points of the side edges of the obstacle from the plurality of intersection points, and determine the length of the obstacle according to the two end points.

[0079] In an implementable manner, the fitting unit 1120 is specifically configured to: respectively convert the pixel coordinates of the plurality of observation points to obtain the world coordinates of the observation points; determine the confidence degrees corresponding to the observation points according to a confidence rule, wherein the observation points include the tire contact points and the end points of the side edges of the pseudo three-dimensional bounding box, and the confidence degree of the tire contact point is greater than the confidence degree of the end point of the side edge of the pseudo three-dimensional bounding box; and fit the fitting line according to the world coordinates and the confidence degrees corresponding to the plurality of observation points.

[0080] In an implementable manner, the fitting unit 1120 is further configured to: calculate the total error of fitting the plurality of observation points and the error corresponding to each observation point according to the world coordinates and the confidence degrees corresponding to the plurality of observation points; in a case where the total error is less than or equal to an error threshold, fit the fitting line according to the world coordinates and the confidence degrees corresponding to the plurality of observation points; and in a case where the total error is greater than the error threshold, screen the plurality of observation points according to the error corresponding to each observation point, so that the total error of fitting the screened plurality of observation points is less than or equal to the error threshold, and fit the fitting line according to the world coordinates and the confidence degrees corresponding to the screened plurality of observation points.

[0081] In an implementable manner, the determining unit 1140 is specifically configured to: convert the pixel coordinates of each intersection point to obtain the world coordinates of each intersection point; calculate the offset of each intersection point on the fitting line according to the world coordinates of each intersection point; and take the intersection point with the largest offset and the intersection point with the smallest offset in the plurality of intersection points as two end points of the side edge of the obstacle.

[0082] In an implementable manner, the size estimation apparatus further includes a constructing unit 1150, which is configured to take one of the two end points as a base point, and construct a first ray and a second ray based on the base point in a world coordinate system, where the first ray and the second ray are respectively used to point to the height and width of the obstacle; the projecting unit 1130 is further configured to project the first ray and the second ray onto at least one image, so that the projections of the first ray and the second ray intersect with the top edge and the side edge of the two-dimensional bounding box on each image at a first intersection point and a second intersection point, respectively; and the determining unit 1140 is further configured to determine the width, the height and / or the center point of the obstacle according to the two end points, the first intersection point and the second intersection point on each image.

[0083] In an implementable manner, the constructing unit 1150 is specifically configured to: determine the heading of the obstacle according to the line connecting the two end points; convert the pixel coordinates of the base point to obtain the world coordinates of the base point; and construct the first ray and the second ray with the world coordinates of the base point as the starting point, where the first ray is perpendicular to the ground on which the obstacle is located and points to the top of the obstacle, and the second ray is perpendicular to the heading and the first ray and points to the side of the obstacle.

[0084] In an implementable manner, the determining unit 1140 is specifically configured to: determine a height set of the obstacle according to the base point and the first intersection point on each image, where the height set includes at least one height of the obstacle; determine a width set of the obstacle according to the base point and the second intersection point on each image, where the width set includes at least one width of the obstacle; determine a center point set of the obstacle according to the two end points, the first intersection point and the second intersection point on each image, where the center point set includes at least one center point of the obstacle; and take the average of the height set, the width set and the center point set respectively to obtain the height, the width and the center point of the obstacle.

[0085] In an embodiment, the present application also provides a size estimation device, which is shown in Figure 12The size estimation device can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, portable wearable devices, and servers, and the server can be implemented by a single server or a server cluster composed of multiple servers. The size estimation device in the embodiment shown in the figure can include a processor 1210 and a memory 1220. The processor 1210 and the memory 1220 are connected through a bus 1230. The processor 1210 is configured to execute a plurality of instructions; and the memory 1220 is configured to store a plurality of instructions, which are adapted to be loaded and executed by the processor 1210 to implement the size estimation method in the above embodiment.

[0086] The processor 1210 can be an electronic control unit (ECU), a central processing unit (CPU), a general-purpose processor, a coprocessor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The processor 1210 can also be a combination of computing functions, such as one or more microprocessor combinations, combinations of DSP and microprocessor, and the like. In the embodiment, the processor 1210 can adopt a single-chip microcomputer, and various control functions can be realized by programming the single-chip microcomputer, such as the functions of collecting, processing, and demodulating the capacity of the battery cell and the cumulative total amount of charging in the embodiment. The processor has the advantages of powerful computing capability and fast processing. Specifically, the processor 1210 is configured to perform the functions of the detection unit 1110, to detect at least one image of a pseudo three-dimensional bounding box including an obstacle, to obtain a plurality of observation points, wherein the plurality of observation points are used to identify the positions of the side bottom edges of the obstacle; to perform the functions of the fitting unit 1120, to fit a fitting line representing the side bottom edges of the obstacle in a world coordinate system according to the plurality of observation points; to perform the functions of the projection unit 1130, to project the fitting line into each image respectively, so that the projection of the fitting line intersects with the bottom edge perpendicular of the pseudo three-dimensional bounding box in a plurality of intersection points; and to perform the functions of the determination unit 1140, to determine two end points of the side bottom edges of the obstacle from the plurality of intersection points, and to determine the length of the obstacle according to the two end points.

[0087] In an implementable manner, the processor 1210 is specifically configured to: convert, according to pixel coordinates of a plurality of observation points, world coordinates of the observation points respectively; determine, according to a confidence rule, confidence degrees corresponding to the observation points, wherein the observation points include tire contact points and end points of side bottom edges of the pseudo three-dimensional bounding boxes, and the confidence degree of the tire contact point is greater than the confidence degree of the end point of the side bottom edge of the pseudo three-dimensional bounding box; and fit to obtain a fitting line according to the world coordinates and the confidence degrees corresponding to the observation points respectively.

[0088] In an implementable manner, the processor 1210 is further configured to: calculate, according to the world coordinates and the confidence degrees corresponding to the observation points respectively, a total error of fitting to the observation points and an error corresponding to each observation point; in a case where the total error is less than or equal to an error threshold, fit to obtain the fitting line according to the world coordinates and the confidence degrees corresponding to the observation points respectively; and in a case where the total error is greater than the error threshold, screen the observation points according to the errors corresponding to the observation points respectively, so that the total error of fitting to the screened observation points is less than or equal to the error threshold, and fit to obtain the fitting line according to the world coordinates and the confidence degrees corresponding to the screened observation points respectively.

[0089] In an implementable manner, the processor 1210 is specifically configured to: convert, according to pixel coordinates of the intersection points, world coordinates of the intersection points; calculate, according to the world coordinates of the intersection points, offset amounts of the intersection points on the fitting line; and take, as two end points of the side bottom edge of the obstacle, an intersection point with the maximum offset amount and an intersection point with the minimum offset amount in the intersection points.

[0090] In an implementable manner, the processor 1210 is further configured to perform the functions of the construction unit 1150, configured to take one of the two end points as a base point, and construct a first ray and a second ray based on the base point in a world coordinate system, wherein the first ray and the second ray are respectively used to point to the height and the width of the obstacle; further configured to project the first ray and the second ray onto at least one image, so that projections of the first ray and the second ray intersect with a top edge and a side edge of a two-dimensional bounding box on each image at a first intersection point and a second intersection point respectively; and further configured to determine the width, the height and / or the center point of the obstacle according to the two end points, the first intersection point and the second intersection point on each image.

[0091] In an implementable manner, the processor 1210 is specifically configured to: determine a heading of the obstacle according to a line connecting the two end points; convert, according to a pixel coordinate of the base point, a world coordinate of the base point; and construct the first ray and the second ray with the world coordinate of the base point as a starting point, wherein the first ray is perpendicular to a ground on which the obstacle is located and points to a top of the obstacle, and the second ray is perpendicular to the heading and the first ray and points to a side of the obstacle.

[0092] In an implementable mode, the processor 1210 is specifically configured to: determine a height set of the obstacle according to the base point and the first intersection point on each image, wherein the height set comprises at least one height of the obstacle; determine a width set of the obstacle according to the base point and the second intersection point on each image, wherein the width set comprises at least one width of the obstacle; determine a center point set of the obstacle according to the two end points, the first intersection point and the second intersection point on each image, wherein the center point set comprises at least one center point of the obstacle; and average the height set, the width set and the center point set respectively to obtain the height, the width and the center point of the obstacle.

[0093] Any combination of the above technical features in the above embodiments can be combined. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the description.

[0094] The above embodiments only express several implementation manners of the present application, the description is more specific and detailed, but it should not be understood as the limitation of the patent scope of the present application. It should be pointed out that, for those skilled in the art, some modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the patent protection scope of the present application should be subject to the appended claims.

Claims

1. A multi-camera based size estimation method, characterized in that, The method comprises the following steps: detecting at least one image of a pseudo three-dimensional bounding box including an obstacle to obtain a plurality of observation points, wherein the plurality of observation points are used to identify positions of a side bottom edge of the obstacle; fitting a fitting line for representing the side bottom edge of the obstacle in a world coordinate system according to the plurality of observation points; projecting the fitting line into each image respectively, so that the projection of the fitting line intersects with the vertical line of the bottom edge of the pseudo three-dimensional bounding box in the images to obtain a plurality of intersection points; determining two end points of the side bottom edge of the obstacle by determining the offset of the plurality of intersection points on the fitting line, and determining the length of the obstacle according to the two end points.

2. The method of claim 1, wherein, The step of fitting the fitting line for representing the side bottom edge of the obstacle in the world coordinate system according to the plurality of observation points comprises the following steps: respectively converting the pixel coordinates of the plurality of observation points into world coordinates of the plurality of observation points; determining the confidence degree corresponding to each observation point according to a confidence rule, wherein the observation points include a tire contact point and end points of the side bottom edge of the pseudo three-dimensional bounding box, and the confidence degree of the tire contact point is greater than the confidence degree of the end points of the side bottom edge of the pseudo three-dimensional bounding box; fitting the fitting line according to the world coordinates and the confidence degrees corresponding to the plurality of observation points.

3. The method of claim 2, wherein, The step of fitting the fitting line according to the world coordinates and the confidence degrees corresponding to the plurality of observation points comprises the following steps: calculating the total error of fitting the plurality of observation points and the error corresponding to each observation point according to the world coordinates and the confidence degrees corresponding to the plurality of observation points; fitting the fitting line according to the world coordinates and the confidence degrees corresponding to the plurality of observation points in the case that the total error is less than or equal to an error threshold; screening the plurality of observation points according to the error corresponding to each observation point in the case that the total error is greater than the error threshold, fitting the fitting line according to the world coordinates and the confidence degrees corresponding to the screened plurality of observation points in the case that the total error of fitting the screened plurality of observation points is less than or equal to the error threshold.

4. The method of claim 1, wherein, The step of determining the two end points of the side bottom edge of the obstacle by determining the offset of the plurality of intersection points on the fitting line comprises the following steps: converting the pixel coordinates of each intersection point into the world coordinates of each intersection point; calculating the offset of each intersection point on the fitting line according to the world coordinates of each intersection point; taking the intersection point with the maximum offset and the intersection point with the minimum offset as the two end points of the side bottom edge of the obstacle.

5. The method of claim 1, wherein, The method further comprises the following steps after the step of determining the two end points of the side bottom edge of the obstacle from the plurality of intersection points: taking one of the two end points as a base point, and constructing a first ray and a second ray in the world coordinate system based on the base point, wherein the first ray and the second ray are respectively used to point to the height and the width of the obstacle. projecting the first ray and the second ray onto the at least one image respectively, so that the projections of the first ray and the second ray intersect with a top side and a side side of a two-dimensional bounding box on each image respectively at a first intersection point and a second intersection point; determining a width, a height and / or a center point of the obstacle according to the two end points, the first intersection point and the second intersection point on the images.

6. The method of claim 5, wherein, The step of constructing the first ray and the second ray based on the base point in the world coordinate system comprises: determining a heading of the obstacle according to a line connecting the two end points; converting the pixel coordinates of the base point to obtain world coordinates of the base point; constructing the first ray and the second ray from the world coordinates of the base point, wherein the first ray is perpendicular to the ground on which the obstacle is located and points to the top of the obstacle, and the second ray is perpendicular to the heading and the first ray and points to the side of the obstacle.

7. The method of claim 5, wherein, The step of determining the width, the height and / or the center point of the obstacle according to the two end points, the first intersection point and the second intersection point on the images comprises: determining a height set of the obstacle according to the base point and the first intersection point on the images, wherein the height set comprises at least one height of the obstacle; determining a width set of the obstacle according to the base point and the second intersection point on the images, wherein the width set comprises at least one width of the obstacle; determining a center point set of the obstacle according to the two end points, the first intersection point and the second intersection point on the images, wherein the center point set comprises at least one center point of the obstacle; averaging the height set, the width set and the center point set respectively to obtain the height, the width and the center point of the obstacle.

8. A size estimation apparatus characterized by comprising: comprises: a detecting unit configured to detect at least one image comprising a pseudo three-dimensional bounding box of an obstacle to obtain a plurality of observation points, wherein the plurality of observation points are used to identify a position of a side bottom edge of the obstacle; a fitting unit configured to fit a fitting line representing the side bottom edge of the obstacle in a world coordinate system according to the plurality of observation points; a projecting unit configured to project the fitting line into each image respectively, so that the projection of the fitting line intersects with a perpendicular line of a bottom edge of the pseudo three-dimensional bounding box in the images at a plurality of intersection points; a determining unit configured to determine two end points of the side bottom edge of the obstacle by determining an offset of the plurality of intersection points on the fitting line, and determine a length of the obstacle according to the two end points.

9. A size estimation device, characterized by, The device comprises a processor and a memory connected through a bus; the processor is configured to execute a plurality of instructions; and the memory is configured to store the plurality of instructions, which are adapted to be loaded and executed by the processor to implement the size estimation method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a plurality of instructions, which are adapted to be loaded and executed by a processor to implement the size estimation method according to any one of claims 1-7.

Images