Method and apparatus for determining three-dimensional information of a detection object

Abstract

The present application provides a method and device for determining three-dimensional information of a detection object, relating to the field of intelligent driving and used for determining the three-dimensional information of the detection object. The method includes: obtaining an image to be detected; the image to be detected includes a first detection object; determining the boundary of the passable area of the first detection object and the ground contact line of the first detection object; the boundary of the passable area includes the boundary of the first detection object in the image to be detected; the ground contact line is the connection line of the intersection points of the first detection object and the ground; and determining the three-dimensional information of the first detection object according to the boundary of the passable area and the ground contact line, which can determine the three-dimensional information of the detection object through the picture obtained by a monocular camera, reduce the hardware cost of determining the three-dimensional information of the first detection object, and reduce the computation amount when the first vehicle processes the image.

Description

Technical Field

[0001] This application relates to the field of intelligent driving, and in particular, to a method and device for determining three-dimensional information of a detection object. Background Art

[0002] In an intelligent driving scenario, a moving vehicle (denoted as the first vehicle in this application) needs to detect the position of other surrounding vehicles (denoted as the second vehicle in this application) relative to the first vehicle in real time, the size of the second vehicle (the length and width of the vehicle), and the direction of the second vehicle (the vehicle orientation, driving direction, etc.), so that the first vehicle can plan its own driving route based on this information and anticipate and avoid potential harmful behaviors of other vehicles.

[0003] When the first vehicle determines the size, direction, and position of the second vehicle relative to the first vehicle, it needs to determine the three-dimensional (3D) information of the second vehicle. Currently, the first vehicle can determine the 3D information of the second vehicle through a binocular 3D method. However, the binocular 3D method requires a binocular camera, which is expensive; and the current algorithms used in the process of determining the 3D information of the second vehicle require a high image annotation cost and a large amount of calculation. Summary of the Invention

[0004] This application provides a method and device for determining three-dimensional information of a detection object, which solves the problems of high hardware requirements and large amounts of calculation data when determining the three-dimensional information of surrounding vehicles in the prior art.

[0005] To achieve the above object, this application adopts the following technical solutions:

[0006] In a first aspect, a method for determining three-dimensional information of a detection object is provided, including: the first vehicle acquires an image to be detected; the image to be detected includes a first detection object; the first vehicle determines the boundary of the passable area of the first detection object and the ground contact line of the first detection object; the boundary of the passable area includes the boundary of the first detection object in the image to be detected; the ground contact line is the connection line of the intersection points of the first detection object and the ground; the first vehicle determines the three-dimensional information of the first detection object based on the boundary of the passable area and the ground contact line.

[0007] Based on the above technical solution, in the method for determining three-dimensional information of a detection object provided by this application, the first vehicle can determine the boundary of the passable area and the ground contact line of the first detection object according to the acquired image to be detected. Further, the first vehicle determines the three-dimensional information represented by the first detection object in the two-dimensional image based on the boundary of the passable area and the ground contact line of the first detection object.

[0008] The above image to be detected can be a two-dimensional image collected by a monocular camera. In this way, the first vehicle can determine the three-dimensional information of the first detection object by collecting the image information of the first detection object through the monocular camera. Compared with the method in the prior art where the first vehicle needs to rely on a binocular camera to collect the image information of the first detection object to determine the three-dimensional information of the first detection object, in this application, the first vehicle uses a monocular camera to collect the image information of the first detection object to determine the three-dimensional information of the first detection object, which can greatly reduce the hardware cost of determining the three-dimensional information of the first detection object.

[0009] In addition, for the method provided in this application for determining the three-dimensional information of the detection object, the first vehicle only needs to mark the boundary of the passable area of the first detection object, and determine the ground contact line of the first detection object according to the boundary of the passable area of the first detection object. The first vehicle can determine the three-dimensional information represented by the first detection object in the two-dimensional image based on the boundary of the passable area of the first detection object, the ground contact line, and the visual relationship of the first detection object in the image to be detected, etc. Therefore, for the method provided in this application for determining the three-dimensional information of the detection object, there is no need for the first vehicle to perform other additional data annotation training, thereby reducing the computational amount of determining the three-dimensional information of the first detection object and reducing the Graphics Processing Unit (GPU) resources occupied by determining the three-dimensional information of the first detection object.

[0010] Combined with the first aspect, in a possible implementation manner, the three-dimensional information of the first detection object is used to determine at least one of the size, direction, and relative position of the first detection object.

[0011] For example, the first vehicle can convert the three-dimensional information represented by the first detection object in the image to be detected into the vehicle body coordinate system of the first vehicle, and determine the corresponding size, direction, and the position of the first detection object relative to the first vehicle in the vehicle body coordinate system of the first vehicle. In this way, the first vehicle can plan the driving route of the first vehicle by combining the corresponding size, direction, and the position of the first detection object relative to the first vehicle of multiple first detection objects around, so as to realize the intelligent driving of the first vehicle.

[0012] Combined with the first aspect, in a possible implementation manner, the boundary of the passable area of the first detection object includes multiple boundary points corresponding to the first identifier and boundary points corresponding to multiple second identifiers; the first identifier also corresponds to the first side of the first detection object, and the second identifier also corresponds to the second side of the first detection object; the first side and the second side are two intersecting sides of the first detection object.

[0013] Based on this, when the first vehicle uses a monocular camera to collect two sides of the first detection object (i.e., the two sides of the first detection object are included in the image to be detected), the first vehicle can respectively mark the boundaries of the passable areas of each side and assign different identifiers to the boundaries of the passable areas of different sides. The first vehicle can determine the boundaries of the passable areas corresponding to each side according to the different identifiers.

[0014] Combined with the first aspect, in a possible implementation, the ground contact line includes a first ground contact line and a second ground contact line; the first ground contact line is a ground contact line determined by fitting a plurality of boundary points corresponding to the first identifier; the second ground contact line is a ground contact line determined by fitting a plurality of boundary points corresponding to the second identifier.

[0015] Based on this, the first vehicle determines the ground contact line corresponding to each side shown in the image to be detected of the first detection object according to the boundaries of the passable areas corresponding to each side. Since the ground contact line is obtained by fitting the points on the boundaries of the passable areas corresponding to each side, the ground contact line corresponding to each side can represent a partial boundary of the projection of this side on the ground (i.e., the outermost part of the projection of each side on the ground).

[0016] Combined with the first aspect, in a possible implementation, the first boundary point is included among the plurality of boundary points corresponding to the first identifier; the first boundary point is the boundary point with the largest distance from the second ground contact line among the plurality of boundary points with the first identifier; the second boundary point is included among the plurality of boundary points corresponding to the second identifier; the second boundary point is the boundary point with the largest distance from the first ground contact line among the plurality of boundary points with the second identifier.

[0017] Based on this, the above-mentioned first boundary point can represent the point with the largest distance from the first vehicle in the first side of the first detection object, that is to say, the first boundary point can represent the vertex of the first side that is farthest from the first vehicle. The second boundary point can represent the point with the largest distance from the first vehicle in the second side of the first detection object, that is to say, the second boundary point can represent the vertex of the second side that is farthest from the first vehicle.

[0018] Combined with the first aspect, in a possible implementation, the three-dimensional information of the first detection object is determined according to three points and two lines corresponding to the first detection object; the first of the three points is the projection of the first boundary point on the ground; the second of the three points is the projection of the second boundary point on the ground; the third of the three points is the intersection point of the straight line passing through the second point and parallel to the second ground contact line and the first ground contact line; the first of the two lines is the connection line between the first point and the third point; the second of the two lines is the connection line between the second point and the third point.

[0019] In combination with the first aspect, in a possible implementation, determine the projection of the first boundary point on the ground as the first point; determine the projection of the second boundary point on the ground as the second point; determine the intersection point of the line passing through the projection of the second boundary point on the ground and parallel to the second ground contact line and the first ground contact line as the third point; determine the connection line between the first point and the third point as the first line; determine the connection line between the second point and the third point as the second line; determine the three-dimensional information of the first detection object according to the first point, the second point, the third point, the first line, and the second line.

[0020] Based on the above technical solution, the projection of the first boundary point on the ground can represent the point on the ground where the projection of the first side is farthest from the first vehicle, the projection of the second boundary point on the ground can represent the point on the ground where the projection of the second side is farthest from the second vehicle, and the third point can represent the intersection point of the projection of the first side on the ground and the projection of the second side on the ground. The first ground contact line can represent the direction of the projection of the first side on the ground. The second ground contact line can represent the direction of the projection of the second side on the ground. Therefore, the first vehicle can determine that the first line is the outermost frame line of the projection of the first side on the ground, and the second line is the outermost frame line of the projection of the second side on the ground.

[0021] Furthermore, the first vehicle can determine the direction of the first detection object according to the directions of the first line and / or the second line, and the positions of the first line and the second line in the image to be detected. The first vehicle can determine the size of the first detection object according to the lengths of the first line and the second line, and the positions of the first line and the second line in the image to be detected. The first vehicle can determine the position of the first detection object relative to the first vehicle according to the positions of the first line and the second line in the image to be detected.

[0022] In this way, the first vehicle only needs to project specific points on the first detection object according to the passable area boundary of the first detection object and the ground contact lines, and then can determine the three-dimensional information of the first detection object, greatly reducing the computational complexity of the first vehicle to determine the three-dimensional information of the first detection object.

[0023] In combination with the first aspect, in a possible implementation, the first vehicle determines the first point according to the first boundary point and the first ground contact line; the first vehicle determines the second point according to the first ground contact line, the second ground contact line, and the second boundary point; the third point is determined according to the first ground contact line, the second ground contact line, and the second point.

[0024] Based on this, the first vehicle can determine the vertices of the projection of the first detection object on the ground according to the passable area boundary and the ground contact lines.

[0025] In combination with the first aspect, in a possible implementation, the first vehicle determines a first straight line; the first straight line is a straight line passing through the first boundary point in the image to be detected and perpendicular to the horizon line; the first vehicle determines the intersection point of the first straight line and the first ground contact line as the first point.

[0026] Based on this, the first vehicle can quickly and accurately determine the projection of the first boundary point on the ground according to the visual relationship between the first ground contact line and the first boundary point in the image to be detected. By determining the projection of the first boundary point on the ground in this way, the first vehicle can further reduce the computational complexity of determining the three-dimensional information of the first detection object.

[0027] In combination with the first aspect, in a possible implementation, the first vehicle determines a second straight line and a third straight line; wherein, the second straight line is a straight line passing through the intersection point of the first ground contact line and the horizon line and the end point of the second ground contact line far from the first ground contact line in the image to be detected; the third straight line is a straight line passing through the second boundary point in the image to be detected and perpendicular to the horizon line; the first vehicle determines the intersection point of the second straight line and the third straight line as the second point.

[0028] Based on this, the first vehicle can quickly and accurately determine the projection of the first boundary point on the ground according to the visual relationship between the first ground contact line, the second ground contact line, and the second boundary point in the image to be detected. By determining the projection of the second boundary point on the ground in this way, the first vehicle can further reduce the computational complexity of determining the three-dimensional information of the first detection object.

[0029] In combination with the first aspect, in a possible implementation, the first vehicle determines a fourth straight line; the fourth straight line is a straight line passing through the second point in the image to be detected and parallel to the second ground contact line; the first vehicle determines the intersection point of the fourth straight line and the first ground contact line as the third point.

[0030] Based on this, the first vehicle can quickly and accurately determine the intersection point of the projections of the first side and the second side on the ground according to the visual relationship between the first ground contact line, the second ground contact line, and the second boundary point in the image to be detected. By determining the intersection point of the projections of the first side and the second side on the ground in this way, the first vehicle can further reduce the computational complexity of determining the three-dimensional information of the first detection object.

[0031] In combination with the first aspect, in a possible implementation, the passable area boundary of the first detection object includes a plurality of boundary points corresponding to the third identifier; the third identifier also corresponds to the third side of the first detection object.

[0032] Based on this, when the first vehicle uses a monocular camera to collect one side of the first detection object (i.e., the detected image includes one side of the first detection object), the first vehicle can mark the boundary of the passable area of this side and assign a corresponding identifier to it. The first vehicle can determine the boundary of the passable area corresponding to this side based on the boundary points of the passable area with this identifier.

[0033] Combined with the first aspect, in a possible implementation, the ground contact line of the first detection object includes a third ground contact line; the third ground contact line is the ground contact line determined by fitting multiple boundary points corresponding to the third identifier.

[0034] Based on this, the first vehicle can determine the ground contact line of this side according to the boundary of the passable area of this side. Since this ground contact line is obtained by fitting the points on the boundary of the passable area corresponding to this side, this ground contact line can represent a partial boundary of the projection of this side on the ground (i.e., the outermost edge part of the projection of each side on the ground).

[0035] Combined with the first aspect, in a possible implementation, among the multiple boundary points with the third identifier, there are a third boundary point and a fourth boundary point; the third boundary point is the point among the multiple boundary points with the third identifier that is farthest from one end of the third ground contact line; the fourth boundary point is the point among the multiple boundary points with the third identifier that is farthest from the other end of the third ground contact line.

[0036] Based on this, the above-mentioned third boundary point and fourth boundary point can represent two vertices of the third side of the first detection object.

[0037] Combined with the first aspect, in a possible implementation, the three-dimensional information of the first detection object is determined according to two points and a line corresponding to the first detection object; the first of the two points is the projection of the third boundary point on the ground; the second of the two points is the projection of the fourth boundary point on the ground.

[0038] Combined with the first aspect, in a possible implementation, determine the projection of the third boundary point on the ground as the first point; determine the projection of the fourth boundary point on the ground as the second point; determine the connection line between the first point and the second point as the first line; according to the first point, the second point, and the first line, determine the three-dimensional information of the first detection object.

[0039] Based on the above technical solution, the projection of the third boundary point on the ground can represent a vertex of the projection of the third side on the ground. The projection of the fourth boundary point on the ground can represent another vertex line of the projection of the third side on the ground. The connection line (the first line) between the projection of the third boundary point on the ground and the projection of the fourth boundary point on the ground can represent the outermost frame line of the projection of the third side on the ground.

[0040] Furthermore, the first vehicle can determine the direction of the first detection object based on the direction of the first line and the position of the first line in the image to be detected. The first vehicle can determine the size of the first detection object based on the length of the first line and the position of the first line in the image to be detected. The first vehicle can determine the position of the first detection object relative to the first vehicle based on the position of the first line in the image to be detected.

[0041] In this way, the first vehicle only needs to project specific points on the first detection object according to the passable area boundary of the first detection object and the ground contact line, and then it can determine the three-dimensional information of the first detection object, greatly reducing the computational complexity of the first vehicle to determine the three-dimensional information of the first detection object.

[0042] Combined with the first aspect, in a possible implementation manner, determining the three-dimensional information of the first detection object according to the passable area boundary and the ground contact line includes: determining the first point according to the third boundary point and the third ground contact line; determining the second point according to the fourth boundary point and the third ground contact line; determining a line according to the first point and the second point.

[0043] Based on this, the first vehicle determines the information of the vertices of the projection of the first detection object on the ground and the outermost boundary of the projection of the first detection object on the ground. The first vehicle can determine the size and direction of the first detection object according to the vertices of the projection of the first detection object on the ground and the outermost boundary. For example, when the first detection object is the second vehicle, the vertices and the outermost boundary line of the projection of the second vehicle on the ground can represent the size of the second vehicle, and the direction of the outermost boundary line can represent the direction of the second vehicle.

[0044] Combined with the first aspect, in a possible implementation manner, determining the first point according to the third boundary point and the third ground contact line includes: determining the fifth straight line; the fifth straight line is a straight line passing through the third boundary point in the image to be detected and perpendicular to the horizon line; determining the intersection point of the fifth straight line and the third ground contact line as the first point.

[0045] Based on this, the first vehicle can quickly and accurately determine the projection of the third boundary point on the ground according to the visual relationship between the third ground contact line and the third boundary point in the image to be detected. The first vehicle determining the projection of the third boundary point on the ground in this way can further reduce the computational complexity of the first vehicle to determine the three-dimensional information of the first detection object.

[0046] Combined with the first aspect, in a possible implementation manner, determining the second point according to the fourth boundary point and the third ground contact line includes: determining the sixth straight line; the sixth straight line is a straight line passing through the fourth boundary point in the image to be detected and perpendicular to the horizon line; determining the intersection point of the sixth straight line and the third ground contact line as the second point.

[0047] Based on this, the first vehicle can quickly and accurately determine the projection of the fourth boundary point on the ground according to the visual relationship between the third ground contact line and the fourth boundary point in the image to be detected. By determining the projection of the fourth boundary point on the ground in this way, the first vehicle can further reduce the computational complexity of determining the three-dimensional information of the first detection object.

[0048] Combined with the first aspect, in a possible implementation manner, the method further includes: inputting the three-dimensional information of the first detection object into the vehicle body coordinate system to determine at least one of the size, direction, and relative position of the first detection object.

[0049] Based on this, by inputting the three-dimensional information of the first detection object into the vehicle body coordinate system, the first vehicle can determine the size, direction, and position of the first detection object relative to the first vehicle in the three-dimensional space.

[0050] In a second aspect, there is provided a device for determining the three-dimensional information of a detection object, characterized by including: a communication unit and a processing unit; the communication unit is used to obtain an image to be detected; the image to be detected includes a first detection object; the processing unit is used to determine the passable area boundary of the first detection object and the ground contact line of the first detection object; the passable area boundary includes the boundary of the first detection object in the image to be detected; the ground contact line is the connection line of the intersection points of the first detection object and the ground; the processing unit is further used to determine the three-dimensional information of the first detection object according to the passable area boundary and the ground contact line.

[0051] Combined with the second aspect, in a possible implementation manner, the three-dimensional information of the first detection object is used to determine at least one of the size, direction, and relative position of the first detection object.

[0052] Combined with the second aspect, in a possible implementation manner, the passable area boundary of the first detection object includes a plurality of boundary points corresponding to a first identifier and boundary points corresponding to a plurality of second identifiers; the first identifier also corresponds to a first side surface of the first detection object, and the second identifier also corresponds to a second side surface of the first detection object; the first side surface and the second side surface are two intersecting side surfaces of the first detection object.

[0053] Combined with the second aspect, in a possible implementation manner, the ground contact line includes a first ground contact line and a second ground contact line; the first ground contact line is the ground contact line determined by fitting a plurality of boundary points corresponding to the first identifier; the second ground contact line is the ground contact line determined by fitting a plurality of boundary points corresponding to the second identifier.

[0054] In combination with the second aspect, in a possible implementation manner, among the multiple boundary points corresponding to the first identifier, there is a first boundary point; the first boundary point is the boundary point with the largest distance from the second ground contact line among the multiple boundary points with the first identifier.

[0055] Among the multiple boundary points corresponding to the second identifier, there is a second boundary point; the second boundary point is the boundary point with the largest distance from the first ground contact line among the multiple boundary points with the second identifier.

[0056] In combination with the second aspect, in a possible implementation manner, the three-dimensional information of the first detection object is determined according to three points and two lines corresponding to the first detection object; among them, the first of the three points is the projection of the first boundary point on the ground; the second of the three points is the projection of the second boundary point on the ground; the third of the three points is the intersection point of the straight line passing through the second point and parallel to the second ground contact line and the first ground contact line; the first of the two lines is the connection line between the first point and the third point; the second of the two lines is the connection line between the second point and the third point.

[0057] In combination with the second aspect, in a possible implementation manner, the processing unit is specifically configured to: determine the projection of the first boundary point on the ground as the first point; determine the projection of the second boundary point on the ground as the second point; determine the intersection point of the straight line passing through the projection of the second boundary point on the ground and parallel to the second ground contact line and the first ground contact line as the third point; determine the connection line between the first point and the third point as the first line; determine the connection line between the second point and the third point as the second line; determine the three-dimensional information of the first detection object according to the first point, the second point, the third point, the first line, and the second line.

[0058] In combination with the second aspect, in a possible implementation manner, the processing unit is specifically configured to: determine the first point according to the first boundary point and the first ground contact line; determine the second point according to the first ground contact line, the second ground contact line, and the second boundary point; determine the third point according to the first ground contact line, the second ground contact line, and the second point.

[0059] In combination with the second aspect, in a possible implementation manner, the processing unit is specifically configured to determine a first straight line; the first straight line is a straight line passing through the first boundary point in the image to be detected and perpendicular to the horizon line; determine the intersection point of the first straight line and the first ground contact line as the first point.

[0060] In combination with the second aspect, in a possible implementation manner, the processing unit is specifically configured to: determine a second straight line and a third straight line; wherein, the second straight line is a straight line in the image to be detected that passes through the intersection point of the first ground contact line and the horizon line, and the end point of the second ground contact line that is far from the first ground contact line; the third straight line is a straight line in the image to be detected that passes through the second boundary point and is perpendicular to the horizon line; determine the intersection point of the second straight line and the third straight line as the second point.

[0061] In combination with the second aspect, in a possible implementation manner, the processing unit is specifically configured to: determine a fourth straight line; the fourth straight line is a straight line in the image to be detected that passes through the second point and is parallel to the second ground contact line; determine the intersection point of the fourth straight line and the first ground contact line as the third point.

[0062] In combination with the second aspect, in a possible implementation manner, the passable area boundary of the first detection object includes multiple boundary points corresponding to the third identifier; the third identifier also corresponds to the third side of the first detection object.

[0063] In combination with the second aspect, in a possible implementation manner, the ground contact lines of the first detection object include a third ground contact line; the third ground contact line is a ground contact line determined by fitting multiple boundary points corresponding to the third identifier.

[0064] In combination with the second aspect, in a possible implementation manner, among the multiple boundary points with the third identifier, there are a third boundary point and a fourth boundary point; the third boundary point is the point that is farthest from one end of the third ground contact line among the multiple boundary points with the third identifier; the fourth boundary point is the point that is farthest from the other end of the third ground contact line among the multiple boundary points with the third identifier.

[0065] In combination with the second aspect, in a possible implementation manner, the three-dimensional information of the first detection object is determined according to two points and a line corresponding to the first detection object; the first point among the two points is the projection of the third boundary point on the ground; the second point among the two points is the projection of the fourth boundary point on the ground.

[0066] In combination with the second aspect, in a possible implementation manner, the processing unit is specifically configured to: determine the projection of the third boundary point on the ground as the first point; determine the projection of the fourth boundary point on the ground as the second point; determine the connection line between the first point and the second point as the first line; determine the three-dimensional information of the first detection object according to the first point, the second point, and the first line.

[0067] In combination with the second aspect, in a possible implementation manner, the processing unit is specifically configured to: determine the first point according to the third boundary point and the third ground contact line; determine the second point according to the fourth boundary point and the third ground contact line; determine a line according to the first point and the second point.

[0068] In combination with the second aspect, in a possible implementation, the processing unit is specifically configured to: determine a fifth straight line; the fifth straight line is a straight line passing through a third boundary point in the image to be detected and perpendicular to the horizon line; determine the intersection point of the fifth straight line and the third ground contact line as the first point.

[0069] In combination with the second aspect, in a possible implementation, the processing unit is specifically configured to: determine a sixth straight line; the sixth straight line is a straight line passing through a fourth boundary point in the image to be detected and perpendicular to the horizon line; determine the intersection point of the sixth straight line and the third ground contact line as the second point.

[0070] In combination with the second aspect, in a possible implementation, the processing unit is further configured to: input the three-dimensional information of the first detection object into the vehicle body coordinate system, and determine at least one of the size, direction, and relative position of the first detection object.

[0071] In a third aspect, the present application provides a device for determining the three-dimensional information of a detection object, including: a processor and a memory, wherein the memory is used to store computer programs and instructions, and the processor is used to execute the computer programs and instructions to implement the method described in the first aspect and any possible implementation manner of the first aspect. The device for determining the three-dimensional information of the detection object may be the first vehicle or a chip in the first vehicle.

[0072] In a fourth aspect, the present application provides an intelligent vehicle, including: a vehicle body, a monocular camera, and a device for determining the three-dimensional information of a detection object described in the second aspect and any possible implementation manner of the second aspect. The monocular camera is used to collect an image to be detected; the device for determining the three-dimensional information of the detection object is used to execute the method for determining the three-dimensional information of the detection object described in the first aspect and any possible implementation manner of the first aspect to determine the three-dimensional information of the detection object.

[0073] In combination with the fourth aspect, in a possible implementation, the intelligent vehicle further includes a display screen; the display screen is used to display the three-dimensional information of the detection object.

[0074] In a fifth aspect, the present application provides an advanced driver assistance system (ADAS), including a device for determining the three-dimensional information of a detection object described in the second aspect and any possible implementation manner of the second aspect. The device for determining the three-dimensional information of the detection object is used to execute the method for determining the three-dimensional information of the detection object described in the first aspect and any possible implementation manner of the first aspect to determine the three-dimensional information of the detection object.

[0075] Sixthly, the present application provides a computer-readable storage medium storing instructions, which, when running on a computer, cause the computer to execute the methods described in the first aspect and any possible implementation manner of the first aspect.

[0076] Seventhly, the present application provides a computer program product containing instructions, which, when running on a computer, cause the computer to execute the methods described in the first aspect and any possible implementation manner of the first aspect.

[0077] It should be understood that the description of technical features, technical solutions, beneficial effects or similar languages in the present application does not imply that all features and advantages can be achieved in any single embodiment. On the contrary, it can be understood that the description of features or beneficial effects means that at least one embodiment includes specific technical features, technical solutions or beneficial effects. Therefore, the description of technical features, technical solutions or beneficial effects in this specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and beneficial effects described in this embodiment can be combined in any appropriate manner. Those skilled in the art will understand that an embodiment can be implemented without one or more specific technical features, technical solutions or beneficial effects of a specific embodiment. In other embodiments, additional technical features and beneficial effects can also be identified in specific embodiments that do not embody all embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 Structural schematic of a vehicle provided by an embodiment of the present application Figure 1 ;

[0079] Figure 2 System architecture diagram of an ADAS system provided by an embodiment of the present application;

[0080] Figure 3 Structural schematic of a computer system provided by an embodiment of the present application;

[0081] Figure 4 Application schematic of a cloud-side instruction autonomous driving vehicle provided by an embodiment of the present application Figure 1 ;

[0082] Figure 5 Application schematic of a cloud-side instruction autonomous driving vehicle provided by an embodiment of the present application Figure 2 ;

[0083] Figure 6 Structural schematic of a computer program product provided by an embodiment of the present application;

[0084] Figure 7Schematic flowchart of a method for determining three-dimensional information of a detection object provided by an embodiment of the present application;

[0085] Figure 8a Schematic diagram of a first detection object provided by an embodiment of the present application;

[0086] Figure 8b Schematic diagram of another first detection object provided by an embodiment of the present application;

[0087] Figure 9 Schematic flowchart of another method for determining three-dimensional information of a detection object provided by an embodiment of the present application;

[0088] Figure 10 Schematic flowchart of another method for determining three-dimensional information of a detection object provided by an embodiment of the present application;

[0089] Figure 11 Schematic diagram of three-dimensional information of a first detection object provided by an embodiment of the present application;

[0090] Figure 12 Schematic flowchart of another method for determining three-dimensional information of a detection object provided by an embodiment of the present application;

[0091] Figure 13 Schematic diagram of three-dimensional information of another first detection object provided by an embodiment of the present application;

[0092] Figure 14 Schematic structural diagram of a device for determining three-dimensional information of a detection object provided by an embodiment of the present application. Detailed implementation manners

[0093] In the description of the present application, unless otherwise specified, " / " means "or". For example, A / B may represent A or B. The "and / or" herein is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, "at least one" means one or more, and "a plurality" means two or more. The terms such as "first" and "second" do not limit the quantity and execution order, and the terms such as "first" and "second" do not necessarily mean different.

[0094] It should be noted that in the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly, using words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0095] An embodiment of the present application provides a method and apparatus for determining three-dimensional information of a detection object. This method is applied in a vehicle or in other devices (such as a cloud server, a mobile terminal, etc.) having the function of controlling a vehicle. The vehicle or other devices can implement the method for determining three-dimensional information of a detection object provided by the embodiments of the present application through components (including hardware and software) included therein. The vehicle determines the three-dimensional information (size, direction, relative position) of the detection object based on the to-be-detected image collected by the image acquisition device, so that the vehicle can plan the driving path of the vehicle according to these three-dimensional information of the detection objects.

[0096] Figure 1 It is a functional block diagram of vehicle 100 provided by an embodiment of the present application. The vehicle 100 can be an intelligent vehicle. In one embodiment, the vehicle 100 determines the three-dimensional information of the detection object based on the to-be-detected image collected by the image acquisition device, so that the vehicle can plan the driving path of the vehicle according to these three-dimensional information of the detection objects.

[0097] The vehicle 100 may include various subsystems, such as a propulsion system 110, a sensor system 120, a control system 130, one or more peripheral devices 140, as well as a power source 150, a computer system 160, and a user interface 170. Optionally, the vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. Additionally, each subsystem and component of the vehicle 100 can be interconnected by wire or wirelessly.

[0098] The propulsion system 110 may include components that provide powered movement for the vehicle 100. In one embodiment, the propulsion system 110 may include an engine 111, a transmission 112, an energy source 113, and wheels 114. The engine 111 can be an internal combustion engine, an electric motor, an air compression engine, or other types of engine combinations, such as a hybrid engine composed of a gasoline engine and an electric motor, or a hybrid engine composed of an internal combustion engine and an air compression engine. The engine 111 converts the energy source 113 into mechanical energy.

[0099] Examples of the energy source 113 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other power sources. The energy source 113 can also provide energy for other systems of the vehicle 100.

[0100] The transmission 112 can transmit the mechanical power from the engine 111 to the wheels 114. The transmission 112 may include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission 112 may also include other devices, such as a clutch. Among them, the drive shaft may include one or more shafts that can be coupled to one or more wheels 114.

[0101] The sensor system 120 may include several sensors that sense information about the environment around the vehicle 100. For example, the sensor system 120 may include a positioning system 121 (the positioning system may be a global positioning system (GPS), or it may be a Beidou system or other positioning systems), an inertial measurement unit (IMU) 122, a radar 123, a lidar 124, and a camera 125. The sensor system 120 may also include sensors that monitor the internal systems of the vehicle 100 (such as an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.). Such detection and identification are key functions for the safe operation of the autonomous driving of the vehicle 100.

[0102] The positioning system 121 can be used to estimate the geographical location of the vehicle 100. The IMU 122 is used to sense changes in the position and orientation of the vehicle 100 based on inertial acceleration. In one embodiment, the IMU 122 can be a combination of an accelerometer and a gyroscope.

[0103] The radar 123 can use radio signals to sense objects within the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing objects, the radar 123 can also be used to sense the speed and / or forward direction of the objects.

[0104] The lidar 124 can use lasers to sense objects in the environment where the vehicle 100 is located. In some embodiments, the lidar 124 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components.

[0105] The camera 125 can be used to capture multiple images of the surrounding environment of the vehicle 100, as well as multiple images inside the vehicle cockpit. The camera 125 can be a static camera or a video camera. The control system 130 can control the operation of the vehicle 100 and its components. The control system 130 may include various elements, including a steering system 131, a throttle 132, a braking unit 133, a computer vision system 134, a route control system 135, and an obstacle avoidance system 136.

[0106] The steering system 131 can be operated to adjust the forward direction of the vehicle 100. For example, in one embodiment, it can be a steering wheel system.

[0107] The throttle 132 is used to control the operating speed of the engine 111 and thus control the speed of the vehicle 100.

[0108] The braking unit 133 is used to control the deceleration of the vehicle 100. The braking unit 133 can use frictional force to slow down the wheels 114. In other embodiments, the braking unit 133 can convert the kinetic energy of the wheels 114 into electric current. The braking unit 133 can also take other forms to slow down the rotational speed of the wheels 114 so as to control the speed of the vehicle 100.

[0109] The computer vision system 134 can operate to process and analyze the images captured by the camera 125 to identify objects and / or features in the surrounding environment of the vehicle 100 and the limb features and facial features of the driver in the vehicle cockpit. The objects and / or features can include traffic signals, road conditions, and obstacles, and the limb features and facial features of the driver include the driver's behavior, line of sight, expression, etc. The computer vision system 134 can use object recognition algorithms, structure from motion (SFM) algorithms, video tracking, and other computer vision techniques. In some embodiments, the computer vision system 134 can be used for mapping the environment, tracking objects, estimating the speed of objects, determining driver behavior, face recognition, and so on.

[0110] The route control system 135 is used to determine the driving route of the vehicle 100. In some embodiments, the route control system 135 can combine data from sensors, the positioning system 121, and one or more pre - determined maps to determine the driving route for the vehicle 100.

[0111] The obstacle avoidance system 136 is used to identify, evaluate, and avoid or otherwise cross potential obstacles in the environment of the vehicle 100.

[0112] Of course, in one example, the control system 130 can add components not shown above; or replace some of the above - shown components with other components; or can also reduce some of the above - shown components.

[0113] The vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users through the peripheral device 140. The peripheral device 140 can include a wireless communication system 141, an on - vehicle computer 142, a microphone 143, and / or a speaker 144.

[0114] In some embodiments, the peripheral device 140 provides a means for a user of the vehicle 100 to interact with the user interface 170. For example, the in-vehicle computer 142 can provide information to the user of the vehicle 100. The user interface 170 can also operate the in-vehicle computer 142 to receive user input. The in-vehicle computer 142 can be operated via a touch screen. In other cases, the peripheral device 140 can provide a means for the vehicle 100 to communicate with other devices located within the vehicle. For example, the microphone 143 can receive audio from the user of the vehicle 100 (e.g., voice commands or other audio inputs). Similarly, the speaker 144 can output audio to the user of the vehicle 100.

[0115] The wireless communication system 141 can wirelessly communicate with one or more devices directly or via a communication network. For example, the wireless communication system 141 can use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or use 4G cellular communication, such as LTE, or use 5G cellular communication. The wireless communication system 141 can utilize WiFi to communicate with a wireless local area network (WLAN). In some embodiments, the wireless communication system 141 can utilize an infrared link, Bluetooth, or ZigBee to communicate directly with a device. The wireless communication system 141 can also communicate with a device using other wireless protocols. For example, various vehicle communication systems. The wireless communication system 141 can include one or more dedicated short-range communications (DSRC) devices.

[0116] The power source 150 can supply power to various components of the vehicle 100. In one embodiment, the power source 150 can be a rechargeable lithium-ion or lead-acid battery. One or more battery packs of such a battery can be configured as a power source to supply power to various components of the vehicle 100. In some embodiments, the power source 150 and the energy source 113 can be implemented together, such as in a pure electric vehicle or a hybrid electric vehicle in a new energy vehicle, etc.

[0117] Some or all of the functions of the vehicle 100 are controlled by the computer system 160. The computer system 160 can include at least one processor 161 that executes instructions 1621 stored in a non-transitory computer-readable medium such as a data storage device 162. The computer system 160 can also be multiple computing devices that control individual components or subsystems of the vehicle 100 in a distributed manner.

[0118] The processor 161 can be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor can be a special-purpose device such as an application specific integrated circuit (ASIC) or other hardware-based processor. Although Figure 1 The functional diagram shows a processor, a memory, and other components within the same physical enclosure, but those of ordinary skill in the art should understand that the processor, computer system, or memory can actually include multiple processors, computer systems, or memories that can be stored within the same physical enclosure, or include multiple processors, computer systems, or memories that may not be stored within the same physical enclosure. For example, the memory can be a hard disk drive or other storage medium located in a different physical enclosure. Thus, references to a processor or computer system will be understood to include references to a collection of processors or computer systems or memories that can operate in parallel, or a collection of processors or computer systems or memories that may not operate in parallel. Instead of using a single processor to perform the steps described herein, some components such as the steering component and the deceleration component can each have its own processor that only performs calculations related to component-specific functions.

[0119] In various aspects described herein, the processor can be located in a device that is remote from the vehicle and communicates wirelessly with the vehicle. In other aspects, some of the processes described herein are executed on a processor disposed within the vehicle while others are executed by a remote processor, including taking the necessary steps to perform a single maneuver.

[0120] In some embodiments, the data storage device 162 can contain instructions 1621 (e.g., program logic) that can be executed by the processor 161 to perform various functions of the vehicle 100, including all or part of the functions described above. The data storage device 162 can also contain additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the propulsion system 110, the sensor system 120, the control system 130, and the peripheral devices 140.

[0121] In addition to the instructions 1621, the data storage device 162 can also store data, such as road maps, route information, the position, direction, speed of the vehicle, and other such vehicle data, as well as other information. Such information can be used by the vehicle 100 and the computer system 160 during operation of the vehicle 100 in autonomous, semi-autonomous, and / or manual modes.

[0122] For example, in a possible embodiment, the data storage device 162 may obtain obstacle information in the surrounding environment acquired by the vehicle based on the sensors in the sensor system 120, such as the positions of obstacles such as other vehicles, road edges, and green belts, the distances between the obstacles and the vehicle, and the distances between the obstacles. The data storage device 162 may also obtain environmental information from the sensor system 120 or other components of the vehicle 100. The environmental information may, for example, indicate whether there are green belts, lanes, pedestrians, etc. near the current environment of the vehicle, or whether there are green belts, pedestrians, etc. near the current environment of the vehicle calculated by the vehicle through a machine learning algorithm. In addition to the above, the data storage device 162 may also store the state information of the vehicle itself and the state information of other vehicles that interact with the vehicle. The state information of the vehicle includes, but is not limited to, the position, speed, acceleration, heading angle, etc. of the vehicle. In this way, the processor 161 may obtain this information from the data storage device 162 and determine the passable area of the vehicle based on the environmental information of the vehicle's environment, the state information of the vehicle itself, the state information of other vehicles, etc., and determine the final driving strategy based on the passable area to control the vehicle 100 for autonomous driving.

[0123] The user interface 170 is configured to provide information to or receive information from the user of the vehicle 100. Optionally, the user interface 170 may interact with one or more input / output devices within the set of peripheral devices 140, such as one or more of the wireless communication system 141, the in-vehicle computer 142, the microphone 143, and the speaker 144.

[0124] The computer system 160 may control the vehicle 100 based on information obtained from various subsystems (e.g., the propulsion system 110, the sensor system 120, and the control system 130) and information received from the user interface 170. For example, the computer system 160 may control the steering system 131 to change the forward direction of the vehicle according to the information from the control system 130, so as to avoid obstacles detected by the sensor system 120 and the obstacle avoidance system 136. In some embodiments, the computer system 160 may control many aspects of the vehicle 100 and its subsystems.

[0125] Optionally, one or more of the above components may be separately installed or associated with the vehicle 100. For example, the data storage device 162 may exist partially or completely separately from the vehicle 100. The above components may be coupled together by wired and / or wireless means for communication.

[0126] Optionally, the above components are only an example. In actual applications, the components in each of the above modules may be added or deleted according to actual needs. Figure 1 It should not be construed as a limitation to the embodiments of the present application.

[0127] An autonomous vehicle moving on a road, such as vehicle 100 above, can determine an adjustment instruction for the current speed based on other vehicles within its surrounding environment. Among them, the objects within the surrounding environment of vehicle 100 can be traffic control devices, or other types of objects such as green belts. In some examples, each object within the surrounding environment can be considered independently, and based on the respective characteristics of the object, such as its current speed, acceleration, distance from the vehicle, etc., an adjustment instruction for the speed of vehicle 100 can be determined.

[0128] Optionally, vehicle 100, which is an autonomous vehicle, or a computer device associated with it (such as Figure 1 computer system 160, computer vision system 134, data storage device 162) can, based on the identified measurement data, obtain the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.), and determine the relative position of obstacles in the surrounding environment with respect to the vehicle at the current moment. Optionally, the boundaries of the passable areas formed by each obstacle are dependent on each other. Therefore, all the obtained measurement data can also be used together to determine the boundaries of the passable area of the vehicle, and the actually impassable areas in the passable area are removed. Vehicle 100 can adjust its driving strategy based on the detected passable area of the vehicle. In other words, an autonomous vehicle can determine what stable state (e.g., accelerating, decelerating, turning, or stopping, etc.) the vehicle needs to adjust to based on the detected passable area of the vehicle. In this process, other factors can also be considered to determine the adjustment instruction for the speed of vehicle 100, such as the lateral position of vehicle 100 on the road being traveled, the curvature of the road, the proximity of static and dynamic objects, etc.

[0129] In addition to providing an instruction to adjust the speed of the autonomous vehicle, the computer device can also provide an instruction to modify the steering angle of vehicle 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from nearby objects (such as a sedan in an adjacent lane).

[0130] The above-mentioned vehicle 100 can be a sedan, a truck, a motorcycle, a bus, a boat, an airplane, a helicopter, a lawn mower, a recreational vehicle, a playground vehicle, construction equipment, a tram, a golf cart, a train, and a trolley, etc., and the embodiments of the present application do not make special limitations.

[0131] In some other embodiments of the present application, the autonomous vehicle can also include a hardware structure and / or a software module, and implement the above-mentioned various functions in the form of a hardware structure, a software module, or a combination of a hardware structure and a software module. Whether a certain function among the above-mentioned various functions is executed in the form of a hardware structure, a software module, or a combination of a hardware structure and a software module depends on the specific application and design constraints of the technical solution.

[0132] In one implementation, referring to Figure 2 , the method for determining the three-dimensional information of the detection object provided by the embodiments of the present application is applied to the ADAS system 200 as shown in Figure 2 . As shown in Figure 2 , the ADAS system 200 includes a hardware system 201, a perception fusion system 202, a planning system 203, and a control system 204.

[0133] Among them, the hardware system 201 is used to collect road information, vehicle information, obstacle information, etc. around the first vehicle. The commonly used hardware system 201 mainly includes cameras, video capture cards, etc. In the embodiments of the present application, the hardware system 201 includes a monocular camera.

[0134] The perception fusion system 202 is used to process the image information collected by the hardware system 201 to determine the target information (including vehicle information, pedestrian information, traffic light information, obstacle information, etc.) around the first vehicle, and the road structure information (including lane line information, curb information, etc.).

[0135] The planning system 203 is used to plan the driving route, driving speed, etc. of the first vehicle according to the target information and the road structure information, and generate planning information.

[0136] The control system 204 is used to convert the planning information generated by the planning system 203 into control information, and send the control information to the first vehicle, so that the first vehicle travels along the driving route and driving speed planned by the planning system 30 according to the control information.

[0137] The vehicle-mounted communication module 205 ( Figure 2 not shown in the figure) is used for information interaction between the vehicle itself and other vehicles.

[0138] The storage component 206 ( Figure 2 not shown in the figure) is used to store the executable codes of the above-mentioned various modules, and running these executable codes can implement part or all of the method processes of the embodiments of the present application.

[0139] In a possible implementation manner of the embodiments of the present application, as shown in Figure 3 : Figure 1The computer system 160 shown includes a processor 301 coupled to a system bus 302. The processor 301 can be one or more processors, and each processor can include one or more processor cores. A video adapter 303 can drive a display 324, which is coupled to the system bus 302. The system bus 302 is coupled to an input / output (I / O) bus 305 via a bus bridge 304. An I / O interface 306 is coupled to the I / O bus 305, and the I / O interface 306 communicates with various I / O devices, such as an input device 307 (e.g., keyboard, mouse, touch screen, etc.), a media tray 308 (e.g., CD-ROM, multimedia interface, etc.), a transceiver 309 (which can send and / or receive radio communication signals), a camera 310 (which can capture static and dynamic digital video images), and an external universal serial bus (USB) port 311. Optionally, the interface connected to the I / O interface 306 can be a USB interface.

[0140] Among them, the processor 301 can be any conventional processor, including a reduced instruction set computer (RISC) processor, a complex instruction set computer (CISC) processor, or a combination of the above. Optionally, the processor 301 can also be a dedicated device such as an application specific integrated circuit (ASIC). Optionally, the processor 301 can also be a neural network processor or a combination of a neural network processor and the above conventional processors.

[0141] Optionally, in various embodiments of the present application, the computer system 160 can be located away from the intelligent vehicle and communicate wirelessly with the intelligent vehicle 100. In other aspects, some processes of the present application can be set to be executed on a processor in the intelligent vehicle, and some other processes are executed by a remote processor, including taking actions required to perform a single maneuver.

[0142] The computer system 160 can communicate with a software deploying server 313 through a network interface 312. Optionally, the network interface 312 can be a hardware network interface, such as a network card. The network 314 can be an external network, such as the Internet, or an internal network, such as Ethernet or a virtual private network (VPN). Optionally, the network 314 can also be a wireless network, such as a WiFi network, a cellular network, etc.

[0143] The hard disk drive interface 315 is coupled to the system bus 302. The hard disk drive interface 315 is connected to the hard disk drive 316. The system memory 317 is coupled to the system bus 302. The data running in the system memory 317 may include the operating system (OS) 318 and the application programs 319 of the computer system 160.

[0144] The operating system (OS) 318 includes, but is not limited to, the Shell 320 and the kernel 321. The Shell 320 is an interface between the user and the kernel 321 of the operating system 318. The Shell 320 is the outermost layer of the operating system 318. The shell manages the interaction between the user and the operating system 318: waits for the user's input, interprets the user's input to the operating system 318, and processes various output results of the operating system 318.

[0145] The kernel 321 consists of the parts in the operating system 318 that are used to manage memory, files, peripherals, and system resources, and directly interacts with the hardware. The kernel 321 of the operating system 318 usually runs processes, provides inter-process communication, and provides functions such as CPU time slice management, interrupts, memory management, and IO management.

[0146] The application programs 319 include programs 323 related to autonomous driving. For example, programs that manage the interaction between an autonomous vehicle and road obstacles, programs that control the driving route or speed of an autonomous vehicle, programs that control the interaction between an autonomous vehicle and other vehicles / autonomous vehicles on the road, etc. The application programs 319 also exist on the system of the deploying server 313. In one embodiment, when the application programs 319 need to be executed, the computer system 160 can download the application programs 319 from the deploying server 313.

[0147] For another example, the application program 319 may be an application program that controls the vehicle to determine a driving strategy based on the passable area of the vehicle and the traditional control module described above. The processor 301 of the computer system 160 calls the application program 319 to obtain the driving strategy.

[0148] The sensor 322 is associated with the computer system 160. The sensor 322 is used to detect the environment around the computer system 160. For example, the sensor 322 can detect animals, cars, obstacles, and / or crosswalks, etc. Further, the sensor 322 can also detect the environment around the above-mentioned objects such as animals, cars, obstacles, and / or crosswalks. For example: the environment around an animal, such as other animals that appear around the animal, weather conditions, the brightness of the environment around the animal, etc. Optionally, if the computer system 160 is located in an autonomous vehicle, the sensor 322 can be at least one of devices such as a camera, an infrared sensor, a chemical detector, a microphone, etc.

[0149] In some other embodiments of the present application, the computer system 160 can also receive information from other computer systems or transfer information to other computer systems. Or, the sensor data collected from the sensor system 120 of the vehicle 100 can be transferred to another computer for processing by the other computer. As Figure 4 shown, the data from the computer system 160 can be transmitted via a network to the computer system 410 on the cloud side for further processing. The network and intermediate nodes can include various configurations and protocols, including the Internet, the World Wide Web, intranets, virtual private networks, wide area networks, local area networks, private networks using proprietary communication protocols of one or more companies, Ethernet, WiFi, and HTTP, as well as various combinations of the foregoing. Such communication can be performed by any device capable of transmitting data to other computers and receiving data from other computers, such as a modem and a wireless interface.

[0150] In one example, the computer system 410 can include a server having multiple computers, such as a load balancing server cluster. In order to receive, process, and transmit data from the computer system 160, the server 420 exchanges information with different nodes of the network. The computer system 410 can have a configuration similar to that of the computer system 160 and have a processor 430, a memory 440, instructions 450, and data 460.

[0151] In one example, the data 460 of the server 420 can include providing weather-related information. For example, the server 420 can receive, monitor, store, update, and transmit various information related to target objects in the surrounding environment. The information can include, for example, target categories, target shape information, and target tracking information in the form of reports, radar information, forecasts, etc.

[0152] See Figure 5, is an example of the interaction between an autonomous vehicle and a cloud service center (cloud server). The cloud service center can receive information (such as data collected by vehicle sensors or other information) from vehicles 513 and 512 within its operating environment 500 via a network 511 such as a wireless communication network. Among them, vehicles 513 and 512 can be intelligent vehicles.

[0153] The cloud service center 520 controls vehicles 513 and 512 according to the received data by running the programs stored in it related to controlling the autonomous driving of the vehicle. The programs related to controlling the autonomous driving of the vehicle can be: a program for managing the interaction between the autonomous vehicle and road obstacles, or a program for controlling the route or speed of the autonomous vehicle, or a program for controlling the interaction between the autonomous vehicle and other autonomous vehicles on the road.

[0154] Exemplarily, the cloud service center 520 can provide a part of the map to vehicles 513 and 512 via the network 511. In other examples, the operations can be divided among different locations. For example, multiple cloud service centers can receive, verify, combine, and / or send information reports. In some examples, information reports and / or sensor data can also be sent between vehicles. Other configurations are also possible.

[0155] In some examples, the cloud service center 520 sends solutions recommended for possible driving situations in the environment to the intelligent vehicle (e.g., informing about an obstacle ahead and how to bypass it). For example, the cloud service center 520 can assist the vehicle in determining how to proceed when facing a specific obstacle in the environment. The cloud service center 520 sends a response indicating how the vehicle should proceed in a given scenario to the intelligent vehicle. For example, based on the collected sensor data, the cloud service center 520 can confirm the existence of a temporary stop sign ahead on the road, or, based on the sensor data of a "lane closed" sign and a construction vehicle, determine that the lane is closed due to construction. Accordingly, the cloud service center 520 sends a recommended operation mode for the vehicle to pass the obstacle (e.g., indicating that the vehicle should change lanes to another road). When the cloud service center 520 observes the video stream within its operating environment 500 and has confirmed that the intelligent vehicle can safely and successfully pass the obstacle, the operation steps used by the intelligent vehicle can be added to the driving information map. Accordingly, this information can be sent to other vehicles in the area that may encounter the same obstacle, so as to assist other vehicles not only in identifying the closed lane but also in knowing how to pass.

[0156] In some embodiments, the disclosed method can be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or encoded on other non-transitory media or articles. Figure 6A conceptual partial view of an example computer program product arranged in accordance with at least some of the embodiments presented herein is shown schematically. The example computer program product includes a computer program for executing a computer process on a computing device. In one embodiment, the example computer program product 600 is provided using a signal-bearing medium 601. The signal-bearing medium 601 may include one or more program instructions 602 which, when executed by one or more processors, may provide all or part of the functionality described above for Figures 2 to 5 or may provide all or part of the functionality described in subsequent embodiments. For example, referring to the embodiment shown in Figure 7 , one or more features in S101 to S103 may be borne by one or more instructions associated with the signal-bearing medium 601. In addition, Figure 6 the program instructions 602 in also describe example instructions.

[0157] In some examples, the signal-bearing medium 601 may include a computer-readable medium 603, such as but not limited to, a hard disk drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a read-only memory (ROM), or a random access memory (RAM), etc. In some embodiments, the signal-bearing medium 601 may include a computer-recordable medium 604, such as but not limited to, a memory, a read / write (R / W) CD, an R / W DVD, etc. In some embodiments, the signal-bearing medium 601 may include a communication medium 605, such as but not limited to, a digital and / or analog communication medium (e.g., an optical fiber cable, a waveguide, a wired communication link, a wireless communication link, etc.). Thus, for example, the signal-bearing medium 601 may be conveyed by a wireless form of the communication medium 605 (e.g., a wireless communication medium compliant with the IEEE 802.11 standard or other transmission protocols). The one or more program instructions 602 may be, for example, computer-executable instructions or logic-implemented instructions. In some examples, such as for Figures 2 to 6The described computing device can be configured to provide various operations, functions, or actions in response to program instructions 602 communicated to the computing device via one or more of computer-readable medium 603, and / or computer-recordable medium 604, and / or communication medium 605. It should be understood that the arrangements described herein are for illustrative purposes only. Thus, those skilled in the art will understand that other arrangements and other elements (e.g., machines, interfaces, functions, sequences, and groups of functions, etc.) can be used instead, and some elements can be omitted altogether depending on the desired results. Additionally, many of the described elements can be implemented as discrete or distributed components, or as functional entities combined with other components in any suitable combination and location.

[0158] The above briefly introduced the application scenario of the method for determining the three-dimensional information of the detection object described in the embodiments of this application.

[0159] To make this application clearer, the following will briefly introduce some concepts related to this application.

[0160] 1. Ground contact line

[0161] The ground contact line refers to the line segment composed of the points where the detection object in the image to be detected actually contacts the ground.

[0162] Taking the detection object as a vehicle as an example, the ground contact line of the vehicle is the connection line of the contact points between the vehicle tires and the ground. The ground contact line of the vehicle can be distinguished according to the four sides of the vehicle (the left side, the right side, the front side, and the rear side respectively). One side of the vehicle corresponds to one ground contact line.

[0163] Specifically, taking a common household car as an example, the contact point between the left front tire of the vehicle and the ground is denoted as contact point 1; the contact point between the right front tire of the vehicle and the ground is denoted as contact point 2; the contact point between the left rear tire of the vehicle and the ground is denoted as contact point 3; the contact point between the right rear tire of the vehicle and the ground is denoted as contact point 4.

[0164] The ground contact line corresponding to the left side of the vehicle is the connection line between contact point 1 and contact point 3.

[0165] The ground contact line corresponding to the right side of the vehicle is the connection line between contact point 2 and contact point 4.

[0166] The ground contact line corresponding to the front side of the vehicle is the connection line between contact point 1 and contact point 2.

[0167] The ground contact line corresponding to the rear side of the vehicle is the connection line between contact point 3 and contact point 4.

[0168] It should be noted that the part of the vehicle's tire in contact with the ground is usually a contact surface (which can be approximately regarded as a rectangle). In this application, the front - left vertex of the contact surface between the front - left tire of the vehicle and the ground can be used as contact point 1; the front - right vertex of the contact surface between the front - right tire of the vehicle and the ground can be used as contact point 2; the rear - left vertex of the contact surface between the rear - left tire of the vehicle and the ground can be used as contact point 3; and the rear - right vertex of the contact surface between the rear - right tire of the vehicle and the ground can be used as contact point 4.

[0169] 2. Neural network model

[0170] A neural network model is an information - processing system composed of a large number of processing units (denoted as neurons) interconnected with each other. The neurons in the neural network model contain corresponding mathematical expressions. After data is input into a neuron, the neuron runs the mathematical expression it contains to calculate the input data and generate output data. Among them, the input data of each neuron is the output data of the previous neuron connected to it; the output data of each neuron is the input data of the next neuron connected to it.

[0171] In a neural network model, after inputting data, the neural network model selects corresponding neurons for the input data according to its own learning and training, and calculates the input data based on these neurons to determine and output the final operation result. At the same time, the neural network can also continuously learn and evolve during the data operation process, and continuously optimize its own operation process according to the feedback on the operation result. The more times the neural network model is trained, the more result feedback is obtained, and the more accurate the calculation result is.

[0172] The neural network model described in the embodiments of this application is used to process the pictures collected by the image acquisition device to determine the boundaries of the passable areas located on each detection object in the image (denoted as the passable area boundaries of the detection object).

[0173] 3. Passable area (freespace)

[0174] The passable area refers to the area through which a vehicle can drive. For example, the open area among pedestrians, obstacles, and other vehicles in the front area detected by the first vehicle is denoted as the passable area of the first vehicle.

[0175] The passable area is generally located on the boundary of the detection object. Therefore, in the embodiments of this application, the passable area located on the boundary of the detection object can be used to characterize the boundary of the first detection object, and then the three - dimensional information of the first detection object can be determined according to the passable area located on the first detection object.

[0176] In the embodiments of the present application, the boundaries of the passable areas of the detected objects output by the neural network model are usually shown by a plurality of points with corresponding identifiers. For example, the passable area on the left side of the second vehicle includes a plurality of points located on the boundary points on the left side of the vehicle. These plurality of points are used to characterize the passable area on the left side of the second vehicle.

[0177] In addition, among these points output by the neural network model, the points located on different sides of the detected object may have different identifiers. For example, these identifiers may include: the identifier "00" for characterizing the points on the boundary of the passable area on the left side of the detected object; the identifier "01" for characterizing the points on the boundary of the passable area on the right side of the detected object; the identifier "10" for characterizing the points on the boundary of the passable area on the front side of the detected object; and the identifier "11" for characterizing the points on the boundary of the passable area on the rear side of the vehicle.

[0178] 4. Horizon line

[0179] The horizon line refers to the straight line parallel to the line of sight in the image. In the embodiments of the present application, the horizon line refers to the straight line in the image that is at the same height as the image acquisition device and parallel to the image acquisition device.

[0180] 5. Vanishing point

[0181] According to the perspective principle of the image, two straight lines parallel to each other on the horizontal plane will intersect at a point on the horizon line in the two-dimensional image, and this intersection point is the vanishing point.

[0182] 6. Vehicle body coordinate system

[0183] The vehicle body coordinate system refers to a three-dimensional coordinate system with the coordinate origin located on the vehicle body. Generally, the origin of the vehicle body coordinate system coincides with the center of mass of the vehicle, the X-axis is along the length direction of the vehicle, pointing to the front of the vehicle, the Y-axis is along the width direction of the vehicle, pointing to the left side of the driver, and the Z-axis is along the height direction of the vehicle, pointing upward.

[0184] The above is a simple introduction to some of the content and concepts involved in the present application.

[0185] Currently, in order to determine the three-dimensional information of the second vehicle around the first vehicle, the following three methods for determining vehicle information are proposed. They are: Method 1, vehicle 2D detection; Method 2, vehicle binocular 3D detection; and Method 3, vehicle laser point cloud detection. Hereinafter, Method 1, Method 2, and Method 3 will be described in detail.

[0186] Method 1, vehicle 2D detection

[0187] The vehicle 2D detection is as follows: the first vehicle determines the image information of the second vehicle shown in the image to be detected; the first vehicle frames the image information of the second vehicle shown in the image to be detected in the form of a rectangular box; the first vehicle calculates the distance between the second vehicle and the first vehicle according to the position of the lower edge of the rectangular box, and determines the relative position between the second vehicle and the first vehicle.

[0188] It can be seen from this that in this method, the first vehicle can only determine the position information of the second vehicle relative to the first vehicle. However, in the intelligent driving scenario, in addition to determining the position information of the vehicle, the vehicle also needs to determine information such as the size and direction of the vehicle to judge whether the surrounding vehicles interfere with its own driving.

[0189] Therefore, relying on simple vehicle 2D detection cannot meet the needs of the first vehicle for information of other vehicles in the intelligent driving scenario.

[0190] Method 2: Vehicle binocular 3D detection

[0191] Binocular 3D detection can obtain the depth information of the detection object by determining the difference between the images of the detection object collected by two image acquisition devices located at different positions. By establishing the corresponding relationship of the same points in the images, the image points of the same spatial physical point in different images are corresponding to form a disparity image, and the 3D information of the detection object can be determined according to the disparity image.

[0192] The vehicle binocular 3D detection is as follows: the first vehicle uses a binocular camera to collect two images of the second vehicle from two angles respectively. The first vehicle calculates the three-dimensional information of the second vehicle according to the deviation of the same points on the second vehicle in the positions of the two images. Through the binocular 3D detection algorithm, the size, direction and position information of the second vehicle relative to the first vehicle can be calculated more accurately.

[0193] However, the hardware equipment of the binocular camera is expensive, the manufacturing requirements of the binocular camera applied to intelligent vehicles are high, and the algorithms currently used in the process of determining the 3D information of the second vehicle require high image annotation costs and computational amounts.

[0194] Method 3: Vehicle laser point cloud detection

[0195] When a laser beam irradiates the surface of an object, the reflected laser will carry information such as azimuth and distance. If the laser beam is scanned according to a certain trajectory, the information of the reflected laser points will be recorded while scanning. Since the scanning is extremely fine, a large number of laser points can be obtained, and thus a laser point cloud can be formed.

[0196] Vehicle laser point cloud detection is as follows: The first vehicle emits laser light around to scan the surrounding detection objects. The first vehicle receives the laser point cloud data returned by the surrounding detection objects, and this point cloud data includes the point cloud data returned by the second vehicle and the point cloud data returned by other detection objects. The first vehicle uses algorithms such as machine learning or deep learning to map the laser point cloud data returned by the second vehicle into a certain data structure. The first vehicle extracts each point or feature in this data, and clusters the point cloud data according to these features, grouping similar point clouds into one category. The first vehicle inputs the clustered point cloud into the corresponding classifier for classification and recognition to determine the point cloud data of the second vehicle. The first vehicle maps the point cloud data of the second vehicle back into three-dimensional point cloud data, constructs a 3D bounding box of the second vehicle, and determines the three-dimensional information of the second vehicle.

[0197] Although vehicle laser point cloud detection has good detection accuracy, the hardware cost of lidar is relatively high, and the data calculation amount of point cloud data is large, which requires a large amount of computing resources and occupies a large amount of GPU resources of the first vehicle.

[0198] To solve the problems in the prior art that when the first vehicle determines the two-dimensional information of the second vehicle, it cannot accurately determine the size and direction information of the second vehicle, and when the first vehicle determines the three-dimensional information of the second vehicle, the hardware cost is high and the computational complexity is high when using vehicle binocular 3D detection or vehicle laser point cloud detection. The embodiments of the present application provide a method for determining the three-dimensional information of a detection object. The first vehicle can determine the passable area boundary and the ground contact line of the first detection object according to the collected image to be detected. Further, the first vehicle determines the three-dimensional information represented by the first detection object in the two-dimensional image according to the passable area boundary and the ground contact line of the first detection object.

[0199] The above image to be detected can be a two-dimensional image collected by a monocular camera. In this way, the first vehicle can determine the three-dimensional information of the first detection object by collecting the image information of the first detection object through the monocular camera. Compared with the method in the prior art that the first vehicle needs to rely on a binocular camera to collect the image information of the first detection object to determine the three-dimensional information of the first detection object, or the method that the first vehicle relies on lidar to determine the three-dimensional information of the first detection object, in the present application, the first vehicle uses a monocular camera to collect the image information of the first detection object to determine the three-dimensional information of the first detection object, which can greatly reduce the hardware cost of determining the three-dimensional information of the first detection object.

[0200] In addition, for the method provided in this application for determining the three-dimensional information of a detection object, the first vehicle only needs to mark the boundary of the passable area of the first detection object and determine the ground contact line of the first detection object based on the boundary of the passable area of the first detection object. The first vehicle can determine the three-dimensional information represented by the first detection object in the two-dimensional image based on the boundary of the passable area of the first detection object, the ground contact line, and the visual relationship of the first detection object in the image to be detected, etc. Therefore, the method provided in this application for determining the three-dimensional information of a detection object does not require the first vehicle to perform other additional data annotation training, thereby reducing the computational complexity of determining the three-dimensional information of the first detection object and reducing the graphics processing unit (GPU) resources occupied by determining the three-dimensional information of the first detection object.

[0201] Hereinafter, the method provided in the embodiments of this application for determining the three-dimensional information of a detection object will be described in detail. As Figure 7 shown, the method includes:

[0202] S101. The first vehicle acquires an image to be detected.

[0203] Among them, the image to be detected includes a first detection object.

[0204] In the field of intelligent driving, the detection object can be a vehicle, a pedestrian, an obstacle, etc. In the embodiments of this application, the second vehicle is taken as an example of the detection object for illustration.

[0205] The above-mentioned image to be detected can be a picture collected by an in-vehicle image acquisition device. The in-vehicle image acquisition device is usually used to collect other vehicles located in front of the vehicle; or, the in-vehicle image acquisition device can also collect the 360° omnidirectional image of the vehicle to obtain information about all other vehicles around the vehicle.

[0206] It should be noted that the image acquisition device described in the embodiments of this application can be a monocular camera. When the first vehicle executes the method provided in the embodiments of this application for determining the three-dimensional information of a detection object, the first vehicle can be an in-vehicle terminal device set in the first vehicle, or other devices with data processing capabilities.

[0207] S102. The first vehicle determines the boundary of the passable area of the first detection object and the ground contact line of the first detection object.

[0208] Among them, the boundary of the passable area of the first detection object includes the boundary of the first detection object in the image to be detected. The ground contact line of the first detection object is the connection line of the intersection points of the first detection object and the ground.

[0209] The boundary of the passable area of the first detection object is the boundary of the passable area of the first detection object output by the neural network model after inputting the image to be detected into the neural network model.

[0210] When the first detection object is the second vehicle, the number of grounding lines of the second vehicle is related to the number of sides of the second vehicle shown in the image to be detected.

[0211] As Figure 8a shown, when the first detection object is a vehicle located in the front right of the first vehicle in the image to be detected, and the left side and the rear side of the second vehicle are shown in the image to be detected, the grounding lines of the second vehicle include two grounding lines, namely the grounding line of the left side of the second vehicle and the grounding line of the rear side of the second vehicle.

[0212] The grounding line of the left side of the second vehicle is the connection line between the grounding points of the tires on the left front side and the left rear side of the second vehicle.

[0213] The grounding line of the rear side of the second vehicle is the connection line between the grounding points of the tires on the left rear side and the right rear side of the second vehicle.

[0214] Or, as Figure 8b shown, when the first detection object is a vehicle located directly in front of the first vehicle in the image to be detected, and only the rear side of the second vehicle is shown in the image to be detected, the grounding line of the second vehicle includes one grounding line, which is the grounding line of the rear side of the second vehicle.

[0215] The grounding line of the rear side of the second vehicle is the connection line between the grounding points of the tires on the left rear side and the right rear side of the second vehicle.

[0216] S103. The first vehicle determines the three-dimensional information of the first detection object according to the passable area boundary and the grounding line.

[0217] The three-dimensional information of the first detection object is used to determine at least one of the size, direction, and relative position of the first detection object.

[0218] The three-dimensional information of the first detection object is used to represent the three-dimensional information shown by the first detection object in the image to be detected. The first vehicle can convert the three-dimensional information shown by the first detection object in the image to be detected into a three-dimensional coordinate system to further determine the true three-dimensional information of the first detection object.

[0219] For example, when the first detection object is the second vehicle, the first vehicle converts the three-dimensional information shown by the second vehicle in the image to be detected into a three-dimensional coordinate system, and can determine the size of the second vehicle (such as the length and width of the second vehicle), the direction of the second vehicle (such as the head orientation of the second vehicle, the possible driving direction of the second vehicle), and the position of the second vehicle in this three-dimensional coordinate system.

[0220] It should be noted that the relative position of the first detection object is related to the three-dimensional coordinate system to which the first vehicle converts the first detection object. For example, when the first vehicle converts the first detection object into the vehicle body coordinate system of the first vehicle, the relative position of the first detection object is the position of the first detection object relative to the first vehicle; when the first vehicle converts the first detection object into the world coordinate system, the relative position of the first detection object is the actual geographical location of the first detection object.

[0221] In a possible implementation, the three-dimensional information of the first detection object includes a plurality of points and a plurality of line segments. The plurality of points are the projections of the endpoints shown by the first detection object in the image to be detected on the ground. At least one of the plurality of line segments includes a line segment generated by the projection of the outermost boundary of the first detection object on the ground; alternatively, the plurality of line segments are the contour lines of the first detection object in the image to be detected. The first vehicle inputs the plurality of line segments into the vehicle body coordinate system of the first vehicle, and at least one of the size, direction, and relative position of the first detection object can be determined.

[0222] Based on the above technical solution, in the method for determining the three-dimensional information of the detection object provided by this application, the first vehicle can determine the passable area boundary and the touchdown line of the first detection object according to the collected image to be detected. Further, the first vehicle determines the three-dimensional information represented by the first detection object in the two-dimensional image according to the passable area boundary and the touchdown line of the first detection object.

[0223] The above image to be detected can be a two-dimensional image collected by a monocular camera. In this way, the first vehicle can determine the three-dimensional information of the first detection object by collecting the image information of the first detection object through the monocular camera. Compared with the method in the prior art in which the first vehicle needs to rely on a binocular camera to collect the image information of the first detection object to determine the three-dimensional information of the first detection object, in this application, the first vehicle uses a monocular camera to collect the image information of the first detection object to determine the three-dimensional information of the first detection object, which can greatly reduce the hardware cost of determining the three-dimensional information of the first detection object.

[0224] In addition, in the method for determining the three-dimensional information of the detection object provided by this application, the first vehicle only needs to mark the passable area boundary of the first detection object and determine the touchdown line of the first detection object according to the passable area boundary of the first detection object. The first vehicle can determine the three-dimensional information represented by the first detection object in the two-dimensional image according to the passable area boundary and the touchdown line of the first detection object, combined with the visual relationship of the first detection object in the image to be detected, etc. Therefore, in the method for determining the three-dimensional information of the detection object provided by this application, there is no need for the first vehicle to perform other additional data annotation training, thereby reducing the computational amount of determining the three-dimensional information of the first detection object and reducing the graphics processing unit (GPU) resources occupied by determining the three-dimensional information of the first detection object.

[0225] Combine Figure 7 , such as Figure 9 As shown, the above S102 can be specifically implemented through the following S1021 - S1023. The following will elaborate on S1021 - S1023.

[0226] S1021. The first vehicle inputs the image to be detected into the neural network model and obtains L points.

[0227] Among them, the image to be detected usually includes one or more detection objects. The above L points are the points on the boundary of the passable area of the one or more detection objects. L is a positive integer.

[0228] In a possible implementation, the above neural network model is a pre - trained neural network model. This neural network model has the ability to mark the boundary of the passable area of the detection object in the image to be detected. The boundary of the passable area is the boundary of the detection object in the image to be detected.

[0229] Specifically, after the first vehicle acquires the image to be detected, it calls the neural network model and inputs the image to be detected into this neural network model, and outputs L points. These L points are the ability to represent the boundary of the passable area of the detection object in the image to be detected.

[0230] Each of the L points output by the neural network model can correspond to an identifier. This identifier is used to characterize which side of the detection object the point is located on.

[0231] For example, the point located on the left side of the detection object corresponds to the first identifier. Correspondingly, this first identifier is used to characterize that the point is located on the left side of the detection object.

[0232] The point located on the right side of the detection object corresponds to the second identifier. Correspondingly, this second identifier is used to characterize that the point is located on the right side of the detection object.

[0233] The point located in front of the detection object corresponds to the third identifier. Correspondingly, this third identifier is used to characterize that the point is located in front of the detection object.

[0234] The point located behind the detection object corresponds to the fourth identifier. Correspondingly, this fourth identifier is used to characterize that the point is located behind the detection object.

[0235] S1022. The first vehicle determines M points from these L points.

[0236] These M points are the points on the boundary of the passable area of the first detection object.

[0237] In a specific implementation, the first vehicle classifies the L points according to one or more detection objects in the image to be detected, and determines the detection object corresponding to each point. After that, the first vehicle determines M points corresponding to the first detection object according to the detection object of each point object. The M points are the points located on the boundary of the passable area of the first detection object. M is a positive integer and M is less than or equal to L.

[0238] S1023. The first vehicle fits the M points to determine the ground contact line of the first detection object.

[0239] In a possible implementation, the first vehicle can use the random sample consensus (RANSAC) fitting algorithm to fit and determine the ground contact line of the target object.

[0240] Specifically, the first vehicle can use the RANSAC fitting algorithm. The process of fitting and determining the ground contact line of the target object includes the following steps a - f, which will be described in detail below:

[0241] Step a. The first vehicle determines K points that are located on the boundary of the passable area of the first detection object and have the same identifier. K is a positive integer.

[0242] Step b. The first vehicle randomly selects T points from the K points and uses the least squares method to fit the T points to obtain a straight line.

[0243] Step c. The first vehicle determines the distance of each point among the K points except the T points from the straight line.

[0244] Step d. The first vehicle determines the points with a distance less than the first threshold as inlier points and determines the number of inlier points.

[0245] Step e. The first vehicle repeatedly executes the above steps b - d to determine multiple straight lines and the number of inlier points corresponding to each straight line among the multiple straight lines.

[0246] Step f. The first vehicle determines the straight line with the largest number of corresponding inlier points among the multiple straight lines as a ground contact line of the first detection object.

[0247] It should be noted that in the above step e, the more the number of straight lines determined by the first vehicle through steps b - d, the higher the accuracy of the final determined result.

[0248] Based on the above technical solution, the first vehicle can determine the boundary of the passable area of the first detection object and the ground contact line of the first detection object according to the image to be detected, using a neural network model and a corresponding fitting algorithm.

[0249] It should be noted that the monocular camera of the first vehicle can capture the image of an object in front of the monocular camera. When the second vehicle is directly in front of the monocular camera, usually only one side of the second vehicle can be captured by the monocular camera of the first vehicle. When the second vehicle is in front of the monocular camera and deviated from the position directly in front of the monocular camera, usually two sides of the second vehicle can be captured by the monocular camera of the first vehicle.

[0250] Therefore, the images of the second vehicle captured by the first vehicle include the following two scenarios: Scenario 1: The first vehicle captures two sides of the second vehicle. Scenario 2: The first vehicle captures one side of the second vehicle. The following is a detailed description of Scenario 1 and Scenario 2 above:

[0251] Scenario 1: The first vehicle captures two sides of the second vehicle.

[0252] Among them, the image information of the second vehicle captured by the first vehicle is related to the image of which direction of the first vehicle captured by the monocular camera.

[0253] For example, when the monocular camera captures the image in front of the first vehicle:

[0254] If the second vehicle is traveling in the same direction as the first vehicle and is located in the front left of the first vehicle, the monocular camera can capture the right side and the rear side of the second vehicle.

[0255] If the second vehicle is traveling in the same direction as the first vehicle and is located in the front right of the first vehicle, the monocular camera can capture the left side and the rear side of the second vehicle.

[0256] If the second vehicle is traveling in the opposite direction to the first vehicle and is located in the front left of the vehicle, the monocular camera can capture the front side and the left side of the second vehicle.

[0257] If the second vehicle is traveling in the opposite direction to the first vehicle and is located in the front right of the vehicle, the monocular camera can capture the front side and the right side of the second vehicle.

[0258] Another example is when the monocular camera captures the image on the left side of the second vehicle:

[0259] If the second vehicle is located on the left side of the first vehicle and deviated from the position directly in front of the monocular camera, the monocular camera can capture the right side of the second vehicle, and one of the front side or the rear side of the second vehicle.

[0260] In addition, the monocular camera can also capture the images of other sides of the second vehicle, which will not be elaborated in this application.

[0261] Scenario 2: The first vehicle captures one side of the second vehicle.

[0262] Among them, the image information of the second vehicle collected by the first vehicle is related to the image of which direction of the first vehicle collected by the monocular camera.

[0263] For example, when the monocular camera collects the image in front of the first vehicle:

[0264] If the second vehicle travels in the same direction as the first vehicle and is directly in front of the first vehicle, the monocular camera can collect the rear side of the second vehicle.

[0265] If the second vehicle travels in the opposite direction to the first vehicle and is directly in front of the first vehicle, the monocular camera can collect the front side of the second vehicle.

[0266] If the first vehicle travels due north and the second vehicle travels due east, the monocular camera can collect the right side of the second vehicle.

[0267] If the first vehicle travels due north and the second vehicle travels due west, the monocular camera can collect the left side of the second vehicle.

[0268] Another example, when the monocular camera collects the image on the left side of the second vehicle:

[0269] If the second vehicle is on the left side of the first vehicle and directly in front of the monocular camera, the monocular camera can collect the right side of the second vehicle.

[0270] As described above, in different scenarios, the number of sides of the second vehicle that the first vehicle can collect is different.

[0271] It should be noted that when the number of sides of the second vehicle collected by the first vehicle is different, the boundary of the passable area of the second vehicle determined by the first vehicle is different, the number of touchdown lines of the second vehicle determined by the first vehicle is different, and the three-dimensional information of the second vehicle determined by the first vehicle is different.

[0272] Specifically, in the scenario where the first vehicle collects two sides of the second vehicle: the boundary of the passable area of the second vehicle determined by the first vehicle is the boundary of the passable areas of these two sides. The touchdown lines of the second vehicle determined by the first vehicle include a first touchdown line and a second touchdown line, and the first touchdown line and the second touchdown line respectively correspond to different sides among these two sides. The three-dimensional information of the second vehicle determined by the first vehicle includes the three-dimensional information composed of these two sides.

[0273] In the scenario where the first vehicle captures a side of the second vehicle: The passable area boundary of the second vehicle determined by the first vehicle is the passable area boundary of this side. The ground contact line of the second vehicle determined by the first vehicle includes a third ground contact line, and the third ground contact line is the ground contact line of this side. The three-dimensional information of the second vehicle determined by the first vehicle includes the three-dimensional information composed of this side.

[0274] Therefore, combining the above Scenario 1 and Scenario 2, in S103, the first vehicle determines the three-dimensional information of the first detection object based on the passable area boundary and the ground contact line, including the following two cases, namely: Case 1: The first vehicle determines the three-dimensional information of the first detection object based on the passable area boundaries and the ground contact lines of two sides of the first detection object; and Case 2: The first vehicle determines the three-dimensional information of the first detection object based on the passable area boundary and the ground contact line of one side of the first detection object.

[0275] The following will elaborate on Case 1 and Case 2 respectively:

[0276] Case 1: The first vehicle determines the three-dimensional information of the first detection object based on the passable area boundaries and the ground contact lines of two sides of the first detection object.

[0277] Combined with the above S102, in Case 1, the passable area boundary of the first detection object determined by the first vehicle according to S102 and the ground contact line of the first detection object respectively have the following characteristics:

[0278] 1. The passable area boundary of the first detection object includes multiple boundary points corresponding to the first identifier, and boundary points corresponding to multiple second identifiers.

[0279] The first identifier also corresponds to the first side of the first detection object, and the second identifier also corresponds to the second side of the first detection object. The first side and the second side are two intersecting sides of the first detection object.

[0280] 2. The ground contact line includes a first ground contact line and a second ground contact line.

[0281] The first ground contact line is the ground contact line determined by fitting multiple boundary points corresponding to the first identifier.

[0282] The second ground contact line is the ground contact line determined by fitting multiple boundary points corresponding to the second identifier.

[0283] 3. Among the multiple boundary points corresponding to the first identifier, there is a first boundary point; the first boundary point is the boundary point with the largest distance from the second ground contact line among the multiple boundary points with the first identifier.

[0284] Among the multiple boundary points corresponding to the second identifier, there is a second boundary point; the second boundary point is the boundary point with the largest distance from the first ground contact line among the multiple boundary points with the second identifier.

[0285] Combined with the above S103, in Case 1, the three-dimensional information of the first detection object determined by the first vehicle according to the above S103 is determined based on three points and two lines corresponding to the first detection object.

[0286] Among them, the first of the three points is the projection of the first boundary point on the ground.

[0287] The second of the three points is the projection of the second boundary point on the ground.

[0288] The third of the three points is the intersection point of the straight line passing through the second point and parallel to the second ground contact line and the first ground contact line.

[0289] The first of the two lines is the connection line between the first point and the third point.

[0290] The second of the two lines is the connection line between the second point and the third point.

[0291] Combined with Figure 7 , as Figure 10 shown, in Case 1, S103 can be specifically implemented through the following S1031 - S1035. Next, a specific description of S1031 - S1035 is given:

[0292] S1031. The first vehicle determines the first point according to the first boundary point and the first ground contact line.

[0293] Among them, the first point is the intersection point of the first ground contact line and the first straight line. The first straight line is a straight line passing through the first boundary point and perpendicular to the horizon line in the image to be detected.

[0294] In a specific implementation manner, combined with Figure 11 the first detection object in the image to be detected shown in, the method for the first vehicle to determine the first point is:

[0295] Step Ⅰ. The first vehicle determines the first straight line; the first straight line is a straight line passing through the first boundary point and perpendicular to the horizon line in the image to be detected.

[0296] In an implementation manner, the first vehicle makes a perpendicular line to the horizon line through the first boundary point, and this perpendicular line is the first straight line.

[0297] Step Ⅱ. The first vehicle determines the intersection point of the first straight line and the first ground contact line as the first point.

[0298] In one implementation, the first vehicle extends the first ground contact line, and this extension line intersects the first straight line at point a. The first vehicle determines this point a as the first point.

[0299] S1032. The first vehicle determines the second point based on the first ground contact line, the second ground contact line, and the second boundary point.

[0300] Among them, the second point is the intersection point of the second straight line and the third straight line. The second straight line is the straight line passing through the intersection point of the first ground contact line and the horizon line, and the vertex of the second ground contact line that is far from the first ground contact line. The third straight line is the straight line passing through the second boundary point and perpendicular to the horizon line in the image to be detected.

[0301] In a specific implementation, in combination with Figure 11 the first detection object in the image to be detected shown in, the method for the first vehicle to determine the second point is:

[0302] Step III. The first vehicle determines the second straight line.

[0303] Among them, the second straight line is the straight line passing through the intersection point of the first ground contact line and the horizon line, and the end point of the second ground contact line that is far from the first ground contact line in the image to be detected.

[0304] In one implementation, the first vehicle extends the ground contact line to obtain the intersection point b of the first ground contact line and the horizon line. The first vehicle determines the end point c of the second ground contact line that is far from the first ground contact line. The first vehicle makes a straight line passing through the above intersection point b and the end point c, and this straight line is the second straight line.

[0305] Step IV. The first vehicle determines the third straight line.

[0306] The third straight line is the straight line passing through the second boundary point and perpendicular to the horizon line in the image to be detected.

[0307] In one implementation, the first vehicle makes a perpendicular line to the horizon line through the second boundary point, and this perpendicular line is the third straight line.

[0308] Step V. The first vehicle determines the intersection point of the second straight line and the third straight line as the second point.

[0309] In one implementation, the first vehicle determines that the second straight line and the third straight line intersect at point c, and the first vehicle determines this point c as the second point.

[0310] S1033. The first vehicle determines the third point based on the first ground contact line, the second ground contact line, and the second point.

[0311] Among them, the third point is the intersection point of the first ground contact line and the fourth straight line. The fourth straight line is the straight line passing through the second point and parallel to the second ground contact line in the image to be detected.

[0312] In a specific implementation, in combination with Figure 11 the first detection object in the image to be detected shown in, the method for the first vehicle to determine the third point is as follows:

[0313] Step Ⅵ: The first vehicle determines the fourth straight line.

[0314] In an implementation, the first vehicle makes a parallel line to the second ground contact line passing through the second point. The first vehicle determines this parallel line as the fourth straight line.

[0315] Step Ⅶ: The first vehicle determines the intersection point of the fourth straight line and the first ground contact line as the third point.

[0316] In an implementation, the determination device determines the intersection point d of the fourth straight line and the first ground contact line, and the first vehicle determines this intersection point d as the third point.

[0317] S1034: The first vehicle determines the first line according to the first point and the third point.

[0318] In an implementation, as Figure 11 shown, the first vehicle makes a line segment a with the first point and the third point as endpoints respectively, and the first vehicle determines this line segment as the first line.

[0319] S1035: The first vehicle determines the second line according to the second point and the third point.

[0320] In an implementation, as Figure 11 shown, the first vehicle makes a line segment b with the second point and the third point as endpoints respectively, and the first vehicle determines this line segment as the second line.

[0321] Case 2: The first vehicle determines the three-dimensional information of the first detection object according to the passable area boundary and the ground contact line on one side of the first detection object.

[0322] Combined with the above S102, in Case 2, the passable area boundary of the first detection object determined by the first vehicle according to the above S102, and the ground contact line of the first detection object respectively have the following characteristics:

[0323] a. The passable area boundary of the first detection object includes multiple boundary points corresponding to the third identifier; the third identifier also corresponds to the third side of the first detection object.

[0324] b. The ground contact lines of the first detection object include the third ground contact line; the third ground contact line is the ground contact line determined by fitting multiple boundary points corresponding to the third identifier.

[0325] c. Among the multiple boundary points with the third identifier, there include the third boundary point and the fourth boundary point.

[0326] The third boundary point is the point among multiple boundary points with the third identifier that is farthest from one end of the third ground contact line.

[0327] The fourth boundary point is the point among multiple boundary points with the third identifier that is farthest from the other end of the third ground contact line.

[0328] Combined with the above S103, in case 2, the three-dimensional information of the first detection object determined by the first vehicle according to the above S103 is determined based on two points and a line corresponding to the first detection object.

[0329] The first of the two points is the projection of the third boundary point on the ground.

[0330] The second of the two points is the projection of the fourth boundary point on the ground.

[0331] Combined with Figure 8, as Figure 12 shown, in case 2, S103 can be specifically implemented through the following S1036 - S1038. Below, S1036 - S1038 will be described in detail.

[0332] S1036: The first vehicle determines the first point according to the third boundary point and the third ground contact line.

[0333] Among them, the first point is the intersection point of the third ground contact line and the fifth straight line. The fifth straight line is a straight line passing through the third boundary point and perpendicular to the third ground contact line.

[0334] In a specific implementation manner, combined with Figure 13 the first detection object in the image to be detected shown in, the method for the first vehicle to determine the first point of the three-dimensional information of the first detection object is:

[0335] Step 1: The first vehicle determines the fifth straight line.

[0336] The fifth straight line is a straight line passing through the third boundary point in the image to be detected and perpendicular to the horizon line.

[0337] In one implementation manner, the first vehicle makes a perpendicular line to the horizon line through the third boundary point, and the first vehicle determines this perpendicular line as the fifth straight line.

[0338] Step 2: The first vehicle determines the intersection point of the fifth straight line and the third ground contact line as the first point.

[0339] In one implementation manner, the first vehicle makes an extension line of the third ground contact line, and the extension line of the third ground contact line intersects the fifth straight line at point e. The first vehicle determines point e as the first point.

[0340] S1037: The first vehicle determines the second point according to the fourth boundary point and the third ground contact line.

[0341] Among them, the second point is the intersection point of the third ground contact line and the sixth straight line. The sixth straight line is a straight line passing through the fourth boundary point and perpendicular to the third ground contact line.

[0342] In a specific implementation, in combination with Figure 13 the first detection object in the to-be-detected image shown in, the method for the first vehicle to determine the second point of the three-dimensional information of the first detection object is:

[0343] Step 3: The first vehicle determines the sixth straight line.

[0344] Among them, the sixth straight line is a straight line passing through the fourth boundary point and perpendicular to the horizon line in the to-be-detected image.

[0345] In an implementation, the first vehicle makes a perpendicular line to the horizon line passing through the fourth boundary point, and the first vehicle determines this perpendicular line as the sixth straight line.

[0346] Step 4: The first vehicle determines the intersection point of the sixth straight line and the third ground contact line as the second point.

[0347] In an implementation, the first vehicle makes an extension line of the third ground contact line. The extension line of the third ground contact line intersects the sixth straight line at point f, and the first vehicle determines point f as the second point.

[0348] S1038: The first vehicle determines the first line according to the first point and the second point.

[0349] In an implementation, as Figure 13 shown, the first vehicle makes a line segment c with the first point and the second point as endpoints respectively, and the first vehicle determines this line segment c as the first line.

[0350] Based on the above technical solution, the first vehicle can determine the three-dimensional information of the first detection object according to the passable area boundary of the first detection object and the ground contact line of the first detection object.

[0351] It should be noted that the above three-dimensional information is the three-dimensional information represented by the first detection object in the to-be-detected image. When the first vehicle needs to determine the true three-dimensional information of the first detection object, it is also necessary to bring the three-dimensional information represented by the first detection object in the to-be-detected image into the vehicle body coordinate system of the first vehicle, so that the first vehicle can determine the position of the first detection object relative to the first vehicle, the size (at least one of length, width, and height) of the first detection object, and the orientation of the first detection object and other information.

[0352] Specifically, in combination with Figure 7 , as Figure 9 shown, after S103, the method further includes:

[0353] S104: The first vehicle inputs the three-dimensional information of the first detection object into the vehicle body coordinate system to determine at least one of the size, direction and relative position of the first detection object.

[0354] In a specific implementation, the first vehicle establishes a first rectangular coordinate system according to the image to be detected; the image to be detected is located in the first rectangular coordinate system. The first rectangular coordinate system can be a matrix pre-set for the image acquisition device, and the pictures collected by the image acquisition device can be mapped to the matrix.

[0355] The first vehicle determines the coordinates of the three-dimensional information of the first detection object in the first rectangular coordinate system. Thereafter, the first vehicle determines the intrinsic parameters and extrinsic parameters of the image acquisition device. The first vehicle converts the coordinates of the three-dimensional information of the first detection object in the first rectangular coordinate system into the coordinates of the vehicle body coordinate system according to the intrinsic parameters and extrinsic parameters of the image acquisition device and the position of the image acquisition device in the vehicle body coordinate system.

[0356] The first vehicle determines the position, movement direction, size and other information of the first detection object according to the coordinates of the three-dimensional information of the first detection object in the vehicle body coordinate system.

[0357] The intrinsic parameters of the image acquisition device are used to characterize some parameters related to the image acquisition device itself, such as the focal length and pixel size of the image acquisition device.

[0358] The external parameters of the image acquisition device are used to characterize the parameters of the image acquisition device in the world coordinate system, such as the position and rotation direction of the image acquisition device in the world coordinate system.

[0359] It should be noted that the first vehicle is pre-set with an intrinsic parameter matrix and an extrinsic parameter matrix of the image acquisition device. When the first vehicle converts a coordinate point in the vehicle body coordinate system into a coordinate point in the first intermediate coordinate system (recorded as the first coordinate point): the first vehicle multiplies the first coordinate point with the extrinsic parameter matrix and the intrinsic parameter matrix in sequence to obtain the coordinate point corresponding to the first coordinate point in the first rectangular coordinate system. When the first vehicle calculates the corresponding point in the vehicle body coordinate system based on the coordinate point in the first rectangular coordinate system, it only needs to perform the opposite operation process to determine the corresponding point in the vehicle body coordinate system based on the coordinate point in the first rectangular coordinate system.

[0360] It should be noted that after S105 , the size of the first detection object determined by the first vehicle includes the length and width of the first monitoring object.

[0361] In order to determine the height of the first detection object, the first vehicle may determine the height of the first monitoring object according to the type of the first detection object.

[0362] For example, when the first detection object is the second vehicle, the first vehicle identifies the vehicle type of the second vehicle and determines the height of the vehicle according to the type of the vehicle.

[0363] In one example, the vehicle type of the second vehicle is the vehicle class, such as: mini car, small car, compact car, medium-sized car, mid-large car, large car, small sport utility vehicle (SUV), compact SUV, medium-sized SUV, mid-large SUV, large SUV, compact multi-purpose vehicle (MPV), medium-sized MPV, mid-large MPV, large MPV, sports car, pickup truck, minivan, light bus, micro-truck, etc.

[0364] The standard sizes of vehicles of each class are pre-configured in the first vehicle. After the first vehicle determines the vehicle class of the second vehicle, it determines the height of the second vehicle according to the vehicle class of the second vehicle.

[0365] In another example, the vehicle type of the second vehicle is the vehicle model (for example, vehicle brand + specific model). The standard sizes of vehicles of each model are pre-configured in the first vehicle. After the first vehicle determines the vehicle model of the second vehicle, it determines the height of the second vehicle according to the vehicle model of the second vehicle.

[0366] It should be noted that according to the above method, the first vehicle can also determine the length and width of the second vehicle according to the type of the second vehicle. The first vehicle can mutually verify the length and width of the second vehicle determined by this method and the length and width determined by the three-dimensional information of the second vehicle in the image to be detected, and determine the accurate length and width of the second vehicle.

[0367] In a possible implementation, after S105, the first vehicle determines the positions, sizes, and movement directions of all vehicles in the image to be detected. The first vehicle plans the driving route of the first vehicle according to the positions, sizes, and movement directions of all vehicles, and the current road information, obstacle information, destination information of the vehicle, etc. determined by other devices in the first vehicle.

[0368] After the first vehicle determines the driving route of the first vehicle, it generates a first control instruction according to the first driving route. The first control instruction is used to instruct the first vehicle to drive according to the planned driving route.

[0369] The first vehicle issues the first control instruction to the first vehicle. The first vehicle performs intelligent driving according to the control instruction issued by the first vehicle.

[0370] Based on the above technical solution, the first vehicle can determine the size, direction, and position of the second vehicle relative to the first vehicle according to the three-dimensional information of the second vehicle. After that, the first vehicle can plan its driving route based on this information to achieve intelligent driving.

[0371] Under the premise of no contradiction, the various solutions in the above embodiments of the present application can be combined.

[0372] The above mainly introduces the solutions of the embodiments of the present application from the perspective of the interaction between various devices. It can be understood that in order to implement the above functions, each device, for example, the first vehicle and the second vehicle, includes at least one of the corresponding hardware structures and software modules for executing various functions. Those skilled in the art should easily realize that, combined with the units and algorithm steps of the examples described in the embodiments disclosed in this article, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0373] It can be understood that in order to implement the functions in the above embodiments, the vehicle includes the corresponding hardware structures and / or software modules for executing various functions. Those skilled in the art should easily realize that, combined with the units and method steps of the examples described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0374] Figure 14 The structural schematic diagram of the device for determining the three-dimensional information of the detection object provided for the embodiments of the present application. These devices for determining the three-dimensional information of the detection object can be used to implement the functions of the processor in the above method embodiments, and thus can also achieve the beneficial effects possessed by the above method embodiments. In the embodiments of the present application, the device for determining the three-dimensional information of the detection object can be, for example, Figure 1 the processor 161 shown in the figure.

[0375] As Figure 14 shown, the device 1400 for determining the three-dimensional information of the detection object includes a processing unit 1410 and a communication unit 1420. The device 1400 for determining the three-dimensional information of the detection object is used to implement the functions of the first vehicle in the above Figure 7 、 Figure 9 , Figure 10 ,or Figure 12 method embodiments shown in the figure.

[0376] When the device 1400 for determining the three-dimensional information of the detection object is used to implement Figure 7 the functions of the processor in the method embodiments shown: The processing unit 1410 is used to execute S102 to S103, and the communication unit 1420 is used to communicate with other entities.

[0377] When the device 1400 for determining the three-dimensional information of the detection object is used to implement Figure 9 the functions of the processor in the method embodiments shown: The processing unit 1410 is used to execute S101, S1021 to S1023, S103, and S104, and the communication unit 1420 is used to communicate with other entities.

[0378] When the device 1400 for determining the three-dimensional information of the detection object is used to implement Figure 10 the functions of the processor in the method embodiments shown: The processing unit 1410 is used to execute S101, S102, and S1031 to S1035, and the communication unit 1420 is used to communicate with other entities.

[0379] When the device 1400 for determining the three-dimensional information of the detection object is used to implement Figure 12 the functions of the processor in the method embodiments shown: The processing unit 1410 is used to execute S101, S102, and S1036 to S1038, and the communication unit 1420 is used to communicate with other entities.

[0380] For a more detailed description of the above-mentioned processing unit 1410 and communication unit 1420, reference can be directly made to Figure 7 、 Figure 9 , Figure 10 or Figure 12 the relevant descriptions in the method embodiments shown, which will not be elaborated here.

[0381] In the implementation process, each step in the method provided in this embodiment can be completed by the integrated logic circuit of the hardware in the processor or the instructions in the form of software. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware processor, or executed and completed by the combination of the hardware and software modules in the processor.

[0382] The processor in this application may include, but is not limited to, at least one of the following: central processing unit (CPU), microprocessor, digital signal processor (DSP), microcontroller unit (MCU), or various computing devices that run software such as artificial intelligence processors. Each computing device may include one or more cores for executing software instructions to perform operations or processing. The processor may be a separate semiconductor chip or may be integrated with other circuits into a semiconductor chip. For example, it may form a system on a chip (SoC) with other circuits (such as codec circuits, hardware acceleration circuits, or various bus and interface circuits), or may be integrated as an embedded processor of an ASIC into the ASIC. The ASIC integrated with the processor may be separately packaged or may be packaged together with other circuits. In addition to the cores for executing software instructions to perform operations or processing, the processor may further include necessary hardware accelerators, such as field programmable gate array (FPGA), programmable logic device (PLD), or logic circuits for implementing dedicated logical operations.

[0383] The memory in the embodiments of this application may include at least one of the following types: read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or may also be electrically erasable programmable read-only memory (EEPROM). In some scenarios, the memory may also be a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0384] The embodiments of this application also provide a computer-readable storage medium, including instructions, which when running on a computer, cause the computer to execute any of the above methods.

[0385] The embodiments of this application also provide a computer program product containing instructions, which when running on a computer, cause the computer to execute any of the above methods.

[0386] An embodiment of the present application further provides a communication system, including: the above-mentioned base station and server.

[0387] An embodiment of the present application further provides a chip, which includes a processor and an interface circuit. The interface circuit is coupled to the processor. The processor is configured to run a computer program or instruction to implement the above method, and the interface circuit is configured to communicate with other modules outside the chip.

[0388] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more integrated media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0389] Although the present application is described herein in connection with various embodiments, however, in the process of implementing the claimed present application, those skilled in the art can understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit can implement several functions recited in the claims. Certain measures are recited in mutually different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0390] Although the present application has been described in connection with specific features and their embodiments, it will be apparent that various modifications and combinations can be made without departing from the spirit and scope of the present application. Accordingly, the present specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these changes and modifications.

[0391] Finally, it should be noted that the above is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for determining three-dimensional information of a detection object, characterized in that Including: Obtain an image to be detected; the image to be detected includes a first detection object. Determine the passable area boundary of the first detection object and the ground contact line of the first detection object; the passable area boundary includes the boundary of the first detection object in the image to be detected; the ground contact line is the connection line of the intersection points of the first detection object and the ground. Determine the three-dimensional information of the first detection object according to the passable area boundary and the ground contact line.

2. The method according to claim 1, wherein The three-dimensional information of the first detection object is used to determine at least one of the size, direction, and relative position of the first detection object.

3. The method according to claim 1 or 2, characterized in that The passable area boundary of the first detection object includes multiple boundary points corresponding to a first identifier and boundary points corresponding to multiple second identifiers; the first identifier also corresponds to a first side of the first detection object, and the second identifier also corresponds to a second side of the first detection object; the first side and the second side are two intersecting sides of the first detection object. The ground contact line includes a first ground contact line and a second ground contact line. The first ground contact line is the ground contact line determined by fitting the multiple boundary points corresponding to the first identifier. The second ground contact line is the ground contact line determined by fitting the multiple boundary points corresponding to the second identifier.

4. The method according to claim 3, wherein The multiple boundary points corresponding to the first identifier include a first boundary point; the first boundary point is the boundary point with the largest distance from the second ground contact line among the multiple boundary points with the first identifier. The multiple boundary points corresponding to the second identifier include a second boundary point; the second boundary point is the boundary point with the largest distance from the first ground contact line among the multiple boundary points with the second identifier.

5. The method according to claim 4, characterized in that, The determining the three-dimensional information of the first detection object includes: Determine the projection of the first boundary point on the ground as a first point. Determine the projection of the second boundary point on the ground as a second point. Determine the intersection point of the line passing through the projection of the second boundary point on the ground and parallel to the second ground contact line and the first ground contact line as a third point. Determine the connection line between the first point and the third point as a first line. Determine the connection line between the second point and the third point as a second line. Determine the three-dimensional information of the first detection object according to the first point, the second point, the third point, the first line, and the second line.

6. The method according to claim 1 or 2, characterized in that, The passable area boundary of the first detection object includes multiple boundary points corresponding to a third identifier; the third identifier also corresponds to a third side of the first detection object; the ground contact line of the first detection object includes a third ground contact line; the third ground contact line is the ground contact line determined by fitting the multiple boundary points corresponding to the third identifier.

7. The method according to claim 6, wherein The multiple boundary points corresponding to the third identifier include a third boundary point and a fourth boundary point. The third boundary point is the point with the farthest distance from one end of the third ground contact line among the multiple boundary points corresponding to the third identifier. The fourth boundary point is the point with the farthest distance from the other end of the third ground contact line among the multiple boundary points corresponding to the third identifier.

8. The method according to claim 7, wherein The determination of the three-dimensional information of the first detection object includes: Determining the projection of the third boundary point on the ground as the first point; Determining the projection of the fourth boundary point on the ground as the second point; Determining the line connecting the first point and the second point as the first line; Determining the three-dimensional information of the first detection object based on the first point, the second point, and the first line.

9. The method according to claim 1 or 2, characterized in that, The method further includes: Inputting the three-dimensional information of the first detection object into the vehicle body coordinate system to determine at least one of the size, direction, and relative position of the first detection object.

10. An apparatus for determining three-dimensional information of a detection object, characterized in that Including: A communication unit and a processing unit; The communication unit is configured to acquire an image to be detected; the image to be detected includes a first detection object; The processing unit is configured to determine the passable area boundary of the first detection object and the ground contact line of the first detection object; the passable area boundary includes the boundary of the first detection object in the image to be detected; the ground contact line is the line connecting the intersection points of the first detection object and the ground; The processing unit is further configured to determine the three-dimensional information of the first detection object based on the passable area boundary and the ground contact line.

11. An apparatus for determining three-dimensional information of a detection object, characterized in that, The device includes a processor and a memory. The memory is configured to store computer programs and instructions, and the processor is configured to execute the computer programs and instructions to implement the method for determining the three-dimensional information of the detection object as described in any one of claims 1-9.

12. An intelligent vehicle, characterized in that, Including a vehicle body, a monocular camera, and the device for determining the three-dimensional information of the detection object as described in claim 10. The monocular camera is configured to collect an image to be detected; the device for determining the three-dimensional information of the detection object is configured to execute the method for determining the three-dimensional information of the detection object as described in any one of claims 1-9 to determine the three-dimensional information of the detection object.

13. The intelligent vehicle according to claim 12, characterized in that, It further includes a display screen; the display screen is configured to display the three-dimensional information of the detection object.

14. An Advanced Driver Assistance System (ADAS), characterized in that, Including the device for determining the three-dimensional information of the detection object as described in claim 10. The device for determining the three-dimensional information of the detection object is configured to execute the method for determining the three-dimensional information of the detection object as described in any one of claims 1-9 to determine the three-dimensional information of the detection object.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions. When the computer instructions run on a computer, the computer is caused to execute the method for determining the three-dimensional information of the detection object as described in any one of claims 1-9.

16. A computer program product, characterized in that, When the computer program product runs on a computer, the computer is caused to execute the method for determining the three-dimensional information of the detection object as described in any one of claims 1-9.

Images