Method for determining vehicle posture, data collection method, device and vehicle

Abstract

The present application discloses a method for determining the posture of a vehicle, a data collection method, device and vehicle for determining the posture of a vehicle. The autonomous driving vehicle obtains a first height representation diagram of an image based on a height representation model and an image collected by a front-view camera, obtains a first sub-image based on an image and a vehicle instance segmentation model, and the image reflects the road conditions in front of the autonomous driving vehicle. The first sub-image is a sub-image corresponding to the first vehicle in front of the autonomous driving vehicle; based on the first height representation diagram and the first sub-image, a second height representation diagram corresponding to the first vehicle is obtained; based on the first contour line in the second height representation diagram, the internal parameter matrix of the front-view camera, and the height of the front-view camera, the posture of the first vehicle is determined. In this way, the height representation diagram of the vehicle is used as an intermediate result, and is calculated in combination with the principle of multi-view geometry to more accurately obtain the posture of other vehicles in front of the autonomous driving vehicle, thereby providing reliable data support for the safe driving of the autonomous driving vehicle.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a method for determining a vehicle posture, a data collection method, a device and a vehicle. Background Art

[0002] In the autonomous driving scenario, there will be various vehicles on the road ahead. The postures of these vehicles will affect the driving decisions of the autonomous driving vehicle. The autonomous driving vehicle needs to successfully avoid these vehicles on the road ahead to avoid friction, collision and other accidents with these vehicles. Therefore, the perception of vehicles on the road ahead of the autonomous driving vehicle is very important. At present, the accuracy of the autonomous driving vehicle in determining the posture of the vehicle on the road ahead is not high.

[0003] Based on this, there is an urgent need to provide a technical solution for determining the vehicle posture for autonomous driving vehicles, which can accurately determine the posture of the vehicle on the road ahead of the autonomous driving vehicle, so as to improve the safety and reliability of the autonomous driving vehicle. Summary of the invention

[0004] The embodiments of the present application provide a method for determining vehicle posture, a data collection method, a device and a vehicle, which overcome the problem of inaccurate vehicle posture recognition on the road ahead during autonomous driving, and improve the safety and reliability of autonomous driving vehicles.

[0005] In a first aspect, an embodiment of the present application provides a method for determining a vehicle posture, which is applied to an autonomous driving vehicle, wherein the autonomous driving vehicle includes a forward-looking camera, a height representation model, and a vehicle instance segmentation model, and the method includes:

[0006] Obtaining a first height representation map of the image according to the height representation model and the image captured by the forward-looking camera, and obtaining at least one sub-image of the image according to the image and the vehicle instance segmentation model, wherein the image reflects the road condition ahead of the autonomous driving vehicle, and the at least one sub-image includes a first sub-image, and the first sub-image is a sub-image corresponding to a first vehicle on the road ahead of the autonomous driving vehicle;

[0007] Obtaining a second height representation map corresponding to the first vehicle according to the first height representation map and the first sub-image, wherein the second height representation map includes a first contour line;

[0008] The posture of the first vehicle is determined according to the first contour line, the intrinsic parameter matrix of the forward-looking camera, and the height of the forward-looking camera relative to the autonomous driving vehicle.

[0009] In one implementation, the height representation model is trained with a training data set, and the qualified model is tested based on a test data set, and the training data set and the test data set are obtained by the method provided in the second aspect described below.

[0010] In one implementation, determining the posture of the first vehicle according to the first contour line, the intrinsic parameter matrix of the forward-looking camera, and the height of the forward-looking camera relative to the autonomous driving vehicle includes:

[0011] Based on the intrinsic parameter matrix of the forward-looking camera, the height of the forward-looking camera relative to the autonomous driving vehicle, and the first point and the second point on the first contour line, determining, in the three-dimensional coordinate system of the forward-looking camera, a first intersection point of a first extension line from the forward-looking camera to the first point and the ground, and a second intersection point of a second extension line from the forward-looking camera to the second point and the ground;

[0012] Based on the first intersection point, the second intersection point and the origin of the three-dimensional coordinate system of the front-view camera, the attitude angle of the first vehicle in the three-dimensional coordinate system of the front-view camera is calculated.

[0013] In one implementation, the second height representation graph further includes a second contour line, and determining the posture of the first vehicle according to the first contour line, an intrinsic parameter matrix of the front-view camera, and a height of the front-view camera relative to the autonomous driving vehicle includes:

[0014] Determining a first attitude angle of the first vehicle according to the first contour line, an intrinsic parameter matrix of the forward-looking camera, and a height of the forward-looking camera relative to the autonomous driving vehicle;

[0015] determining a second attitude angle of the first vehicle according to the second contour line, the intrinsic parameter matrix of the forward-looking camera, and the height of the forward-looking camera relative to the autonomous driving vehicle;

[0016] Based on the first attitude angle and the second attitude angle, a posture of the first vehicle is determined.

[0017] In one implementation, the at least one sub-image further includes a second sub-image, where the second sub-image is a sub-image corresponding to a second vehicle on the road ahead of the autonomous driving vehicle, and the method further includes:

[0018] Obtaining a third height representation map corresponding to the second vehicle according to the first height representation map and the second sub-image, wherein the third height representation map includes a third contour line;

[0019] The posture of the second vehicle is determined according to the third contour line, the intrinsic parameter matrix of the forward-looking camera, and the height of the forward-looking camera relative to the autonomous driving vehicle.

[0020] In one implementation, the height representation model and the vehicle instance segmentation model are integrated into a perception system of the autonomous driving vehicle.

[0021] In a second aspect, an embodiment of the present application further provides a data collection method for determining a vehicle posture, the method comprising:

[0022] Obtaining a first image and a first point cloud data corresponding to the same moment of the data acquisition vehicle;

[0023] Based on the first point cloud data, generating a first label corresponding to the first image, the first label being a height representation map of the first image;

[0024] The first image and the first label are correspondingly saved in a data set, and the data set is used to train or test a height representation model, and the height representation model is applied to the method provided in the first aspect above.

[0025] In one implementation, generating a first label corresponding to the first image based on the first point cloud data includes:

[0026] Projecting the first point cloud data onto the first image to obtain second point cloud data, and obtaining at least one sub-image of the first image according to the first image and a vehicle instance segmentation model, wherein the at least one sub-image includes a first sub-image, and the first sub-image is a sub-image corresponding to a first vehicle on the road ahead of the data acquisition vehicle;

[0027] Obtaining third point cloud data corresponding to the first vehicle according to the second point cloud data and the first sub-image;

[0028] Based on the height value in the third point cloud data, pixel points in the first sub-image are assigned values ​​to obtain a label of the first vehicle in the first image, where the first label includes a label of the first vehicle.

[0029] In one implementation, obtaining third point cloud data corresponding to the first vehicle according to the second point cloud data and the first sub-image includes:

[0030] Based on the second point cloud data and the first sub-image, obtaining fourth point cloud data corresponding to the first vehicle;

[0031] The fourth point cloud data is interpolated to obtain the third point cloud data, and the third point cloud data corresponds one-to-one to the pixels in the first sub-image.

[0032] In one implementation, the data collection vehicle includes a forward-looking camera, a laser radar installed on the top of the data collection vehicle, and a synchronous collection unit. The synchronous collection unit is connected to the forward-looking camera and the laser radar to instruct the forward-looking camera and the laser radar to collect data simultaneously.

[0033] In a third aspect, an embodiment of the present application further provides a device for determining a vehicle posture, which is applied to an autonomous driving vehicle, wherein the autonomous driving vehicle includes a forward-looking camera, a height representation model, and a vehicle instance segmentation model, and the device includes:

[0034] a first obtaining unit, configured to obtain a first height representation map of the image according to the height representation model and the image captured by the front-view camera, wherein the image reflects the road conditions ahead of the autonomous driving vehicle;

[0035] a second obtaining unit, configured to obtain at least one sub-image of the image according to the image and the vehicle instance segmentation model, wherein the at least one sub-image includes a first sub-image, and the first sub-image is a sub-image corresponding to a first vehicle on a road ahead of the autonomous driving vehicle;

[0036] A third obtaining unit, configured to obtain a second height representation map corresponding to the first vehicle according to the first height representation map and the first sub-image, wherein the second height representation map includes a first contour line;

[0037] The first determination unit is used to determine the posture of the first vehicle according to the first contour line, the intrinsic parameter matrix of the forward-looking camera, and the height of the forward-looking camera relative to the autonomous driving vehicle.

[0038] In one implementation, the height representation model is trained with a training data set, and the qualified model is tested based on a test data set, and the training data set and the test data set are obtained by the method provided by the second aspect.

[0039] In one implementation, the first determining unit includes:

[0040] a first determining subunit, configured to determine, in a three-dimensional coordinate system of the forward-looking camera, a first intersection point of a first extension line from the forward-looking camera to the first point and the ground, and a second intersection point of a second extension line from the forward-looking camera to the second point and the ground, based on an intrinsic parameter matrix of the forward-looking camera, a height of the forward-looking camera relative to the autonomous driving vehicle, and a first point and a second point on the first contour line;

[0041] A calculation subunit is used to calculate the attitude angle of the first vehicle in the three-dimensional coordinate system of the front-view camera based on the first intersection point, the second intersection point and the origin of the three-dimensional coordinate system of the front-view camera.

[0042] In one implementation, the second height representation graph further includes a second contour line, and the first determining unit includes:

[0043] a second determination subunit, configured to determine a first attitude angle of the first vehicle according to the first contour line, an intrinsic parameter matrix of the forward-looking camera, and a height of the forward-looking camera relative to the autonomous driving vehicle;

[0044] a third determining subunit, configured to determine a second attitude angle of the first vehicle according to the second contour line, an intrinsic parameter matrix of the forward-looking camera, and a height of the forward-looking camera relative to the autonomous driving vehicle;

[0045] The fourth determining subunit is used to determine the posture of the first vehicle based on the first posture angle and the second posture angle.

[0046] In one implementation, the at least one sub-image further includes a second sub-image, where the second sub-image is a sub-image corresponding to a second vehicle on the road ahead of the autonomous driving vehicle, and the device further includes:

[0047] a fourth obtaining unit, configured to obtain a third height representation map corresponding to the second vehicle according to the first height representation map and the second sub-image, wherein the third height representation map includes a third contour line;

[0048] The second determination unit is used to determine the posture of the second vehicle according to the third contour line, the intrinsic parameter matrix of the forward-looking camera, and the height of the forward-looking camera relative to the autonomous driving vehicle.

[0049] In one implementation, the height representation model and the vehicle instance segmentation model are integrated into a perception system of the autonomous driving vehicle.

[0050] In a fourth aspect, an embodiment of the present application further provides a data acquisition device for determining a vehicle posture, the device comprising:

[0051] A first obtaining unit, used to obtain a first image and a first point cloud data of a data acquisition vehicle corresponding to the same moment;

[0052] a generating unit, configured to generate a first label corresponding to the first image based on the first point cloud data, wherein the first label is a height representation map of the first image;

[0053] A storage unit is used to store the first image and the first label in a data set correspondingly, wherein the data set is used to train or test a height representation model, and the height representation model is applied to the method provided in the first aspect.

[0054] In one implementation, the generating unit includes:

[0055] a first obtaining subunit, configured to project the first point cloud data onto the first image to obtain second point cloud data, and obtain at least one sub-image of the first image according to the first image and a vehicle instance segmentation model, wherein the at least one sub-image includes a first sub-image, and the first sub-image is a sub-image corresponding to a first vehicle on the road ahead of the data acquisition vehicle;

[0056] A second obtaining subunit, configured to obtain third point cloud data corresponding to the first vehicle according to the second point cloud data and the first sub-image;

[0057] An assignment subunit is used to assign values ​​to pixel points in the first sub-image based on the height values ​​in the third point cloud data to obtain a label of the first vehicle in the first image, where the first label includes a label of the first vehicle.

[0058] In one implementation, the second obtaining subunit includes:

[0059] A first obtaining module, configured to obtain fourth point cloud data corresponding to the first vehicle based on the second point cloud data and the first sub-image;

[0060] The second acquisition module is used to interpolate the fourth point cloud data to obtain the third point cloud data, and the third point cloud data corresponds to the pixel points in the first sub-image one by one.

[0061] In one implementation, the data collection vehicle includes a forward-looking camera, a laser radar installed on the top of the data collection vehicle, and a synchronous collection unit. The synchronous collection unit is connected to the forward-looking camera and the laser radar to instruct the forward-looking camera and the laser radar to collect data simultaneously.

[0062] In a fifth aspect, an embodiment of the present application provides an autonomous driving vehicle, the autonomous driving vehicle comprising a forward-looking camera and a perception system, the perception system comprising a height representation model, a vehicle instance segmentation model, and a processing module, wherein:

[0063] The forward-looking camera is used to obtain an image reflecting the road conditions ahead of the autonomous driving vehicle;

[0064] The perception system is used to determine the posture of any other vehicle in the image according to the method provided by the first aspect, wherein:

[0065] The height representation model is used to obtain a height representation map of the image;

[0066] The vehicle instance segmentation model is used to obtain at least one sub-image of the image;

[0067] The processing module is used to determine the posture of any other vehicle on the road ahead of the autonomous driving vehicle based on the height representation map of the image and at least one sub-image of the image.

[0068] In a sixth aspect, an embodiment of the present application further provides a data acquisition system, the data acquisition system comprising a data acquisition vehicle and a server, the data acquisition vehicle comprising a synchronous acquisition unit, a front-view camera, and a laser radar installed on the top of the data acquisition vehicle, wherein:

[0069] The synchronous acquisition unit is used to trigger the front-view camera and the laser radar to simultaneously collect the road conditions in front of the data acquisition vehicle;

[0070] The forward-looking camera is used to obtain a first image at a first moment based on the control of the synchronous acquisition unit;

[0071] The laser radar is used to obtain first point cloud data at the first moment based on the control of the synchronous acquisition unit;

[0072] The server is used to obtain a data set according to the method provided in the second aspect, and the data set is used to train or test the height representation model, and the height representation model is applied to the method provided in the first aspect.

[0073] In the seventh aspect, an embodiment of the present application further provides a model generation server, which is used to train and test the initial height model based on the data set obtained by the data acquisition system provided by the sixth aspect, so as to obtain the height representation model in the method provided by the first aspect.

[0074] In an eighth aspect, an embodiment of the present application provides a device, the device comprising a processor and a memory:

[0075] The memory is used to store computer programs;

[0076] The processor is used to execute the method described in any embodiment of the first aspect or the second aspect according to the computer program.

[0077] In the ninth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method described in any embodiment of the first aspect or the second aspect.

[0078] It can be seen that the embodiments of the present application have the following beneficial effects:

[0079] The embodiment of the present application provides a method for determining the posture of a vehicle, which is applied to an autonomous driving vehicle, wherein the autonomous driving vehicle includes a front-view camera, a height representation model, and a vehicle instance segmentation model, wherein the method includes: obtaining a first height representation map of the image according to the height representation model and the image captured by the front-view camera, obtaining at least one sub-image of the image according to the image and the vehicle instance segmentation model, wherein the image reflects the road conditions ahead of the autonomous driving vehicle, and the at least one sub-image includes a first sub-image, which is a sub-image corresponding to a first vehicle on the road ahead of the autonomous driving vehicle; obtaining a second height representation map corresponding to the first vehicle according to the first height representation map and the first sub-image, wherein the second height representation map includes a first contour line; determining the posture of the first vehicle according to the first contour line, the intrinsic parameter matrix of the front-view camera, and the height of the front-view camera relative to the autonomous driving vehicle. It can be seen that in this method, the height representation map output by the height representation model using deep learning is used as an intermediate result, and theoretical calculations are performed in combination with the principle of multi-view geometry, so as to more accurately obtain the posture of other vehicles driving ahead of the autonomous driving vehicle, thereby providing reliable data support for the safe driving of the autonomous driving vehicle.

[0080] In addition, the embodiment of the present application also provides a data acquisition method for determining the posture of a vehicle, in which a first image and a first point cloud data corresponding to the same moment of the data acquisition vehicle are obtained; based on the first point cloud data, a first label corresponding to the first image is generated, and the first label is a height representation diagram of the first image; the first image and the first label are correspondingly saved in a data set, and the data set is used to train or test the height representation model, and the height representation model is applied to the method for determining the posture of a vehicle provided in the embodiment of the present application. It can be seen that in this method, the data acquisition vehicle can simultaneously collect point cloud data and images that reflect the situation in front of the data acquisition vehicle, and use the point cloud data to generate the posture label of the image. In this way, there is no need to manually label a large amount of sample data, which not only saves the labor cost consumed in the process of generating sample data, but also greatly improves the accuracy of the sample data. In this way, the sample data obtained by the method trains and tests the height representation model, making it possible to obtain a more reasonable height representation model for determining the posture of the vehicle more quickly, thereby providing a guarantee for safer driving of the autonomous driving vehicle integrating the height representation model. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0082] Figure 1 A flow chart of a method for determining a vehicle posture provided in an embodiment of the present application;

[0083] Figure 2 A schematic diagram of a height representation model provided in an embodiment of the present application;

[0084] Figure 3 A flow chart of a data collection method for determining a vehicle posture provided in an embodiment of the present application;

[0085] Figure 4 A schematic diagram of the structure of a device for determining a vehicle posture provided in an embodiment of the present application;

[0086] Figure 5 A schematic diagram of the structure of a data acquisition device for determining a vehicle posture provided in an embodiment of the present application;

[0087] Figure 6 A schematic diagram of the structure of an autonomous driving vehicle provided in an embodiment of the present application;

[0088] Figure 7 A structural diagram of a data acquisition system provided in an embodiment of the present application;

[0089] Figure 8 A schematic diagram of the structure of a device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0090] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It is understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. In addition, it should be noted that, for ease of description, only the parts related to the present application are shown in the accompanying drawings, not all structures.

[0091] The recognition result of the posture of the vehicle on the road ahead by the autonomous driving vehicle often affects the driving decision of the autonomous driving vehicle. At present, autonomous driving vehicles usually determine the posture of the vehicle on the road ahead only based on image detection technology or deep learning methods. In these methods, when encountering some extreme scenarios, it is easy to have the problem of low recognition accuracy or even failure of the recognition method, which will cause immeasurable consequences for the autonomous driving process of the autonomous driving vehicle.

[0092] Based on this, in order to improve the safety of autonomous vehicles, the embodiment of the present application provides a method for accurately determining the posture of a vehicle. The method combines the prediction results of a deep learning method with the precise theoretical calculation of the multi-view geometry principle, and overcomes the problem that the autonomous vehicle is not accurate enough in recognizing the posture of the vehicle on the road ahead during the autonomous driving process. Specifically, the method for determining the posture of a vehicle provided in the embodiment of the present application is applied to an autonomous driving vehicle, the autonomous driving vehicle includes a front-view camera, a height representation model, and a vehicle instance segmentation model. The method includes: obtaining a first height representation map of the image according to the height representation model and the image captured by the front-view camera, obtaining at least one sub-image of the image according to the image and the vehicle instance segmentation model, wherein the image reflects the road ahead of the autonomous driving vehicle, and the at least one sub-image includes a first sub-image, the first sub-image is a sub-image corresponding to the first vehicle on the road ahead of the autonomous driving vehicle; according to the first height representation map and the first sub-image, obtaining a second height representation map corresponding to the first vehicle, the second height representation map includes a first contour line; according to the first contour line, the intrinsic parameter matrix of the front-view camera, and the height of the front-view camera relative to the autonomous driving vehicle, determining the posture of the first vehicle.

[0093] It can be seen that in this method, the height representation map output by the height representation model of deep learning is used as an intermediate result, and theoretical calculations are performed in combination with the principle of multi-perspective geometry to more accurately obtain the posture of other vehicles driving in front of the autonomous driving vehicle, thereby providing reliable data support for the safe driving of autonomous driving vehicles.

[0094] It should be noted that the subject implementing the method for determining the vehicle posture may be a device for determining the vehicle posture provided by the embodiment of the present application, and the device may be carried on an autonomous driving vehicle, for example, it may be carried on an autonomous driving system of an autonomous driving vehicle. The autonomous driving vehicle in the embodiment of the present application may be any vehicle capable of implementing the method for determining the vehicle posture in the embodiment of the present application, for example, it may be a new energy vehicle equipped with the corresponding function of the method for determining the vehicle posture in the embodiment of the present application.

[0095] In addition, the embodiment of the present application also provides a data collection method for determining the posture of a vehicle, in which a first image and a first point cloud data corresponding to the same moment of the data collection vehicle are obtained; based on the first point cloud data, a first label corresponding to the first image is generated, and the first label is a height representation diagram of the first image; the first image and the first label are correspondingly saved in a data set, and the data set is used to train or test a height representation model, and the height representation model is applied to the method for determining the posture of a vehicle provided in the embodiment of the present application. It can be seen that in this method, the data collection vehicle can simultaneously collect point cloud data and images that reflect the situation in front of the data collection vehicle, and use the point cloud data to generate the posture label of the image. In this way, there is no need to manually label a large amount of sample data, which not only saves the labor cost consumed in the process of generating sample data, but also greatly improves the accuracy of the sample data.

[0096] In this way, the sample data obtained through this method is used to train and test the height representation model, making it possible to obtain a more reasonable height representation model for determining the vehicle posture more quickly, thereby providing a guarantee for safer driving of the autonomous driving vehicle integrating the height representation model.

[0097] It should be noted that the subject implementing the data collection method for determining the vehicle posture can be a data collection device for determining the vehicle posture provided in the embodiment of the present application, which can be carried on a data collection vehicle, or can be two functional modules that are mutually communicated and integrated into the data collection vehicle and the server. The data collection vehicle in the embodiment of the present application can be any vehicle that can implement the data collection method for determining the vehicle posture in the embodiment of the present application.

[0098] To facilitate understanding of the specific implementation of the method for determining the vehicle posture provided in the embodiment of the present application, it will be described below with reference to the accompanying drawings.

[0099] Figure 1 A flow chart of a method for determining vehicle posture provided in an embodiment of the present application. The method is applied to an autonomous driving vehicle, the autonomous driving vehicle comprising a front-view camera, a height representation model, and a vehicle instance segmentation model. Optionally, the height representation model and the vehicle instance segmentation model can be integrated into the perception system of the autonomous driving vehicle as two functional modules.

[0100] like Figure 1 As shown, the method may include the following S101 to S104:

[0101] S101, obtaining a first height representation graph of the image according to the height representation model and the image captured by the forward-looking camera, wherein the image reflects the road condition ahead of the autonomous driving vehicle.

[0102] The forward-looking camera refers to a camera installed in front of the autonomous vehicle and capable of capturing the road conditions in front of the autonomous vehicle, such as a camera installed on the front windshield of the autonomous vehicle. The forward-looking camera can collect images continuously or at preset intervals. The image in S101 can be any frame of the image collected by the forward-looking camera, and each frame of the image collected by the forward-looking camera can execute the method to determine the posture of the vehicle in the image.

[0103] The height representation model is a model that has been trained with a training data set and has passed the test based on a test data set. Passing the test means that the trained height representation model can more accurately predict the height of the vehicle in the image. For example, a preset accuracy threshold (such as 90%) can be set to test the trained height representation model. If the accuracy of the height representation model is greater than the preset accuracy threshold, the height representation model is considered to have passed the test and is a qualified model.

[0104] It should be noted that the training data set and the test data set in the embodiment of the present application are obtained through the data collection method provided in the embodiment of the present application. For specific contents, please refer to the relevant description of the corresponding embodiments of S301 to S303 below.

[0105] The height representation model may be a model that uses a deep learning structure to learn the vehicle height representation. As an example, the height representation model may refer to Figure 2 As shown, it includes an encoding part 201, a decoding part 202 and an output part 203, wherein the encoding part 201 is used to extract image features, the decoding part 202 is used to regress the height representation features, and the output part 203 is used to output the height representation results. Figure 2 This is only an exemplary introduction to the height representation model, and the embodiments of the present application do not limit its specific structure. Any model structure that can implement the functions of the height representation model in the embodiments of the present application is acceptable.

[0106] The height representation model may be a model that takes an image as input and outputs a height representation map. As an example, in S101, the autonomous driving vehicle may input an image captured by a front-view camera into the height representation model, and the height representation model may output a first height representation map of the image.

[0107] The height representation map can be represented as data including a matrix (h, w, a), where h is the height of the image, w is the width of the image, a is the height corresponding to the pixel, and the value of a is greater than or equal to 0 and less than or equal to 1. The height representation map is used to reflect the height of each pixel in the image.

[0108] S102, obtaining at least one sub-image of the image according to the image and the vehicle instance segmentation model, wherein the at least one sub-image includes a first sub-image, and the first sub-image is a sub-image corresponding to a first vehicle on the road ahead of the autonomous driving vehicle.

[0109] The vehicle instance segmentation model is a trained and tested model that is used to segment vehicles in an image to obtain vehicle instances in the image. The present application does not limit the specific structure of the vehicle instance segmentation model, and any model structure that can implement the vehicle instance segmentation model function in the present application embodiment is acceptable.

[0110] The vehicle instance segmentation model may be a model that takes an image as input and outputs a vehicle instance. As an example, in S102, the autonomous driving vehicle may input an image captured by a front-view camera into the vehicle instance segmentation model, and the vehicle instance segmentation model may output at least one sub-image of the image, each of which includes a vehicle on the road ahead of the autonomous driving vehicle.

[0111] Since the method for determining the vehicle posture in the embodiment of the present application executes the same steps for each other vehicle on the road ahead of the autonomous driving vehicle, the following description takes the first vehicle on the road ahead of the autonomous driving vehicle as an example, and the first vehicle corresponds to the first sub-image in the at least one sub-image.

[0112] It can be seen that, through S101 and S102, the first height representation image of the image and the first sub-image of the first vehicle are obtained respectively, which provide a data basis for the subsequent determination of the posture of the first vehicle.

[0113] It should be noted that there is no restriction on the execution order of S101 and S102. S101 may be executed first and then S102, or S102 may be executed first and then S101, or S101 and S102 may be executed simultaneously.

[0114] S103: Obtain a second height representation map corresponding to the first vehicle according to the first height representation map and the first sub-image, where the second height representation map includes a first contour line.

[0115] The first height representation image corresponds to the entire image, and the first sub-image corresponds to the first vehicle. Then, in order to determine the posture of the first vehicle, a second height representation image corresponding to the first vehicle can be obtained according to S103. The second height representation image reflects the height of each pixel in the first sub-image, including the height of the pixel corresponding to the first vehicle.

[0116] S104: Determine the posture of the first vehicle according to the first contour line, the intrinsic parameter matrix of the forward-looking camera, and the height of the forward-looking camera relative to the autonomous driving vehicle.

[0117] It should be noted that the first vehicle in the first sub-image includes multiple contour lines, and the first contour line refers to any one of the contour lines. The embodiment of the present application uses part or all of these contour lines to determine the vehicle posture.

[0118] As an example, the autonomous driving vehicle can determine the posture of the first vehicle based on a contour line. In specific implementation, S104 may include: S104a1, based on the intrinsic parameter matrix K of the front-view camera, the height H of the front-view camera relative to the autonomous driving vehicle, the first point p0 and the second point p1 on the first contour line, in the three-dimensional coordinate system of the front-view camera, determine the first intersection point p0_3d of the first extension line from the front-view camera to the first point p0 and the ground, and the second intersection point p1_3d of the second extension line from the front-view camera to the second point p1 and the ground; S104a2, based on the first intersection point p0_3d, the second intersection point p1_3d and the origin O of the three-dimensional coordinate system of the front-view camera, calculate the posture angle A of the first vehicle in the three-dimensional coordinate system of the front-view camera. The posture angle A can be used to reflect the posture of the first vehicle relative to the autonomous driving vehicle, and is used by the autonomous driving vehicle to determine the autonomous driving strategy.

[0119] As another example, the autonomous driving vehicle may determine the posture of the first vehicle based on multiple contour lines. In specific implementation, taking the autonomous driving vehicle determining the posture of the first vehicle based on two contour lines (i.e., the first contour line and the second contour line) as an example, S104 may include: S104b1, determining the first posture angle A1 of the first vehicle according to the first contour line, the intrinsic parameter matrix K of the front-view camera, and the height H of the front-view camera relative to the autonomous driving vehicle; S104b2, determining the second posture angle A2 of the first vehicle according to the second contour line, the intrinsic parameter matrix K of the front-view camera, and the height H of the front-view camera relative to the autonomous driving vehicle; S104b3, determining the posture A' of the first vehicle based on the first posture angle A1 and the second posture angle A2. Among them, the specific implementation of S104b1 and S104b2 can refer to the methods described in S104a1 to S104a3 above. For S104b3, A1 and A2 may be optimized to determine A'. For example, A1 and A2 may be weighted averaged to determine A'. The weight may be determined based on experience or experiment. For another example, A1 and A2 may be directly averaged to determine A'.

[0120] In some implementations, the at least one sub-image obtained in S102 may further include a second sub-image, where the second sub-image is a sub-image corresponding to a second vehicle on the road ahead of the autonomous driving vehicle. Then, the method may further include:

[0121] S105, obtaining a third height representation map corresponding to the second vehicle according to the first height representation map and the second sub-image, wherein the third height representation map includes a third contour line;

[0122] S106: Determine the posture of the second vehicle according to the third contour line, the intrinsic parameter matrix of the forward-looking camera, and the height of the forward-looking camera relative to the autonomous driving vehicle.

[0123] It should be noted that the specific implementation of S105 and S106 and the effects achieved can refer to the relevant description of S103 and S104.

[0124] It can be seen that through this method, the height representation map output by the height representation model of deep learning is used as the intermediate result, and the multi-view geometry principle is combined for theoretical calculation to more accurately obtain the posture of the other car driving in front of the autonomous driving vehicle, which provides reliable data support for the safe driving of the autonomous driving vehicle.

[0125] The embodiment of the present application also provides a data collection method for determining a vehicle posture. The method can be applied to the following data collection system 700, and can also be applied to the server 720 in the data collection system 700. Figure 3 A schematic diagram of a data collection method for determining vehicle posture provided in an embodiment of the present application. Figure 3 As shown, the method may include the following S301 to S303:

[0126] S301, obtaining a first image and a first point cloud data of a data acquisition vehicle corresponding to the same moment.

[0127] The data collection vehicle may include: a forward-looking camera, a laser radar installed on the top of the data collection vehicle, and a synchronous collection unit, wherein the synchronous collection unit is connected to the forward-looking camera and the laser radar to instruct the forward-looking camera and the laser radar to collect data simultaneously.

[0128] As an example, the synchronous acquisition unit of the data acquisition vehicle triggers the front-view camera and the laser radar to work, and obtains the first image collected by the front-view camera and the first point cloud data collected by the laser radar at that moment. It can be understood that at the same moment, the situation on the road in front of the data acquisition vehicle is the same, so the information represented by the first image and the first point cloud data is consistent with the actual scene. The embodiment of the present application uses this feature to automatically generate sample data that can be used to train and test the initial height model to obtain the height representation model.

[0129] S302: Generate a first label corresponding to the first image based on the first point cloud data, where the first label is a height representation map of the first image.

[0130] In the embodiment of the present application, in order to save the work of manually labeling vehicles in the image, a first label corresponding to the first image is automatically generated based on the first point cloud data.

[0131] As an example, S302 may include: S3021, projecting the first point cloud data c onto the first image x to obtain the second point cloud data g1, and obtaining at least one sub-image of the first image according to the first image x and the vehicle instance segmentation model, the at least one sub-image including the first sub-image x1, the first sub-image x1 being a sub-image corresponding to the first vehicle v1 on the road ahead of the data acquisition vehicle; S3022, obtaining the third point cloud data g3 corresponding to the first vehicle v1 according to the second point cloud data g1 and the first sub-image x1; S3023, assigning values ​​to the pixels in the first sub-image x1 based on the height values ​​in the third point cloud data g3, obtaining the label L1 of the first vehicle v1 in the first image x, the first label L including the label L1 of the first vehicle v1. It should be noted that the first label L also includes the labels of other vehicles in the first image x, and the method for determining the labels of other vehicles can refer to the method for determining the label L1 of the first vehicle v1 in this example.

[0132] For example, the method may include: S3022a, based on the second point cloud data g1 and the first sub-image x1, obtaining the fourth point cloud data g2 corresponding to the first vehicle v1; S3022b, interpolating the fourth point cloud data g2 to obtain the third point cloud data g3, wherein the third point cloud data g3 corresponds to the pixel points in the first sub-image x1 one by one. Since the first point cloud data c cannot correspond to each pixel point of the first image x, the fourth point cloud data g2 may be interpolated in S3022 to obtain the third point cloud data g3 corresponding to the pixel points on the first sub-image x1 of the first vehicle v1 one by one, so as to prepare for the subsequent assignment to determine the height representation of each pixel point.

[0133] For S3023, the height value in the third point cloud data g3 can be taken and normalized, and the normalized height value can be assigned to the pixel point corresponding to the first sub-image. In this way, each pixel point in the first sub-image can correspond to (h, w, a) (i.e., height label), where for the pixel point under the vehicle, a is 0; for the pixel point on the roof, a is 1, and other pixel points not related to the first vehicle are 255.

[0134] S303, correspondingly saving the first image and the first label in a data set, wherein the data set is used to train or test a height representation model, and the height representation model is applied to the method for determining a vehicle posture provided in an embodiment of the present application.

[0135] It should be noted that, for the first image, each sub-image corresponds to a label, and the set of these labels can be recorded as the first label, that is, the number of vehicles and labels included in the first image corresponds one to one, and each vehicle corresponds to a label. In this way, a set of sample data is obtained, and all sample data obtained based on this method are saved in a data set to obtain a data set for training or testing a height representation model.

[0136] In this way, through this method, there is no need to manually label a large amount of sample data, which not only saves the labor cost consumed in the process of generating sample data, but also greatly improves the accuracy of sample data. Therefore, the data set obtained by this method is used to train and test the height representation model, making it possible to obtain a more reasonable height representation model for determining the vehicle posture more quickly, providing a guarantee for safer driving of the autonomous driving vehicle integrated with the height representation model.

[0137] It should be noted that in the embodiment of the present application, the data set obtained based on the method can be divided into a training data set and a test data set, and the model generation service trains and tests the height representation model based on the training data set and the test data set, respectively, to obtain the height representation model used in the method for determining the vehicle posture provided in the embodiment of the present application. As an example, the process of training the height representation model using the training data set can be understood as: inputting the images in each sample data in the training data set into the initial height model in turn, obtaining the corresponding predicted height representation, using the predicted height representation and the calibrated height representation corresponding to the image in the sample data, and continuously correcting the initial height model until all the sample data in the training data set are used for training, and the height representation model finally obtained is the trained height representation model. The process of testing the trained height representation model using a test data set can be understood as: inputting the images in each sample data in the test data set into the trained height representation model in turn to obtain the corresponding predicted height representation; if the predicted height representation is consistent with the calibrated height representation corresponding to the image in the sample data, then the test result corresponding to the sample data is considered accurate; otherwise, it is considered inaccurate; if the test result of the sample data in the test data set indicates that the accuracy of the trained height representation model reaches a preset accuracy threshold, then it is considered that the trained height representation model can be used as the height representation model in S101; otherwise, the height representation model is retrained and predicted.

[0138] Accordingly, the embodiment of the present application also provides a device 400 for determining a vehicle posture, see Figure 4 The device 400 is applied to an autonomous driving vehicle, wherein the autonomous driving vehicle includes a front-view camera, a height representation model, and a vehicle instance segmentation model, and the device 400 includes:

[0139] A first obtaining unit 401 is used to obtain a first height representation map of the image according to the height representation model and the image captured by the front-view camera, wherein the image reflects the road conditions ahead of the autonomous driving vehicle;

[0140] A second obtaining unit 402 is configured to obtain at least one sub-image of the image according to the image and the vehicle instance segmentation model, wherein the at least one sub-image includes a first sub-image, and the first sub-image is a sub-image corresponding to a first vehicle on the road ahead of the autonomous driving vehicle;

[0141] A third obtaining unit 403 is configured to obtain a second height representation map corresponding to the first vehicle according to the first height representation map and the first sub-image, wherein the second height representation map includes a first contour line;

[0142] The first determination unit 404 is used to determine the posture of the first vehicle according to the first contour line, the intrinsic parameter matrix of the forward-looking camera, and the height of the forward-looking camera relative to the autonomous driving vehicle.

[0143] In one implementation, the height representation model is trained with a training data set, and the qualified model is tested based on a test data set, and the training data set and the test data set are obtained by the method provided by the second aspect.

[0144] In one implementation, the first determining unit 404 includes:

[0145] a first determining subunit, configured to determine, in a three-dimensional coordinate system of the forward-looking camera, a first intersection point of a first extension line from the forward-looking camera to the first point and the ground, and a second intersection point of a second extension line from the forward-looking camera to the second point and the ground, based on an intrinsic parameter matrix of the forward-looking camera, a height of the forward-looking camera relative to the autonomous driving vehicle, and a first point and a second point on the first contour line;

[0146] A calculation subunit is used to calculate the attitude angle of the first vehicle in the three-dimensional coordinate system of the front-view camera based on the first intersection point, the second intersection point and the origin of the three-dimensional coordinate system of the front-view camera.

[0147] In one implementation, the second height representation diagram further includes a second contour line, and the first determining unit 404 includes:

[0148] a second determination subunit, configured to determine a first attitude angle of the first vehicle according to the first contour line, an intrinsic parameter matrix of the forward-looking camera, and a height of the forward-looking camera relative to the autonomous driving vehicle;

[0149] a third determining subunit, configured to determine a second attitude angle of the first vehicle according to the second contour line, an intrinsic parameter matrix of the forward-looking camera, and a height of the forward-looking camera relative to the autonomous driving vehicle;

[0150] The fourth determining subunit is used to determine the posture of the first vehicle based on the first posture angle and the second posture angle.

[0151] In one implementation, the at least one sub-image further includes a second sub-image, where the second sub-image is a sub-image corresponding to a second vehicle on the road ahead of the autonomous driving vehicle, and the device 400 further includes:

[0152] a fourth obtaining unit, configured to obtain a third height representation map corresponding to the second vehicle according to the first height representation map and the second sub-image, wherein the third height representation map includes a third contour line;

[0153] The second determination unit is used to determine the posture of the second vehicle according to the third contour line, the intrinsic parameter matrix of the forward-looking camera, and the height of the forward-looking camera relative to the autonomous driving vehicle.

[0154] In one implementation, the height representation model and the vehicle instance segmentation model are integrated into a perception system of the autonomous driving vehicle.

[0155] It should be noted that the above description is a description of a device for determining a vehicle posture. For specific implementation methods and effects achieved, please refer to Figure 1 The description of the embodiment of the method for determining the vehicle posture shown is not repeated here.

[0156] Accordingly, the present application also provides a data acquisition device 500 for determining a vehicle posture, see Figure 5 The device 500 comprises:

[0157] A first obtaining unit 501 is used to obtain a first image and a first point cloud data of a data acquisition vehicle corresponding to the same moment;

[0158] A generating unit 502 is configured to generate a first label corresponding to the first image based on the first point cloud data, wherein the first label is a height representation map of the first image;

[0159] The saving unit 503 is used to save the first image and the first label in a data set correspondingly, and the data set is used to train or test the height representation model, and the height representation model is applied to the method for determining the vehicle posture provided in an embodiment of the present application.

[0160] In one implementation, the generating unit 502 includes:

[0161] a first obtaining subunit, configured to project the first point cloud data onto the first image to obtain second point cloud data, and obtain at least one sub-image of the first image according to the first image and a vehicle instance segmentation model, wherein the at least one sub-image includes a first sub-image, and the first sub-image is a sub-image corresponding to a first vehicle on the road ahead of the data acquisition vehicle;

[0162] A second obtaining subunit, configured to obtain third point cloud data corresponding to the first vehicle according to the second point cloud data and the first sub-image;

[0163] An assignment subunit is used to assign values ​​to pixel points in the first sub-image based on the height values ​​in the third point cloud data to obtain a label of the first vehicle in the first image, where the first label includes a label of the first vehicle.

[0164] In one implementation, the second obtaining subunit includes:

[0165] A first obtaining module, configured to obtain fourth point cloud data corresponding to the first vehicle based on the second point cloud data and the first sub-image;

[0166] The second acquisition module is used to interpolate the fourth point cloud data to obtain the third point cloud data, and the third point cloud data corresponds to the pixel points in the first sub-image one by one.

[0167] In one implementation, the data collection vehicle includes a forward-looking camera, a laser radar installed on the top of the data collection vehicle, and a synchronous collection unit. The synchronous collection unit is connected to the forward-looking camera and the laser radar to instruct the forward-looking camera and the laser radar to collect data simultaneously.

[0168] It should be noted that the above description is a description of a data acquisition device for determining a vehicle posture. For specific implementation methods and effects achieved, please refer to Figure 3 The description of the embodiment of the data collection method for determining the vehicle posture shown is not repeated here.

[0169] In addition, the present application embodiment provides an autonomous driving vehicle 600, such as Figure 6 As shown, the autonomous driving vehicle 600 includes a front-view camera 610 and a perception system 620, wherein the perception system 620 includes a height representation model 621, a vehicle instance segmentation model 622, and a processing module 623, wherein:

[0170] The forward-looking camera 610 is used to obtain an image reflecting the road conditions ahead of the autonomous driving vehicle 600;

[0171] The perception system 620 is used to determine the posture of any other vehicle in the image according to the method for determining the posture of a vehicle provided in an embodiment of the present application, wherein:

[0172] The height representation model 621 is used to obtain a height representation map of the image;

[0173] The vehicle instance segmentation model 622 is used to obtain at least one sub-image of the image;

[0174] The processing module 623 is used to determine the posture of any other vehicle on the road ahead of the autonomous driving vehicle based on the height representation map of the image and at least one sub-image of the image.

[0175] In addition, the present application embodiment also provides a data acquisition system 700, such as Figure 7As shown, the data collection system 700 includes a data collection vehicle 710 and a server 720, wherein the data collection vehicle 710 includes a synchronous collection unit 711, a front-view camera 712, and a laser radar 713 installed on the top of the data collection vehicle 710, wherein:

[0176] The synchronous acquisition unit 711 is used to trigger the front-view camera 712 and the laser radar 713 to simultaneously collect the road conditions in front of the data acquisition vehicle 710;

[0177] The forward-looking camera 712 is used to obtain a first image at a first moment based on the control of the synchronous acquisition unit;

[0178] The laser radar 713 is used to obtain first point cloud data at the first moment based on the control of the synchronous acquisition unit;

[0179] The server 720 is used to obtain a data set according to the data collection method for determining the vehicle posture provided in the embodiment of the present application, and the data set is used to train or test the height representation model, and the height representation model is applied to the method for determining the vehicle posture provided in the embodiment of the present application.

[0180] In addition, an embodiment of the present application also provides a model generation server, which is used to train and test the initial height model based on the data set obtained by the aforementioned data acquisition system 700, and obtain the height representation model in the method for determining the vehicle posture provided in an embodiment of the present application.

[0181] In addition, the embodiment of the present application also provides a device 800, such as Figure 8 As shown, the device 800 includes a processor 801 and a memory 802:

[0182] The memory 802 is used to store computer programs;

[0183] The processor 801 is used to execute a method for determining a vehicle posture or a data collection method for determining a vehicle posture provided in an embodiment of the present application according to the computer program.

[0184] In addition, an embodiment of the present application also provides a computer-readable storage medium, which is used to store a computer program, and the computer program is used to execute a method for determining a vehicle posture or a data collection method for determining a vehicle posture provided in an embodiment of the present application.

[0185] Through the description of the above implementation methods, it can be known that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in each embodiment of the present application or some parts of the embodiments.

[0186] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for system embodiments and device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device and system embodiments described above are merely schematic, in which the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0187] The above is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. It should be noted that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the present application, and these improvements and modifications should also be regarded as the protection scope of the present application.

Claims

1. A method for determining a vehicle posture, characterized in that: Applied to an autonomous driving vehicle, the autonomous driving vehicle comprising a front-view camera, a height representation model and a vehicle instance segmentation model, the method comprising: A first height representation graph of the image is obtained according to the height representation model and the image captured by the front-view camera, and at least one sub-image of the image is obtained according to the image and the vehicle instance segmentation model, wherein the image reflects the road conditions ahead of the autonomous driving vehicle, and the at least one sub-image includes a first sub-image, which is a sub-image corresponding to a first vehicle on the road ahead of the autonomous driving vehicle; the height representation model is a model for learning vehicle height representation using a deep learning structure; and the first height representation graph is used to reflect the height of each pixel in the image; According to the first height representation map and the first sub-image, a second height representation map corresponding to the first vehicle is obtained, wherein the second height representation map includes a first contour line; the second height representation map is used to reflect the height of each pixel point in the first sub-image; Determine the posture of the first vehicle according to the first contour line, the intrinsic parameter matrix of the forward-looking camera, and the height of the forward-looking camera relative to the autonomous driving vehicle, including: determining, in the three-dimensional coordinate system of the forward-looking camera, a first intersection point of a first extension line from the forward-looking camera to the first point and the ground, and a second intersection point of a second extension line from the forward-looking camera to the second point and the ground based on the intrinsic parameter matrix of the forward-looking camera, the height of the forward-looking camera relative to the autonomous driving vehicle, and a first point and a second point on the first contour line; and calculate the posture angle of the first vehicle in the three-dimensional coordinate system of the forward-looking camera based on the first intersection point, the second intersection point, and the origin of the three-dimensional coordinate system of the forward-looking camera.

2. The method according to claim 1, characterized in that The height representation model is a model that has been trained with a training data set and has passed the test based on a test data set. The process of obtaining the training data set and the test data set includes: obtaining a first image and a first point cloud data corresponding to the same moment of the data acquisition vehicle; generating a first label corresponding to the first image based on the first point cloud data, wherein the first label is a height representation map of the first image; and correspondingly saving the first image and the first label in the data set, dividing the data set into the training data set and the test data set.

3. The method according to claim 1, characterized in that The second height representation diagram further includes a second contour line, and determining the posture of the first vehicle according to the first contour line, the intrinsic parameter matrix of the front-view camera, and the height of the front-view camera relative to the autonomous driving vehicle includes: Determining a first attitude angle of the first vehicle according to the first contour line, an intrinsic parameter matrix of the forward-looking camera, and a height of the forward-looking camera relative to the autonomous driving vehicle; determining a second attitude angle of the first vehicle according to the second contour line, the intrinsic parameter matrix of the forward-looking camera, and the height of the forward-looking camera relative to the autonomous driving vehicle; Based on the first attitude angle and the second attitude angle, a posture of the first vehicle is determined.

4. The method according to claim 1, characterized in that: The at least one sub-image further includes a second sub-image, where the second sub-image is a sub-image corresponding to a second vehicle on the road ahead of the autonomous driving vehicle, and the method further includes: Obtaining a third height representation map corresponding to the second vehicle according to the first height representation map and the second sub-image, wherein the third height representation map includes a third contour line; The posture of the second vehicle is determined according to the third contour line, the intrinsic parameter matrix of the forward-looking camera, and the height of the forward-looking camera relative to the autonomous driving vehicle.

5. The method according to any one of claims 1 to 4, characterized in that: The height representation model and the vehicle instance segmentation model are integrated into a perception system of the autonomous driving vehicle.

6. A data collection method for determining a vehicle posture, characterized in that: The method comprises: Obtaining a first image and a first point cloud data corresponding to the same moment of the data acquisition vehicle; Based on the first point cloud data, generating a first label corresponding to the first image, the first label being a height representation map of the first image; The first image and the first label are correspondingly saved in a data set, and the data set is used to train or test a height representation model, and the height representation model is applied to the method described in any one of claims 1 to 5 above.

7. The method according to claim 6, characterized in that The step of generating a first label corresponding to the first image based on the first point cloud data includes: Projecting the first point cloud data onto the first image to obtain second point cloud data, and obtaining at least one sub-image of the first image according to the first image and a vehicle instance segmentation model, wherein the at least one sub-image includes a first sub-image, and the first sub-image is a sub-image corresponding to a first vehicle on the road ahead of the data acquisition vehicle; Obtaining third point cloud data corresponding to the first vehicle according to the second point cloud data and the first sub-image; Based on the height value in the third point cloud data, pixel points in the first sub-image are assigned values ​​to obtain a label of the first vehicle in the first image, where the first label includes a label of the first vehicle.

8. The method according to claim 7, characterized in that The obtaining, according to the second point cloud data and the first sub-image, third point cloud data corresponding to the first vehicle comprises: Based on the second point cloud data and the first sub-image, obtaining fourth point cloud data corresponding to the first vehicle; The fourth point cloud data is interpolated to obtain the third point cloud data, and the third point cloud data corresponds one-to-one to the pixels in the first sub-image.

9. The method according to any one of claims 6 to 8, characterized in that: The data collection vehicle includes a forward-looking camera, a laser radar installed on the top of the data collection vehicle, and a synchronous collection unit. The synchronous collection unit is connected to the forward-looking camera and the laser radar to instruct the forward-looking camera and the laser radar to collect data simultaneously.

10. An autonomous driving vehicle, characterized in that: The autonomous driving vehicle includes a forward-looking camera and a perception system, wherein the perception system includes a height representation model, a vehicle instance segmentation model, and a processing module, wherein: The forward-looking camera is used to obtain an image reflecting the road conditions ahead of the autonomous driving vehicle; The perception system is used to determine the posture of any other vehicle in the image according to the method according to any one of claims 1 to 5, wherein: The height representation model is used to obtain a height representation map of the image; The vehicle instance segmentation model is used to obtain at least one sub-image of the image; The processing module is used to determine the posture of any other vehicle on the road ahead of the autonomous driving vehicle based on the height representation map of the image and at least one sub-image of the image.

11. A data acquisition system, characterized in that: The data acquisition system includes a data acquisition vehicle and a server, wherein the data acquisition vehicle includes a synchronous acquisition unit, a front-view camera, and a laser radar installed on the top of the data acquisition vehicle, wherein: The synchronous acquisition unit is used to trigger the front-view camera and the laser radar to simultaneously collect the road conditions in front of the data acquisition vehicle; The forward-looking camera is used to obtain a first image at a first moment based on the control of the synchronous acquisition unit; The laser radar is used to obtain first point cloud data at the first moment based on the control of the synchronous acquisition unit; The server is used to obtain a data set according to the method described in any one of claims 6 to 9, wherein the data set is used to train or test a height representation model, and the height representation model is applied to the method described in any one of claims 1 to 5.

Images