Vehicle posture detection method, device, computer equipment and storage medium

Abstract

The present application provides a vehicle posture detection method, device, computer equipment and storage medium, which relate to the field of computer technology. The method comprises: obtaining a pre-collected two-dimensional image of a vehicle to be detected at a preset viewing angle, and three-dimensional point cloud data of the vehicle to be detected at a preset viewing angle; identifying the key points of the vehicle in the two-dimensional image to obtain the two-dimensional coordinates of multiple key points in the vehicle to be detected; performing three-dimensional mapping on the two-dimensional coordinates of multiple key points to obtain the three-dimensional coordinates of multiple key points; determining the preset reference coordinate system of the vehicle to be detected based on the three-dimensional point cloud data and the three-dimensional coordinates of multiple key points; determining the spatial posture of the vehicle to be detected based on the preset reference coordinate system. The present application can realize accurate detection of vehicle posture.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a vehicle posture detection method, device, computer equipment and storage medium. Background Art

[0002] With the development of artificial intelligence technology, during construction operations, construction machines can be remotely controlled by simulating the cockpit of a real construction machine and integrating visual image transmission.

[0003] Remote control of construction machines not only allows operators to work in a comfortable environment, but also ensures the operator's personal safety and improves work quality and efficiency.

[0004] During the remote control process, in order to ensure the safety of the operation and even further realize the automatic operation of the construction machine, it is necessary to ensure the accurate positioning of the working object of the construction machine. Summary of the invention

[0005] The purpose of the present disclosure is to provide a vehicle posture detection method, device, computer equipment and storage medium to address the deficiencies in the above-mentioned prior art, so as to achieve accurate detection of vehicle posture.

[0006] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0007] In a first aspect, an embodiment of the present application provides a vehicle posture detection method, the method comprising:

[0008] Acquire a pre-collected two-dimensional image of the vehicle to be detected at a preset viewing angle, and three-dimensional point cloud data of the vehicle to be detected at the preset viewing angle;

[0009] Identify key points of the vehicle on the two-dimensional image to obtain two-dimensional coordinates of multiple key points in the vehicle to be detected;

[0010] Performing three-dimensional mapping on the two-dimensional coordinates of the multiple key points to obtain the three-dimensional coordinates of the multiple key points;

[0011] Determining a preset reference coordinate system of the vehicle to be detected according to the three-dimensional point cloud data and the three-dimensional coordinates of the plurality of key points;

[0012] The spatial position and posture of the vehicle to be detected is determined according to the preset reference coordinate system.

[0013] In a second aspect, an embodiment of the present application further provides a vehicle posture detection device, the device comprising:

[0014] A vehicle data acquisition module, used to acquire a pre-collected two-dimensional image of the vehicle to be detected at a preset viewing angle, and three-dimensional point cloud data of the vehicle to be detected at the preset viewing angle;

[0015] A key point recognition module, used to recognize key points of the vehicle in the two-dimensional image, and obtain two-dimensional coordinates of multiple key points in the vehicle to be detected;

[0016] A three-dimensional mapping module, used for performing three-dimensional mapping on the two-dimensional coordinates of the multiple key points to obtain the three-dimensional coordinates of the multiple key points;

[0017] A reference coordinate system determination module, used to determine a preset reference coordinate system of the vehicle to be detected based on the three-dimensional point cloud data and the three-dimensional coordinates of the multiple key points;

[0018] The spatial posture determination module is used to determine the spatial posture of the vehicle to be detected according to the preset reference coordinate system.

[0019] In the third aspect, an embodiment of the present application also provides a computer device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor, and when the computer device is running, the processor communicates with the storage medium through the bus, and the processor executes the program instructions to perform the steps of the vehicle posture detection method as described in any one of the first aspects.

[0020] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the vehicle posture detection method as described in any one of the first aspects are executed.

[0021] The beneficial effects of this application are:

[0022] The vehicle posture detection method, device, computer equipment and storage medium provided in the present application are based on the two-dimensional image and three-dimensional point cloud data of the vehicle to be detected. By performing three-dimensional mapping of key points in the two-dimensional image, a preset reference coordinate system of the vehicle to be detected is determined according to the three-dimensional coordinates of the mapped key points and the three-dimensional point cloud data, so as to determine the spatial posture of the vehicle to be detected according to the preset reference coordinate system, thereby realizing fast and accurate vehicle posture detection, so as to accurately perform path planning and obstacle avoidance for automatic loading operations according to the vehicle posture, thereby ensuring the efficiency and safety of the operation process, realizing unlimited automatic loading operations and reducing labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present disclosure and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 An architecture diagram of a vehicle posture detection system provided in an embodiment of the present application;

[0025] Figure 2 Schematic diagram of the process of the vehicle posture detection method provided in the embodiment of the present application Figure 1 ;

[0026] Figure 3 Schematic diagram of the installation of multiple lidar combinations;

[0027] FIG4( a) is a schematic diagram of left-view key points provided in an embodiment of the present application;

[0028] FIG4( b ) is a schematic diagram of right viewing angle key points provided in an embodiment of the present application;

[0029] FIG4( c ) is a schematic diagram of key points of the rear viewing angle provided in an embodiment of the present application;

[0030] Figure 5 Schematic diagram of the process of the vehicle posture detection method provided in the embodiment of the present application Figure 2 ;

[0031] Figure 6 Schematic diagram of the process of the vehicle posture detection method provided in the embodiment of the present application Figure 3 ;

[0032] Figure 7 Schematic diagram 4 of the process flow of the vehicle posture detection method provided in the embodiment of the present application;

[0033] Figure 8 Schematic diagram of the process of the vehicle posture detection method provided in the embodiment of the present application Figure 5 ;

[0034] Fig. 9 A schematic diagram of point cloud data of the left baffle area provided in an embodiment of the present application;

[0035] Fig.10 Schematic diagram of the process of the vehicle posture detection method provided in the embodiment of the present application Figure 6 ;

[0036] Fig.11 A schematic diagram of boundary point cloud data provided in an embodiment of the present application;

[0037] Fig.12 Schematic diagram of the process of the vehicle posture detection method provided in the embodiment of the present application Figure 7 ;

[0038] Fig.13 A schematic diagram of a preset reference coordinate system provided in an embodiment of the present application;

[0039] Fig.14 A schematic diagram of the structure of a vehicle posture detection device provided in an embodiment of the present application;

[0040] Fig.15 A schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments.

[0042] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0043] In addition, the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0044] It should be noted that, in the absence of conflict, the features in the embodiments of the present application may be combined with each other.

[0045] Before describing the solution provided by the present application in detail, the application scenario of the present application is first introduced.

[0046] The construction machines targeted by the remote operation and automatic operation involved in this application are mainly excavators, and their operating objects are loading vehicles with a cargo box, i.e., trucks. Of course, it can also be applied to other construction machines and operating objects that require precise positioning of the operating objects before remote control or automatic operation can be performed, and this application does not impose any restrictions on this.

[0047] For tasks that require repeated operations such as excavation or dumping, even if they are remotely controlled, the operator is required to continuously perform complex operations in the simulated cockpit, which still places extremely high demands on the operator's operation. Loading, one of the routine operations of an excavator, is an operation that operators must concentrate highly in the actual operation process to complete. If they are not careful, the excavator is likely to collide with the loading vehicle, which can easily cause a safety accident.

[0048] In order to reduce the safety risks caused by the need for operators to concentrate during remote operation, release the labor of operators, and improve overall operation efficiency, the excavator can be controlled to perform automatic loading operations. Automatic loading operations only require operators to enter the trajectory path of the loading operation and the posture information of the loading vehicle in the remote interface to realize automatic loading operations. Therefore, it is very important to detect the posture information of the loading vehicle during automatic loading operations.

[0049] Existing solutions for acquiring object pose information are mainly divided into deep learning and point cloud registration. Among them, the solution based on deep learning to acquire object pose information needs to rely on a large amount of training data, and the model deployment needs to balance accuracy and inference speed. It also needs to occupy computer video memory resources. If the registration time is too long, the efficiency of automatic loading operations may be far from the standard of experienced operators. The solution based on point cloud registration to acquire object pose information is not robust to data noise and has a high dependence on initial values. In automatic loading operations, as the amount of material in the bucket increases, the impact on the accuracy of registration gradually increases. Poor registration accuracy may cause the excavator to collide with the loading vehicle during loading, and safety cannot be guaranteed.

[0050] In view of the defects of the above-mentioned prior art, the present application intends to provide a vehicle posture detection method, device, computer equipment and storage medium, which is based on the two-dimensional image and three-dimensional point cloud data of the vehicle to be detected, by performing three-dimensional mapping of the key points in the two-dimensional image, and determining the preset reference coordinate system of the vehicle to be detected according to the three-dimensional coordinates of the mapped key points and the three-dimensional point cloud data, so as to determine the spatial posture of the vehicle to be detected according to the preset reference coordinate system, realize fast and accurate vehicle posture detection, and ensure the efficiency and safety of the operation process.

[0051] The vehicle posture detection method provided in this application is applied to the vehicle posture detection system. Figure 1 , is an architecture diagram of a vehicle posture detection system provided in an embodiment of the present application, such as Figure 1 As shown, the vehicle posture detection system includes: a camera, a laser radar, a computer device and a motion control obstacle avoidance module, wherein the camera, the laser radar and the motion control obstacle avoidance module are respectively connected to the computer device for communication, the camera is used to collect a two-dimensional image of the vehicle to be detected at a preset viewing angle and send it to the computer device, the laser radar is used to collect three-dimensional point cloud data of the vehicle to be detected at a preset viewing angle and send it to the computer device, the computer device executes the vehicle posture detection method provided in the present application according to the two-dimensional image and the three-dimensional point cloud data, determines the spatial posture of the vehicle to be detected, and sends the spatial posture of the vehicle to be detected to the motion control obstacle avoidance module, and the motion control obstacle avoidance module performs path planning and obstacle avoidance for automatic loading operations according to the spatial posture of the vehicle to be detected.

[0052] Please refer to Figure 2 , which is a flow chart of the vehicle posture detection method provided in the embodiment of the present application Figure 1 ,like Figure 2 As shown, the method may include:

[0053] S10: Acquire a pre-collected two-dimensional image of the vehicle to be detected at a preset viewing angle, and three-dimensional point cloud data of the vehicle to be detected at the preset viewing angle.

[0054] In this embodiment, a camera and a laser radar are installed at the working position of the construction machine, i.e., an excavator. According to the relative position relationship between the vehicle to be detected and the construction machine, the camera can obtain a two-dimensional image of the vehicle to be detected at a preset viewing angle corresponding to the relative position relationship, and the laser radar can also obtain three-dimensional point cloud data of the vehicle to be detected at the preset viewing angle. The three-dimensional point cloud data of the vehicle to be detected at the preset viewing angle is obtained by clustering the laser point cloud data.

[0055] According to the relative position relationship between the vehicle to be detected and the construction machine, the preset viewing angles may include: left viewing angle, right viewing angle and rear viewing angle, wherein the division of the relative directions of front, rear, left and right is based on the driver's seat of the vehicle to be detected.

[0056] In some embodiments, due to the limitations of the laser radar's viewing angle and laser density, a single laser radar cannot obtain complete three-dimensional point cloud data at a preset viewing angle of the vehicle to be detected. Therefore, multiple laser radars are required to be installed in combination to ensure that the viewing angle and density of multiple laser radars can cover the entire vehicle to be detected at the preset viewing angle.

[0057] For examples, please refer to Figure 3 , which is a schematic diagram of the combined installation of multiple laser radars, such as Figure 3As shown in the figure, laser radars are installed on the left, middle and right to ensure that the horizontal field of view covered by the three laser radars reaches 180°. According to the relative positions of the three laser radars, the three-dimensional point cloud data obtained by the three laser radars are spliced ​​through the calibration algorithm to obtain the complete three-dimensional point cloud data of the vehicle to be detected at the preset viewing angle.

[0058] S20: Identify key points of the vehicle in the two-dimensional image to obtain two-dimensional coordinates of multiple key points in the vehicle to be detected.

[0059] In this embodiment, the vehicle key points are the key points corresponding to the main body of the vehicle to be detected at a preset viewing angle. Since the two-dimensional image is acquired by the camera at a preset viewing angle, multiple key points can be the key points of the vehicle to be detected on the side corresponding to the preset viewing angle. In addition, since a construction machine is performing loading operations on the vehicle to be detected, multiple key points can also be the key points of the bucket of the vehicle to be detected on the side corresponding to the preset viewing angle, where the key points can include vertices, boundary points, etc.

[0060] In some embodiments, the method of performing vehicle key point recognition on a two-dimensional image may be: performing key point recognition on the two-dimensional image using a pre-trained key point recognition model, wherein the key point recognition model is trained using multi-view vehicle training images pre-marked with key points. For example, the backbone network of the key point recognition model may be HRNet.

[0061] For example, please refer to FIG4(a), which is a schematic diagram of the left perspective key points provided in an embodiment of the present application, please refer to FIG4(b), which is a schematic diagram of the right perspective key points provided in an embodiment of the present application, and please refer to FIG4(c), which is a schematic diagram of the rear perspective key points provided in an embodiment of the present application. Figure 4(a)-Figure 4(c) As shown, the key points of the left perspective include: key points 1-15, key points 5-8 constitute the left side fender area of ​​the truck bed of the vehicle to be detected; the key points of the right perspective include: key points 16-30, key points 20-23 constitute the right side fender area of ​​the truck bed of the vehicle to be detected, and the key points of the rear perspective include: key points 6, 7, 21, 22, which constitute the rear side fender area of ​​the truck bed of the vehicle to be detected.

[0062] It should be noted that the following embodiments are described by taking the key points of the left perspective as an example, and the implementation methods of other perspectives are the same as that of the left perspective.

[0063] S30: Perform three-dimensional mapping on the two-dimensional coordinates of the multiple key points to obtain the three-dimensional coordinates of the multiple key points.

[0064] In this embodiment, three-dimensional mapping is used to determine the position coordinates of multiple key points of the vehicle to be detected in three-dimensional space. In some embodiments, the three-dimensional coordinates mapped by the two-dimensional coordinates of multiple key points can be determined through reprojection and clustering algorithms based on the intrinsic parameters of the camera and the external parameters of the camera and lidar.

[0065] Among them, the internal parameters of the camera may include at least: camera principal point, distortion parameters and focal length. The internal parameters of the camera can be obtained, for example, through a checkerboard calibration plate and Zhang Zhengyou calibration algorithm, which is not limited in this embodiment.

[0066] The external parameters of the camera and the laser radar may include: a rotation matrix and a translation matrix. The external parameters of the camera and the laser radar may be obtained, for example, through an April tag calibration board, which is not limited in this embodiment.

[0067] S40: Determine a preset reference coordinate system of the vehicle to be detected based on the three-dimensional point cloud data and the three-dimensional coordinates of multiple key points.

[0068] In this embodiment, the target area is the area formed by the target key points among multiple key points in the two-dimensional image. The target area can be any area of ​​the vehicle to be detected. In the loading operation, the target area at least includes the truck bed baffle area of ​​the vehicle to be detected, such as the left baffle area formed by key points 5-8 in Figure 4 (a), the right baffle area formed by key points 20-23, and the rear view key points include: key points 6, 7, 21, 22, forming the rear side baffle area. The target part is the part of the vehicle to be detected in the three-dimensional space corresponding to the target area, generally the truck bed of the vehicle to be detected.

[0069] According to the three-dimensional point cloud data of the preset perspective and the three-dimensional coordinates of multiple key points, a reference coordinate system under the preset perspective is constructed based on the position of the target part in the vehicle to be detected, wherein the reference coordinate system is composed of a horizontal coordinate axis (x-axis), a longitudinal coordinate axis (y-axis), a vertical coordinate axis (z-axis) and a coordinate origin (point o).

[0070] S50: Determine the spatial position and posture of the vehicle to be detected according to a preset reference coordinate system.

[0071] In this embodiment, the center position of the vehicle to be detected and the offset angles of the vehicle to be detected in different directions are determined according to a unique preset reference coordinate system composed of a coordinate origin and three coordinate axes to determine the spatial position and posture of the vehicle to be detected.

[0072] Furthermore, after determining the spatial posture of the vehicle to be detected, during the automatic loading operation, the spatial posture of the vehicle to be detected is transmitted to the motion control obstacle avoidance module of the remote control system, so that the motion control obstacle avoidance module can perform path planning and obstacle avoidance for the automatic loading operation according to the spatial posture of the vehicle to be detected.

[0073] The vehicle posture detection method provided in the above embodiment is based on the two-dimensional image and three-dimensional point cloud data of the vehicle to be detected. By performing three-dimensional mapping on the key points in the two-dimensional image, the preset reference coordinate system of the vehicle to be detected is determined according to the three-dimensional coordinates of the mapped key points and the three-dimensional point cloud data, so as to determine the spatial posture of the vehicle to be detected according to the preset reference coordinate system, thereby realizing fast and accurate vehicle posture detection, so as to accurately perform path planning and obstacle avoidance for automatic loading operations according to the vehicle posture, thereby ensuring the efficiency and safety of the operation process, realizing unlimited automatic loading operations, and reducing labor costs.

[0074] A possible implementation method of determining the preset reference coordinate system is described below in conjunction with an embodiment.

[0075] Please refer to Figure 5 , which is a flow chart of the vehicle posture detection method provided in the embodiment of the present application Figure 2 ,like Figure 5 As shown, the above S40 determines the preset reference coordinate system of the vehicle to be detected based on the three-dimensional point cloud data and the three-dimensional coordinates of multiple key points, which may include:

[0076] S41: Determine target three-dimensional point cloud data corresponding to a target area in the vehicle to be detected from the three-dimensional point cloud data.

[0077] In this embodiment, in order to ensure that the reference coordinate system established for the vehicle to be detected is more accurate, the reference coordinate system is not established using all the three-dimensional point cloud data of the vehicle to be detected, and only the target three-dimensional point cloud data corresponding to the target area is screened out from all the three-dimensional point cloud data.

[0078] In some embodiments, target three-dimensional point cloud data corresponding to the target area may be determined from the three-dimensional point cloud data based on location information, size information, etc. of the target area.

[0079] In a possible implementation, the above S41 determines the target three-dimensional point cloud data corresponding to the target area in the to-be-detected vehicle from the three-dimensional point cloud data, which may include:

[0080] According to the two-dimensional coordinates of multiple key points and the corresponding relationship between the preset two-dimensional image and the three-dimensional point cloud data, the target three-dimensional point cloud data is determined from the three-dimensional point cloud data.

[0081] In this embodiment, the correspondence between the two-dimensional image and the three-dimensional point cloud data can be determined based on the intrinsic parameters of the camera and the external parameters of the camera and the lidar. According to the correspondence between the two-dimensional image and the three-dimensional point cloud data, the two-dimensional projection points corresponding to the three-dimensional point cloud data are determined. According to the two-dimensional coordinates of multiple key points, the three-dimensional point cloud data whose two-dimensional projection points are located in the target area is determined as the target three-dimensional point cloud data.

[0082] S42: Calculate the three-dimensional direction vector of the vehicle to be detected based on the target three-dimensional point cloud data.

[0083] In this embodiment, the target three-dimensional point cloud data is fitted in three dimensions respectively, and the direction vectors of the three dimensions are determined according to the fitting results. The three-dimensional point cloud data in each dimensional direction can be determined from the target three-dimensional point cloud data, and the direction vector of each dimension is determined by fitting the three-dimensional point cloud data in each dimensional direction.

[0084] S43: Determine the coordinate origin according to the three-dimensional coordinates and three-dimensional direction vectors of the multiple key points.

[0085] In this embodiment, the three-dimensional coordinates of the vehicle to be detected can be determined according to the three-dimensional coordinates of multiple key points and the three-dimensional direction vector, and the shape structure of the target part can be determined according to the three-dimensional coordinates of the vehicle to be detected, and the coordinate origin can be determined according to the center point of the shape structure of the target part. The shape structure of the target part is a rectangle.

[0086] S44: Determine a preset reference coordinate system according to the coordinate origin and the three-dimensional direction vector.

[0087] In this embodiment, a Cartesian coordinate system intersecting at the coordinate origin is constructed according to the three-dimensional direction vectors as a reference coordinate system for the vehicle to be detected, wherein the coordinate axes corresponding to the three-dimensional direction vectors intersect each other and are perpendicular to the coordinate origin.

[0088] The vehicle posture detection method provided in the above embodiment calculates the three-dimensional direction vector through the target three-dimensional point cloud data corresponding to the target area, and determines the coordinate origin according to the three-dimensional coordinates and three-dimensional direction vectors of multiple key points, so as to determine the reference coordinate system according to the coordinate origin and the three-dimensional direction vector, thereby ensuring the construction of an accurate coordinate system, thereby realizing accurate detection of the vehicle posture.

[0089] A possible implementation method of determining the three-dimensional direction vector is described below in conjunction with an embodiment.

[0090] Please refer to Figure 6 , which is a flow chart of the vehicle posture detection method provided in the embodiment of the present application Figure 3 ,like Figure 6 As shown, the process of calculating the three-dimensional direction vector of the vehicle to be detected according to the target three-dimensional point cloud data in S42 may include:

[0091] S51: Determine a normal vector of the target area and a boundary direction vector of the target area according to the target three-dimensional point cloud data.

[0092] In this embodiment, the normal vector of the target area is a direction vector perpendicular to the target area, and the plane equation of the target area can be calculated according to the target three-dimensional point cloud data to determine the normal vector perpendicular to the plane according to the plane equation.

[0093] The boundary direction vector of the target area can be a direction vector corresponding to the boundary of the target area, wherein the upper boundary and the lower boundary of the target area are the direction vectors in the horizontal direction, and the left boundary and the right boundary of the target area are the direction vectors in the vertical direction. The boundary direction vector can be determined based on the boundary point cloud data in the target three-dimensional point cloud data.

[0094] In some embodiments, a boundary direction vector can be selected from the direction vectors of the upper boundary and the lower boundary, and another boundary direction vector can be selected from the direction vectors of the left boundary and the right boundary; or the direction vectors of the upper boundary and the lower boundary can be averaged to obtain a boundary direction vector, and the direction vectors of the left boundary and the right boundary can be averaged to obtain another boundary direction vector.

[0095] S52: Determine a three-dimensional direction vector according to the normal vector of the target area and the boundary direction vector of the target area.

[0096] In this embodiment, a normal vector of the target area and two boundary direction vectors of the target area are used as the horizontal, longitudinal and vertical direction vectors respectively to obtain a three-dimensional direction vector.

[0097] In one possible implementation, see Figure 7 , which is a flow chart of the vehicle posture detection method provided in the embodiment of the present application, as shown in FIG. Figure 7 As shown, the above S52 determines the three-dimensional direction vector according to the normal vector of the target area and the boundary direction vector of the target area, which may include:

[0098] S521: Determine two direction vectors respectively according to the normal vector of the target area and the boundary direction vector of the target area.

[0099] S522: Determine another direction vector according to the two direction vectors and the constraint relationship between the direction vectors.

[0100] In this embodiment, since the three direction vectors are perpendicular to each other, in order to reduce the amount of calculation, there is no need to calculate the two boundary direction vectors of the target area. Only the normal vector and one boundary direction vector of the target area need to be calculated, and the other direction vector can be determined based on the constraint relationship of being perpendicular to each other.

[0101] Among them, if the boundary direction vector of the target area is the direction vector of the boundary of the target area in the horizontal direction (upper boundary or lower boundary), and the normal vector of the target area is perpendicular to the target area, then the boundary direction vector of the target area is used as the horizontal axis direction vector (x-axis), and the normal vector of the target area is used as the vertical axis direction vector (y-axis). According to the horizontal axis direction vector and the vertical axis direction vector, the vertical axis direction vector (z-axis) is determined.

[0102] The vehicle posture detection method provided in the above embodiment determines the normal vector and boundary direction vector of the target area based on the three-dimensional point cloud data of the target area to obtain a three-dimensional direction vector, thereby ensuring that an accurate coordinate system can be constructed, thereby achieving accurate detection of the vehicle posture.

[0103] A possible implementation method of determining the normal vector and the boundary direction vector is described below in conjunction with an embodiment.

[0104] Please refer to Figure 8 , which is a flow chart of the vehicle posture detection method provided in the embodiment of the present application Figure 5 ,like Figure 8 As shown, the process of determining the normal vector of the target area and the boundary direction vector of the target area according to the target three-dimensional point cloud data in S51 may include:

[0105] S511: Perform plane fitting on the target area according to the target three-dimensional point cloud data to obtain a normal vector of the target area.

[0106] In this embodiment, a preset plane fitting algorithm is used to fit the target three-dimensional point cloud data to obtain a target area plane composed of the target three-dimensional point cloud data, and the target area plane is represented by a plane model equation Ax+By+Cz+D=0, where (A, B, C) is the normal vector of the target area. The plane fitting algorithm can be a RANSAC plane fitting algorithm.

[0107] For examples, please refer to Fig. 9 , is a schematic diagram of point cloud data of the left baffle area provided in an embodiment of the present application, such as Fig. 9 As shown, the left baffle area is formed according to key points 5-8, and the target three-dimensional point cloud data of the left baffle area is determined from the three-dimensional point cloud data. By performing plane fitting on the target three-dimensional point cloud data of the left baffle area, the normal vector of the left baffle area can be obtained.

[0108] S512: Perform boundary fitting on the target area according to the target three-dimensional point cloud data to obtain a boundary direction vector of the target area.

[0109] In this embodiment, a preset straight line fitting algorithm is used to fit the target three-dimensional point cloud data to obtain any boundary corresponding to the target area, and the boundary equation (x-x1) / E=(y-y1) / F=(z-z1) / G is used to represent any boundary of the target area, wherein (x1, y1, z1) is a three-dimensional point cloud data of any boundary, (E, F, G) is the boundary direction vector, and the straight line fitting algorithm can be, for example, the RANSAC straight line fitting algorithm.

[0110] In one possible implementation, see Fig.10 , which is a flow chart of the vehicle posture detection method provided in the embodiment of the present application Figure 6 ,like Fig.10 As shown, the process of performing boundary fitting on the target area according to the target three-dimensional point cloud data in S512 to obtain the boundary direction vector of the target area may include:

[0111] S61: Extracting boundaries based on the target three-dimensional point cloud data to determine boundary point cloud data corresponding to the target area.

[0112] In this embodiment, a boundary extraction algorithm is used to extract the boundary of the target three-dimensional point cloud data, and the boundary point cloud data. Before the boundary extraction is performed, a preset filtering algorithm can also be used to filter the target three-dimensional point cloud data to filter the noise point cloud data in the target three-dimensional point cloud data to improve the accuracy of boundary extraction. The filtering algorithm can be, for example, a voxel filtering downsampling algorithm.

[0113] For examples, please refer to Fig.11 , is a schematic diagram of boundary point cloud data provided in an embodiment of the present application, such as Fig.11 As shown, by extracting the boundary of the target three-dimensional point cloud data, the boundary point cloud data constituting the target area is obtained.

[0114] S62: Perform straight line fitting according to the boundary point cloud data to obtain the boundary direction vector of the target area.

[0115] In this embodiment, according to the boundary direction vector to be obtained, a straight line fitting is performed on the point cloud data corresponding to any boundary in the boundary point cloud data to obtain the boundary direction vector corresponding to the boundary. If the boundary direction vector in the vertical direction needs to be obtained, a straight line fitting is performed on the point cloud data corresponding to the left boundary or the right boundary; if the boundary direction vector in the horizontal direction needs to be obtained, a straight line fitting is performed on the point cloud data corresponding to the upper boundary or the lower boundary.

[0116] The vehicle posture detection method provided in the above embodiment obtains the normal vector and boundary direction vector of the target area by plane fitting and straight line fitting of the target three-dimensional point cloud data, so as to determine the three-dimensional direction vector based on the normal vector and the boundary direction vector, thereby ensuring that an accurate coordinate system can be constructed, thereby realizing accurate detection of the vehicle posture.

[0117] A possible implementation method of determining the coordinate origin is described below in conjunction with an embodiment.

[0118] Please refer to Fig.12 , which is a flow chart of the vehicle posture detection method provided in the embodiment of the present application Figure 7 ,like Fig.12 As shown, the above S43 determines the coordinate origin according to the three-dimensional coordinates and three-dimensional direction vectors of the multiple key points, which may include:

[0119] S431: Determine the three-dimensional coordinates of parallel key points corresponding to the multiple key points according to the three-dimensional coordinates of the multiple key points, the three-dimensional direction vectors and the vehicle data of the vehicle to be detected.

[0120] In this embodiment, the target key points corresponding to the target part can be determined from multiple key points. The parallel key points are key points that constitute the target parts and are parallel to the target key points in a preset direction. The vehicle data of the vehicle to be detected is the size data of the vehicle to be detected, more specifically, it can be the size data of the vehicle to be detected in a preset direction.

[0121] The three-dimensional coordinates of a parallel key point parallel to the target key point are determined in the preset direction according to the three-dimensional coordinates of the target key point, the direction vector of the preset direction, and the size data in the preset direction.

[0122] S432: Calculate the coordinate origin according to the three-dimensional coordinates of the multiple key points and the three-dimensional coordinates of the parallel key points.

[0123] In this embodiment, according to the three-dimensional coordinates of the target key point among multiple key points and the three-dimensional coordinates of the parallel key points corresponding to the target key point, a rectangular area formed by the target key point and the parallel key points is determined, and the center point of the rectangular area is determined as the coordinate origin.

[0124] For examples, please refer to Fig.13 , is a schematic diagram of a preset reference coordinate system provided in an embodiment of the present application, such as Fig.13As shown, key points P1 and P4 are the key points corresponding to key points 5 and 6 in the three-dimensional space. According to the three-dimensional coordinates of key points P1 and P2, as well as the longitudinal direction vector and the length data of the vehicle to be detected in the longitudinal direction, the three-dimensional coordinates corresponding to the parallel key points P2 and P3 are determined. According to the rectangular car formed by key points P1, P2, P3 and P4, the midpoint of the car is determined as the coordinate origin. According to the three-dimensional direction vector, a reference coordinate system of the vehicle to be detected is established at the coordinate origin to determine the spatial posture of the vehicle to be detected.

[0125] The vehicle posture detection method provided in the above embodiment determines the three-dimensional coordinates of parallel key points based on the three-dimensional coordinates of the key points, the three-dimensional direction vector and the vehicle data of the vehicle to be detected, so as to determine the coordinate origin based on the three-dimensional coordinates of the key points and the three-dimensional coordinates of the parallel key points, thereby ensuring that an accurate coordinate system can be constructed, thereby realizing accurate detection of the vehicle posture.

[0126] Based on the above method embodiment, the present application embodiment also provides a vehicle posture detection device. Fig.14 , is a schematic diagram of the structure of a vehicle posture detection device provided in an embodiment of the present application, such as Fig.14 As shown, the device may include:

[0127] The vehicle data acquisition module 10 is used to acquire a two-dimensional image of the vehicle to be detected at a preset viewing angle acquired by a camera, and three-dimensional point cloud data of the vehicle to be detected at a preset viewing angle acquired by a laser radar;

[0128] The key point recognition module 20 is used to recognize the key points of the vehicle in the two-dimensional image and obtain the two-dimensional coordinates of multiple key points in the vehicle to be detected;

[0129] A three-dimensional mapping module 30 is used to perform three-dimensional mapping on the two-dimensional coordinates of multiple key points to obtain the three-dimensional coordinates of the multiple key points;

[0130] A reference coordinate system determination module 40 is used to determine a preset reference coordinate system of the vehicle to be detected based on the three-dimensional point cloud data and the three-dimensional coordinates of a plurality of key points;

[0131] The spatial posture determination module 50 is used to determine the spatial posture of the vehicle to be detected according to a preset reference coordinate system.

[0132] The vehicle posture detection device provided in the above embodiment is based on the two-dimensional image and three-dimensional point cloud data of the vehicle to be detected. By performing three-dimensional mapping of key points in the two-dimensional image, a preset reference coordinate system of the vehicle to be detected is determined according to the three-dimensional coordinates of the mapped key points and the three-dimensional point cloud data, so as to determine the spatial posture of the vehicle to be detected according to the preset reference coordinate system, thereby realizing fast and accurate vehicle posture detection, so as to accurately perform path planning and obstacle avoidance for automatic loading operations according to the vehicle posture, thereby ensuring the efficiency and safety of the operation process, realizing unlimited automatic loading operations, and reducing labor costs.

[0133] Optionally, the reference coordinate system determination module 40 may include:

[0134] A target point cloud determination unit is used to determine target three-dimensional point cloud data corresponding to a target area in a vehicle to be detected from the three-dimensional point cloud data;

[0135] A direction vector calculation unit, used to calculate the three-dimensional direction vector of the vehicle to be detected according to the target three-dimensional point cloud data;

[0136] A coordinate origin determination unit, used to determine the coordinate origin according to the three-dimensional coordinates and three-dimensional direction vectors of multiple key points;

[0137] The reference coordinate system determining unit is used to determine a preset reference coordinate system according to a coordinate origin and a three-dimensional direction vector.

[0138] In a possible implementation, the target point cloud determination unit is specifically used to determine target three-dimensional point cloud data from three-dimensional point cloud data according to the two-dimensional coordinates of multiple key points and the correspondence between the preset two-dimensional image and the three-dimensional point cloud data.

[0139] The vehicle posture detection device provided in the above embodiment calculates the three-dimensional direction vector through the target three-dimensional point cloud data corresponding to the target area, and determines the coordinate origin according to the three-dimensional coordinates and three-dimensional direction vectors of multiple key points, so as to determine the reference coordinate system of the vehicle to be detected according to the coordinate origin and the three-dimensional direction vector, thereby ensuring the construction of an accurate coordinate system, thereby realizing accurate detection of the vehicle posture.

[0140] Optionally, the direction vector calculation unit is specifically used to determine the normal vector of the target area and the boundary direction vector of the target area according to the target three-dimensional point cloud data; determine the three-dimensional direction vector according to the normal vector of the target area and the boundary direction vector of the target area.

[0141] In a possible implementation, the direction vector calculation unit is specifically used to determine two direction vectors according to the normal vector of the target area and the boundary direction vector of the target area; and determine another direction vector according to the two direction vectors and a constraint relationship between the direction vectors.

[0142] The vehicle posture detection device provided in the above embodiment determines the normal vector and boundary direction vector of the target area based on the three-dimensional point cloud data of the target area to obtain a three-dimensional direction vector, thereby ensuring that an accurate coordinate system can be constructed, thereby achieving accurate detection of the vehicle posture.

[0143] Optionally, the direction vector calculation unit is specifically used to perform plane fitting on the target area according to the target three-dimensional point cloud data to obtain the normal vector of the target area; perform boundary fitting on the target area according to the target three-dimensional point cloud data to obtain the boundary direction vector of the target area.

[0144] Optionally, a direction vector calculation unit is specifically used to extract the boundary according to the target three-dimensional point cloud data to determine the boundary point cloud data corresponding to the target area; and perform straight line fitting according to the boundary point cloud data to obtain the boundary direction vector of the target area.

[0145] The vehicle posture detection device provided in the above embodiment obtains the normal vector and boundary direction vector of the target area by performing plane fitting and straight line fitting on the target three-dimensional point cloud data, so as to determine the three-dimensional direction vector based on the normal vector and the boundary direction vector, thereby ensuring that an accurate coordinate system can be constructed, thereby realizing accurate detection of the vehicle posture.

[0146] Optionally, a coordinate origin determination unit is specifically used to determine the three-dimensional coordinates of parallel key points corresponding to multiple key points based on the three-dimensional coordinates of the multiple key points, the three-dimensional direction vectors and the vehicle data of the vehicle to be detected; and calculate the coordinate origin based on the three-dimensional coordinates of the multiple key points and the three-dimensional coordinates of the parallel key points.

[0147] The vehicle posture detection device provided in the above embodiment determines the three-dimensional coordinates of parallel key points based on the three-dimensional coordinates of the key points, the three-dimensional direction vector and the vehicle data of the vehicle to be detected, so as to determine the coordinate origin based on the three-dimensional coordinates of the key points and the three-dimensional coordinates of the parallel key points, thereby ensuring that an accurate coordinate system can be constructed, thereby realizing accurate detection of the vehicle posture.

[0148] The above-mentioned device is used to execute the method provided by the aforementioned embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0149] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), or one or more microprocessors, or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0150] Please refer to Fig.15 , is a schematic diagram of a computer device provided in an embodiment of the present application, such as Fig.15 As shown, the computer device 100 includes: a processor 101, a storage medium 102 and a bus. The storage medium 102 stores program instructions executable by the processor 101. When the computer device 100 is running, the processor 101 and the storage medium 102 communicate through the bus, and the processor 101 executes the program instructions to execute the above-mentioned vehicle posture detection method.

[0151] Specifically, the processor executes the steps of the above vehicle posture detection method including:

[0152] Acquire a two-dimensional image of the vehicle to be detected at a preset viewing angle captured by a camera, and three-dimensional point cloud data of the vehicle to be detected at a preset viewing angle captured by a lidar; identify key points of the vehicle in the two-dimensional image to obtain two-dimensional coordinates of multiple key points in the vehicle to be detected; perform three-dimensional mapping on the two-dimensional coordinates of multiple key points to obtain three-dimensional coordinates of multiple key points; determine a preset reference coordinate system of the vehicle to be detected based on the three-dimensional point cloud data and the three-dimensional coordinates of multiple key points; determine the spatial posture of the vehicle to be detected based on the preset reference coordinate system.

[0153] The vehicle posture detection method executed by the processor of the above-mentioned embodiment is based on the two-dimensional image and three-dimensional point cloud data of the vehicle to be detected. By performing three-dimensional mapping on the key points in the two-dimensional image, the preset reference coordinate system of the vehicle to be detected is determined according to the three-dimensional coordinates of the mapped key points and the three-dimensional point cloud data, so as to determine the spatial posture of the vehicle to be detected according to the preset reference coordinate system, thereby realizing fast and accurate vehicle posture detection, so as to accurately perform path planning and obstacle avoidance for automatic loading operations according to the vehicle posture, thereby ensuring the efficiency and safety of the operation process, realizing unlimited automatic loading operations, and reducing labor costs.

[0154] In a possible implementation, the processor performs the above-mentioned determination of the preset reference coordinate system of the vehicle to be detected based on the three-dimensional point cloud data and the three-dimensional coordinates of the multiple key points, which may include:

[0155] Determine the target three-dimensional point cloud data corresponding to the target area in the vehicle to be detected from the three-dimensional point cloud data; calculate the three-dimensional direction vector of the vehicle to be detected based on the target three-dimensional point cloud data; determine the coordinate origin based on the three-dimensional coordinates and three-dimensional direction vectors of multiple key points; determine the preset reference coordinate system based on the coordinate origin and the three-dimensional direction vector.

[0156] In some embodiments, the processor performs the above-mentioned determination of target three-dimensional point cloud data corresponding to the target area in the vehicle to be detected from the three-dimensional point cloud data, which may include:

[0157] According to the two-dimensional coordinates of multiple key points and the corresponding relationship between the preset two-dimensional image and the three-dimensional point cloud data, the target three-dimensional point cloud data is determined from the three-dimensional point cloud data.

[0158] The vehicle posture detection method executed by the processor of the above embodiment calculates the three-dimensional direction vector through the target three-dimensional point cloud data corresponding to the target area, and determines the coordinate origin according to the three-dimensional coordinates and three-dimensional direction vectors of multiple key points, so as to determine the reference coordinate system of the vehicle to be detected according to the coordinate origin and the three-dimensional direction vector, thereby ensuring the construction of an accurate coordinate system, thereby realizing accurate detection of the vehicle posture.

[0159] In a possible implementation, the processor performs the above process of calculating the three-dimensional direction vector of the vehicle to be detected according to the target three-dimensional point cloud data, which may include:

[0160] According to the target three-dimensional point cloud data, the normal vector of the target area and the boundary direction vector of the target area are determined; according to the normal vector of the target area and the boundary direction vector of the target area, the three-dimensional direction vector is determined.

[0161] In some embodiments, the processor performs the above-mentioned determination of the three-dimensional direction vector according to the normal vector of the target area and the boundary direction vector of the target area, which may include:

[0162] Two direction vectors are determined according to the normal vector of the target area and the boundary direction vector of the target area; another direction vector is determined according to the two direction vectors and the constraint relationship between the direction vectors.

[0163] The vehicle posture detection method executed by the processor of the above embodiment determines the normal vector and boundary direction vector of the target area based on the three-dimensional point cloud data of the target area to obtain a three-dimensional direction vector, thereby ensuring that an accurate coordinate system can be constructed, thereby achieving accurate detection of the vehicle posture.

[0164] In a possible implementation, the processor performs the above process of determining the normal vector of the target area and the boundary direction vector of the target area according to the target three-dimensional point cloud data, which may include:

[0165] According to the target three-dimensional point cloud data, the target area is plane-fitted to obtain the normal vector of the target area; according to the target three-dimensional point cloud data, the target area is boundary-fitted to obtain the boundary direction vector of the target area.

[0166] In some embodiments, the processor performs the above process of performing boundary fitting on the target area according to the target three-dimensional point cloud data to obtain the boundary direction vector of the target area, which may include:

[0167] Boundary extraction is performed based on the target three-dimensional point cloud data to determine the boundary point cloud data corresponding to the target area; straight line fitting is performed based on the boundary point cloud data to obtain the boundary direction vector of the target area.

[0168] The vehicle posture detection method executed by the processor of the above-mentioned embodiment obtains the normal vector and boundary direction vector of the target area by plane fitting and straight line fitting of the target three-dimensional point cloud data, so as to determine the three-dimensional direction vector based on the normal vector and the boundary direction vector, thereby ensuring that an accurate coordinate system can be constructed, thereby realizing accurate detection of the vehicle posture.

[0169] In a possible implementation, the processor performs the above-mentioned determination of the coordinate origin according to the three-dimensional coordinates and three-dimensional direction vectors of the multiple key points, which may include:

[0170] According to the three-dimensional coordinates of the multiple key points, the three-dimensional direction vectors and the vehicle data of the vehicle to be detected, the three-dimensional coordinates of the parallel key points corresponding to the multiple key points are determined; according to the three-dimensional coordinates of the multiple key points and the three-dimensional coordinates of the parallel key points, the coordinate origin is calculated.

[0171] The vehicle posture detection method executed by the processor of the above embodiment determines the three-dimensional coordinates of parallel key points based on the three-dimensional coordinates of the key points, the three-dimensional direction vectors and the vehicle data of the vehicle to be detected, so as to determine the coordinate origin based on the three-dimensional coordinates of the key points and the three-dimensional coordinates of the parallel key points, thereby ensuring that an accurate coordinate system can be constructed, thereby realizing accurate detection of the vehicle posture.

[0172] Optionally, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned vehicle posture detection method are executed.

[0173] Specifically, the steps of executing the above vehicle posture detection method by the computer program include:

[0174] Acquire a two-dimensional image of the vehicle to be detected at a preset viewing angle captured by a camera, and three-dimensional point cloud data of the vehicle to be detected at a preset viewing angle captured by a lidar; identify key points of the vehicle in the two-dimensional image to obtain two-dimensional coordinates of multiple key points in the vehicle to be detected; perform three-dimensional mapping on the two-dimensional coordinates of multiple key points to obtain three-dimensional coordinates of multiple key points; determine a preset reference coordinate system of the vehicle to be detected based on the three-dimensional point cloud data and the three-dimensional coordinates of multiple key points; determine the spatial posture of the vehicle to be detected based on the preset reference coordinate system.

[0175] The vehicle posture detection method executed by the computer program of the above-mentioned embodiment is based on the two-dimensional image and three-dimensional point cloud data of the vehicle to be detected. By performing three-dimensional mapping on the key points in the two-dimensional image, the preset reference coordinate system of the vehicle to be detected is determined according to the three-dimensional coordinates of the mapped key points and the three-dimensional point cloud data, so as to determine the spatial posture of the vehicle to be detected according to the preset reference coordinate system, thereby realizing fast and accurate vehicle posture detection, so as to accurately perform path planning and obstacle avoidance for automatic loading operations according to the vehicle posture, thereby ensuring the efficiency and safety of the operation process, realizing unlimited automatic loading operations, and reducing labor costs.

[0176] In a possible implementation, the computer program performs the above-mentioned determination of the preset reference coordinate system of the vehicle to be detected based on the three-dimensional point cloud data and the three-dimensional coordinates of the multiple key points, which may include:

[0177] Determine the target three-dimensional point cloud data corresponding to the target area in the vehicle to be detected from the three-dimensional point cloud data; calculate the three-dimensional direction vector of the vehicle to be detected based on the target three-dimensional point cloud data; determine the coordinate origin based on the three-dimensional coordinates and three-dimensional direction vectors of multiple key points; determine the preset reference coordinate system based on the coordinate origin and the three-dimensional direction vector.

[0178] In some embodiments, the computer program executing the above-mentioned determination of target three-dimensional point cloud data corresponding to the target area in the vehicle to be detected from the three-dimensional point cloud data may include:

[0179] According to the two-dimensional coordinates of multiple key points and the corresponding relationship between the preset two-dimensional image and the three-dimensional point cloud data, the target three-dimensional point cloud data is determined from the three-dimensional point cloud data.

[0180] The vehicle posture detection method executed by the computer program of the above embodiment calculates the three-dimensional direction vector through the target three-dimensional point cloud data corresponding to the target area, and determines the coordinate origin according to the three-dimensional coordinates and three-dimensional direction vectors of multiple key points, so as to determine the reference coordinate system of the vehicle to be detected according to the coordinate origin and the three-dimensional direction vector, thereby ensuring the construction of an accurate coordinate system, thereby realizing accurate detection of the vehicle posture.

[0181] In a possible implementation, the computer program performs the above process of calculating the three-dimensional direction vector of the vehicle to be detected based on the target three-dimensional point cloud data, which may include:

[0182] According to the target three-dimensional point cloud data, the normal vector of the target area and the boundary direction vector of the target area are determined; according to the normal vector of the target area and the boundary direction vector of the target area, the three-dimensional direction vector is determined.

[0183] In some embodiments, the computer program executing the above-mentioned determination of the three-dimensional direction vector according to the normal vector of the target area and the boundary direction vector of the target area may include:

[0184] Two direction vectors are determined according to the normal vector of the target area and the boundary direction vector of the target area; another direction vector is determined according to the two direction vectors and the constraint relationship between the direction vectors.

[0185] The vehicle posture detection method executed by the computer program of the above embodiment determines the normal vector and boundary direction vector of the target area according to the three-dimensional point cloud data of the target area to obtain the three-dimensional direction vector, thereby ensuring that an accurate coordinate system can be constructed, thereby realizing accurate detection of the vehicle posture.

[0186] In a possible implementation, the computer program performs the above process of determining the normal vector of the target area and the boundary direction vector of the target area according to the target three-dimensional point cloud data, which may include:

[0187] According to the target three-dimensional point cloud data, the target area is plane-fitted to obtain the normal vector of the target area; according to the target three-dimensional point cloud data, the target area is boundary-fitted to obtain the boundary direction vector of the target area.

[0188] In some embodiments, the computer program performs the above process of performing boundary fitting on the target area according to the target three-dimensional point cloud data to obtain the boundary direction vector of the target area, which may include:

[0189] Boundary extraction is performed based on the target three-dimensional point cloud data to determine the boundary point cloud data corresponding to the target area; straight line fitting is performed based on the boundary point cloud data to obtain the boundary direction vector of the target area.

[0190] The vehicle posture detection method executed by the computer program of the above embodiment obtains the normal vector and boundary direction vector of the target area by plane fitting and straight line fitting of the target three-dimensional point cloud data, so as to determine the three-dimensional direction vector based on the normal vector and the boundary direction vector, thereby ensuring that an accurate coordinate system can be constructed, thereby realizing accurate detection of the vehicle posture.

[0191] In a possible implementation, the computer program performs the above-mentioned determination of the coordinate origin according to the three-dimensional coordinates and three-dimensional direction vectors of the plurality of key points, which may include:

[0192] According to the three-dimensional coordinates of the multiple key points, the three-dimensional direction vectors and the vehicle data of the vehicle to be detected, the three-dimensional coordinates of the parallel key points corresponding to the multiple key points are determined; according to the three-dimensional coordinates of the multiple key points and the three-dimensional coordinates of the parallel key points, the coordinate origin is calculated.

[0193] The vehicle posture detection method executed by the computer program of the above embodiment determines the three-dimensional coordinates of parallel key points based on the three-dimensional coordinates of the key points, the three-dimensional direction vectors and the vehicle data of the vehicle to be detected, so as to determine the coordinate origin based on the three-dimensional coordinates of the key points and the three-dimensional coordinates of the parallel key points, thereby ensuring that an accurate coordinate system can be constructed, thereby realizing accurate detection of the vehicle posture.

[0194] In the several embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0195] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0196] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0197] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, abbreviated: ROM), random access memory (English: Random Access Memory, abbreviated: RAM), disk or optical disk and other media that can store program codes.

[0198] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A vehicle posture detection method, characterized in that: The method comprises: Acquire a pre-collected two-dimensional image of the vehicle to be detected at a preset viewing angle, and three-dimensional point cloud data of the vehicle to be detected at the preset viewing angle; Identify key points of the vehicle on the two-dimensional image to obtain two-dimensional coordinates of multiple key points in the vehicle to be detected; Performing three-dimensional mapping on the two-dimensional coordinates of the multiple key points to obtain the three-dimensional coordinates of the multiple key points; Determining a preset reference coordinate system of the vehicle to be detected according to the three-dimensional point cloud data and the three-dimensional coordinates of the plurality of key points; Determining the spatial position of the vehicle to be detected according to the preset reference coordinate system; Determining a preset reference coordinate system of the vehicle to be detected based on the three-dimensional point cloud data and the three-dimensional coordinates of the plurality of key points includes: Determine target three-dimensional point cloud data corresponding to the target area in the vehicle to be detected from the three-dimensional point cloud data; Calculating the three-dimensional direction vector of the vehicle to be detected according to the target three-dimensional point cloud data; Determine a coordinate origin according to the three-dimensional coordinates of the plurality of key points and the three-dimensional direction vector; The preset reference coordinate system is determined according to the coordinate origin and the three-dimensional direction vector.

2. The method according to claim 1, characterized in that The step of determining target three-dimensional point cloud data corresponding to the target area in the to-be-detected vehicle from the three-dimensional point cloud data comprises: The target three-dimensional point cloud data is determined from the three-dimensional point cloud data according to the two-dimensional coordinates of the multiple key points and a corresponding relationship between a preset two-dimensional image and the three-dimensional point cloud data.

3. The method according to claim 1, characterized in that The step of calculating the three-dimensional direction vector of the vehicle to be detected according to the target three-dimensional point cloud data includes: Determining a normal vector of the target area and a boundary direction vector of the target area according to the target three-dimensional point cloud data; The three-dimensional direction vector is determined according to the normal vector of the target area and the boundary direction vector of the target area.

4. The method according to claim 3, characterized in that The step of determining the normal vector of the target area and the boundary direction vector of the target area according to the target three-dimensional point cloud data includes: Performing plane fitting on the target area according to the target three-dimensional point cloud data to obtain a normal vector of the target area; According to the target three-dimensional point cloud data, the boundary of the target area is fitted to obtain the boundary direction vector of the target area.

5. The method according to claim 3, characterized in that The determining the three-dimensional direction vector according to the normal vector of the target area and the boundary direction vector of the target area includes: Determine two direction vectors respectively according to the normal vector of the target area and the boundary direction vector of the target area; Another direction vector is determined according to the two direction vectors and the constraint relationship between the direction vectors.

6. The method according to claim 4, characterized in that The step of performing boundary fitting on the target area according to the target three-dimensional point cloud data to obtain a boundary direction vector of the target area includes: Extracting boundaries according to the target three-dimensional point cloud data to determine boundary point cloud data corresponding to the target area; A straight line fitting is performed according to the boundary point cloud data to obtain a boundary direction vector of the target area.

7. The method according to claim 1, characterized in that The determining of the coordinate origin according to the three-dimensional coordinates of the plurality of key points and the three-dimensional direction vectors comprises: Determine the three-dimensional coordinates of parallel key points corresponding to the multiple key points according to the three-dimensional coordinates of the multiple key points, the three-dimensional direction vector and the vehicle data of the vehicle to be detected; The coordinate origin is calculated according to the three-dimensional coordinates of the multiple key points and the three-dimensional coordinates of the parallel key points.

8. A vehicle posture detection device, characterized in that: The device comprises: A vehicle data acquisition module, used to acquire a pre-collected two-dimensional image of the vehicle to be detected at a preset viewing angle, and three-dimensional point cloud data of the vehicle to be detected at the preset viewing angle; A key point recognition module, used to recognize key points of the vehicle in the two-dimensional image, and obtain two-dimensional coordinates of multiple key points in the vehicle to be detected; A three-dimensional mapping module, used for performing three-dimensional mapping on the two-dimensional coordinates of the multiple key points to obtain the three-dimensional coordinates of the multiple key points; A reference coordinate system determination module, used to determine a preset reference coordinate system of the vehicle to be detected based on the three-dimensional point cloud data and the three-dimensional coordinates of the multiple key points; A spatial posture determination module, used to determine the spatial posture of the vehicle to be detected according to the preset reference coordinate system; The reference coordinate system determination module comprises: A target point cloud determination unit, used to determine target three-dimensional point cloud data corresponding to a target area in the vehicle to be detected from the three-dimensional point cloud data; A direction vector calculation unit, used to calculate the three-dimensional direction vector of the vehicle to be detected according to the target three-dimensional point cloud data; A coordinate origin determination unit, configured to determine a coordinate origin according to the three-dimensional coordinates of the plurality of key points and the three-dimensional direction vector; A reference coordinate system determining unit is used to determine the preset reference coordinate system according to the coordinate origin and the three-dimensional direction vector.

9. A computer device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor, and when the computer device is running, the processor and the storage medium communicate through the bus, and the processor executes the program instructions to perform the steps of the vehicle posture detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the vehicle posture detection method as described in any one of claims 1 to 7 are executed.

Images