Target shape estimation method and apparatus

Abstract

The application discloses a target shape estimation method and device, relates to the technical field of automatic driving, and is used for determining a target orientation, performing rectangular fitting on the target orientation, and thus determining target shape information. The method comprises the following steps: determining at least one target visible edge in a plurality of edges on a target outer envelope according to the target outer envelope, wherein the target outer envelope is determined by target point cloud data, and each target visible edge comprises a point that can be observed by a sensor at an observation point; determining one or more initial orientations according to the orientation of the at least one target visible edge; determining a target orientation according to the one or more initial orientations; and performing rectangular fitting according to the target orientation, and thus determining target shape information.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a target shape estimation method and apparatus. Background Technology

[0002] LiDAR (Light Detection and Ranging) possesses high-precision, high-resolution ranging and angle measurement capabilities. By processing and analyzing the point cloud data collected by LiDAR, it is possible to detect, track, and identify targets in the surrounding environment. During autonomous driving, vehicles use perception modules to detect the surrounding environment, obtaining shape information of targets (such as other vehicles). Based on this shape information, they can track targets, determine their behavioral intentions, and formulate corresponding autonomous driving strategies. Therefore, target shape estimation is an essential function of the perception module and is crucial for autonomous driving. Target shape information includes the target's outline, size, and orientation. A vehicle's outline can be well described by a rectangle, and LiDAR typically only observes the point cloud data corresponding to the visible outline portion. Therefore, in existing technologies, the shape of the target vehicle is usually estimated based on a target shape model, such as an L-shape, a rectangle, or two orthogonal lines, and the point cloud data observed by LiDAR. The direction of the longer side of the L-shape or rectangle represents the target vehicle's orientation. However, the target shape model is quite sensitive to the distribution of point cloud data. If the point cloud data is sparse, or if the distribution changes over time or relative to the lidar's viewing angle, the data points inside the target will affect the rectangular or L-shaped distribution of the point cloud data. Therefore, in the existing technology, the robustness of target orientation estimation is low, and the accuracy of target shape analysis is also low. Summary of the Invention

[0003] This application provides a target shape estimation method and apparatus, which determines the target outer envelope based on target point cloud data, determines the target orientation using the visible edges of the target outer envelope, and then determines the target shape information by performing rectangle fitting on the target orientation, thereby improving the stability of target orientation estimation and the accuracy of target shape analysis.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] In a first aspect, embodiments of this application provide a target shape estimation method applied in autonomous driving. The method includes: determining at least one visible edge on the target's outer envelope, wherein the target's outer envelope is determined by target point cloud data, and each visible edge includes points observable by a sensor at the observation point, such as a radar sensor. Then, determining one or more initial orientations based on the orientation of the at least one visible edge. Further determining the target orientation based on these one or more initial orientations. Finally, performing rectangle fitting based on the target orientation to determine the target shape information.

[0006] In the target shape estimation method described in this application embodiment, through the above process, firstly, at least one visible edge of the target on the target's outer envelope is determined, and then one or more initial orientations are determined based on the orientation of this at least one visible edge. These one or more initial orientations are then further optimized to obtain the target orientation, and finally, rectangle fitting is performed on the target orientation to determine the target shape information. Firstly, using the visible edge information on the target's outer envelope determined from the target point cloud data for target orientation estimation can reduce the influence of data points inside the target on the target orientation estimation, improving the robustness of the target orientation estimation. Secondly, using the orientation of the visible edge of the target to determine the value of the initial orientation, and then using the initial orientation instead of [0, 2π] for target orientation estimation, can reduce the computational complexity in the target orientation estimation process. Finally, optimizing these one or more initial orientations to obtain the target orientation can further improve the accuracy of the target orientation estimation and the accuracy of determining the target shape information.

[0007] In one possible implementation, determining at least one visible edge on the target envelope, specifically includes: determining a dividing line between at least one visible edge and at least one invisible edge based on the target envelope. Then, determining at least one visible edge and at least one invisible edge based on the dividing line. The dividing line is the line connecting the vertex corresponding to the maximum azimuth angle and the vertex corresponding to the minimum azimuth angle in the target envelope. The invisible edges include points that cannot be observed by the sensor at the observation point. At least one vertex of the invisible edge is located on a different side of the dividing line from the observation point, and at least one vertex of the visible edge is located on the same side of the dividing line from the observation point.

[0008] In one possible implementation, determining one or more initial orientations based on the orientation of at least one target visible edge specifically includes: for each first target visible edge among the at least one target visible edge, analyzing the two endpoints of the first target visible edge according to a first preset algorithm to determine the orientation of the first target visible edge as the initial orientation corresponding to that first target visible edge. Wherein, the first target visible edge is either a target visible edge whose length exceeds the average length among the at least one target visible edge, or the first target visible edge is the longest target visible edge among the at least one target visible edge.

[0009] Through the above process, based on the visible edge information on the outer envelope of the target, the orientation of the first visible edge of the target is selected as the initial orientation, and the target orientation is estimated based on the initial orientation to narrow the range of target orientation values. This improves the accuracy of determining the target orientation while reducing the computational complexity in the target orientation estimation process.

[0010] In one possible implementation, the first preset algorithm is: Where S represents the set of one or more initial orientations, Indicates the orientation of the visible edge of the first target. Indicates the orientation of the visible edge of the first target. The initial orientation θ corresponding to the visible edge of the first target i , and Let n and i represent the coordinates of the two endpoints of the visible edge of the first target in a two-dimensional coordinate system with the observation point as the origin, n being the number of visible edges of the first target, and i being an integer greater than 0.

[0011] In one possible implementation, determining the target orientation based on one or more initial orientations specifically includes: for each visible edge of a first target, rotating the initial orientation corresponding to the visible edge of the first target by different angles within a first preset angle range to obtain multiple candidate target orientations corresponding to the visible edge of the first target. Then, for each candidate target orientation, determining one or more points on the visible edge of the first target as interpolation points, and then determining one or more target distances corresponding to the one or more interpolation points. The target distance corresponding to each interpolation point is the minimum distance between the interpolation point and two adjacent edges of a first rectangle, which is obtained by rectangle fitting of the candidate target orientations. Subsequently, the multiple target distances obtained are summed to determine the sum of target distances corresponding to the candidate target orientations. Among the multiple candidate target orientations corresponding to each visible edge of the first target, the candidate target orientation corresponding to the minimum sum of target distances is determined as the first target orientation, and this first target orientation is determined as the target orientation.

[0012] Through the above process, multiple candidate target orientations are determined based on the initial orientation on the visible edge of the first target. These candidate orientations are then filtered using interpolation points on the visible edge to determine the final target orientation. Narrowing the range of target orientation values ​​using the visible edge of the first target and filtering from multiple candidate orientations improves the accuracy of target orientation determination. Furthermore, using interpolation points on the visible edge of the first target to filter candidate orientations reduces the influence of data points within the target on the target orientation estimation, further enhancing the robustness and accuracy of the target orientation estimation.

[0013] In one possible implementation, determining the target orientation based on one or more initial orientations specifically includes: for each visible edge of a first target, rotating the initial orientation corresponding to the visible edge of the first target by different angles within a first preset angle range to obtain multiple candidate target orientations corresponding to the visible edge of the first target. Then, for each candidate target orientation, determining one or more points on the visible edge of the first target as interpolation points, and then determining one or more target distances corresponding to these interpolation points. The target distance corresponding to the interpolation point is the minimum distance among the distances between the interpolation point and two adjacent sides of a first rectangle, which is obtained by rectangle fitting of the candidate target orientations. The variance of the obtained multiple target distances is calculated to determine the target distance distribution variance corresponding to the candidate target orientation. Among the multiple candidate target orientations corresponding to each visible edge of the first target, the candidate target orientation corresponding to the smallest target distance distribution variance is determined as the first target orientation, and this first target orientation is determined as the target orientation.

[0014] In one possible implementation, determining the first target orientation as the target orientation specifically includes: among one or more preset target orientations corresponding to the first target orientation, determining the preset target orientation with the smallest difference from the target velocity direction as the target orientation.

[0015] Through the above process, based on the target velocity direction in the target tracking information, the target orientation with the smallest difference from the target velocity direction is determined from one or more preset target orientations corresponding to the target orientation, which can solve the problem of ambiguous target orientation.

[0016] In one possible implementation, determining the first target orientation as the target orientation specifically includes: determining the average value of the historical trajectory direction of the target within a preset time period, then calculating the difference between each preset target orientation corresponding to the first target orientation and the average value, and determining the preset target orientation corresponding to the smallest difference as the target orientation.

[0017] Through the above process, based on the target trajectory direction in the target tracking information, among one or more preset target orientations corresponding to the target orientation, the preset target orientation with the smallest difference from the average value of the target trajectory direction is determined as the target orientation, which can solve the problem of ambiguous target orientation.

[0018] In one possible implementation, the target shape information is determined by performing rectangle fitting based on the target orientation. Specifically, this includes: using a second preset algorithm to perform rectangle fitting on the target orientation to determine the target shape information. The second preset algorithm is L1 = max(P i T ·e1)-min(P i T ·e1), L2=max(P i T ·e2)-min(P i T ·e2), V1=[min(P i T ·e1),min(P i T ·e2)]·A,V2=[max(P i T ·e1),min(P i T ·e2)]·A,V3=[max(P i T ·e1),max(P i T ·e2)]·A,V4=[min(P i T ·e1),max(P i T ·e2)]·A, e1 = [cos(θ), sin(θ)], e2 = [-sin(θ), cos(θ)]. L1 and L2 are the lengths of the two perpendicular sides of the fitted rectangle corresponding to the target orientation, respectively. P i T Let θ be the transpose of the matrix of interpolation points on the visible edge of the target corresponding to the target orientation, e1 and e2 represent the direction vectors corresponding to L1 and L2 respectively, V1, V2, V3 and V4 are the vertices of the rectangle, A is the transformation matrix, and θ is the target orientation.

[0019] Secondly, embodiments of this application provide a target shape estimation device applied in autonomous driving. The device includes a processing unit. The processing unit is configured to determine at least one visible edge on the target's outer envelope based on the target's outer envelope, wherein the target's outer envelope is determined from target point cloud data, and each visible edge includes points observable by a sensor at the observation point. Then, based on the orientation of the at least one visible edge, one or more initial orientations are determined. Furthermore, based on these one or more initial orientations, the target orientation is determined. Finally, rectangle fitting is performed based on the target orientation to determine the target shape information.

[0020] In one possible implementation, the processing unit is specifically configured to determine a dividing line between at least one visible edge and at least one invisible edge of the target, based on the target's outer envelope. Then, based on this dividing line, at least one visible edge and at least one invisible edge of the target are determined. The dividing line is the line connecting the vertex corresponding to the maximum azimuth angle and the vertex corresponding to the minimum azimuth angle in the target's outer envelope. The invisible edges of the target include points that cannot be observed by the sensor at the observation point. At least one vertex of the invisible edge of the target is located on a different side of the dividing line from the observation point, and at least one vertex of the visible edge of the target is located on the same side of the dividing line from the observation point.

[0021] In one possible implementation, the processing unit is further configured to, for each of the at least one target visible edges, analyze the two endpoints of the first target visible edge according to a first preset algorithm, and determine the orientation of the first target visible edge as the initial orientation corresponding to the first target visible edge. Wherein, the first target visible edge is either a target visible edge whose length exceeds the average length among the at least one target visible edges, or the first target visible edge is the longest target visible edge among the at least one target visible edges.

[0022] In one possible implementation, the first preset algorithm is: Where S represents a set of one or more initial orientations, Indicates the orientation of the visible edge of the first target. Indicates the edge corresponding to the first target. The initial orientation θ corresponding to the visible edge of the first target i , and Let n and i represent the coordinates of the two endpoints of the visible edge of the first target in a two-dimensional coordinate system with the observation point as the origin, n being the number of visible edges of the first target, and i being an integer greater than 0.

[0023] In one possible implementation, the processing unit is further configured to, for each visible edge of the first target, rotate the initial orientation corresponding to the visible edge of the first target by different angles within a first preset angle range to obtain multiple candidate target orientations corresponding to the visible edge of the first target. Then, for each candidate target orientation, one or more points on the visible edge of the first target are determined as interpolation points, and then one or more target distances corresponding to the one or more interpolation points are determined. The target distance corresponding to the interpolation point is the minimum distance between the interpolation point and two adjacent edges of a first rectangle, which is obtained by rectangle fitting of the candidate target orientations. The multiple target distances are summed to determine the sum of target distances corresponding to the candidate target orientations. Among the multiple candidate target orientations corresponding to each visible edge of the first target, the candidate target orientation corresponding to the minimum sum of target distances is determined as the first target orientation.

[0024] In one possible implementation, the processing unit is further configured to, for each visible edge of the first target, rotate the initial orientation corresponding to the visible edge of the first target by different angles within a first preset angle range to obtain multiple candidate target orientations corresponding to the visible edge of the first target. Then, for each candidate target orientation, one or more points on the visible edge of the first target are determined as interpolation points, and then one or more target distances corresponding to the one or more interpolation points are determined. The target distance corresponding to the interpolation point is the minimum distance between the interpolation point and two adjacent sides of a first rectangle, which is obtained by rectangle fitting of the candidate target orientations. The variance of the multiple target distances is calculated to determine the target distance distribution variance corresponding to the candidate target orientation. Then, among the multiple candidate target orientations corresponding to each visible edge of the first target, the candidate target orientation corresponding to the minimum target distance distribution variance is determined as the first target orientation, and this first target orientation is determined as the target orientation.

[0025] In one possible implementation, the device further includes a determining unit, which is used to determine, among one or more preset target orientations corresponding to the first target orientation, the preset target orientation with the smallest difference from the target velocity direction as the target orientation.

[0026] In one possible implementation, the device further includes a determining unit, which is used to determine the average value of the historical trajectory direction of the target within a preset time period, and then calculate the difference between each preset target orientation corresponding to the first target orientation and the average value, and determine the preset target orientation corresponding to the smallest difference as the target orientation.

[0027] In one possible implementation, the processing unit is further configured to use a second preset algorithm to perform rectangle fitting on the target orientation to determine the target shape information. The second preset algorithm is L1 = max(P i T·e1)-min(P i T ·e1), L2=max(P i T ·e2)-min(P i T ·e2), V1=[min(P i T ·e1),min(P i T ·e2)]·A,V2=[max(P i T ·e1),min(P i T ·e2)]·A,V3=[max(P i T ·e1),max(P i T ·e2)]·A,V4=[min(P i T ·e1),max(P i T ·e2)]·A, e1 = [cos(θ), sin(θ)], e2 = [-sin(θ), cos(θ)]. L1 and L2 are the lengths of the two perpendicular sides of the fitted rectangle corresponding to the target orientation, respectively. P i T Let θ be the transpose of the matrix of interpolation points on the visible edge of the target corresponding to the target orientation, e1 and e2 represent the direction vectors corresponding to L1 and L2 respectively, V1, V2, V3 and V4 are the vertices of the rectangle, A is the transformation matrix, and θ is the target orientation.

[0028] Thirdly, this application provides a target shape estimation apparatus, characterized in that it includes: a processor, a memory, and a communication interface. The communication interface is used for communicating with other devices or communication networks, and the memory is used to store one or more programs, said one or more programs including computer-executable instructions. When the apparatus is running, the processor executes the computer-executable instructions stored in the memory to cause the apparatus to perform the target shape estimation method described in the first aspect and any of its various optional implementations.

[0029] Fourthly, this application provides a computer-readable storage medium storing one or more programs, wherein the computer-readable storage medium stores instructions, the one or more programs including instructions that, when executed by a computer, cause the computer to perform the target shape estimation method described in the first aspect and any of its various alternative implementations.

[0030] Fifthly, this application provides a computer program product containing instructions, characterized in that, when the instructions are executed on a computer, the computer performs the target shape estimation method described in the first aspect and any of its various alternative implementations. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of a search-type target shape estimation method in the prior art;

[0032] Figure 2 A structural schematic diagram of an autonomous vehicle provided in this application embodiment. Figure 1 ;

[0033] Figure 3 A structural schematic diagram of an autonomous vehicle provided in this application embodiment. Figure 2 ;

[0034] Figure 4 A schematic diagram of the structure of a computer system provided in an embodiment of this application;

[0035] Figure 5 This is a schematic diagram of the structure of a chip system provided in an embodiment of this application;

[0036] Figure 6 This application provides an illustration of a cloud-based command-driven autonomous vehicle. Figure 1 ;

[0037] Figure 7 This application provides an illustration of a cloud-based command-driven autonomous vehicle. Figure 2 ;

[0038] Figure 8 This application provides a schematic diagram of the structure of a computer program product according to an embodiment of the present application.

[0039] Figure 9 This is a schematic diagram of the target shape estimation method provided in the embodiments of this application;

[0040] Figure 10 (a) is a schematic diagram of a target outer envelope provided in an embodiment of this application;

[0041] Figure 10 (b) is a schematic diagram of a target visible edge provided in an embodiment of this application;

[0042] Figure 11 (a) is a schematic diagram of an initial orientation provided in an embodiment of this application;

[0043] Figure 11 (b) is a schematic diagram of a candidate initial orientation provided in an embodiment of this application;

[0044] Figure 12 A schematic diagram illustrating the determination of target orientation provided in an embodiment of this application;

[0045] Figure 13 (a) is a schematic diagram of a target orientation provided in an embodiment of this application;

[0046] Figure 13 (b) is a schematic diagram of a target shape provided in an embodiment of this application;

[0047] Figure 14 Schematic diagram of the target shape estimation device provided in the embodiments of this application Figure 1 ;

[0048] Figure 15 Schematic diagram of the target shape estimation device provided in the embodiments of this application Figure 2 . Detailed Implementation

[0049] Before introducing the embodiments of this application, the technical terms involved in the embodiments of this application will be introduced:

[0050] Point cloud data: Point cloud data refers to scanned data recorded in the form of points, each of which can be represented by three-dimensional coordinates. In addition to geometric location, point cloud data may also include reflection intensity information. Reflection intensity information is the echo intensity collected by the laser scanner receiving device. This intensity information is related to the surface material and roughness of the target, the angle of incidence, the emission energy of the instrument, and the laser wavelength.

[0051] In existing technologies, methods for estimating the shape of a target vehicle using collected point cloud data are generally divided into two categories: non-search-based target shape estimation methods and search-based target shape estimation methods. A brief introduction to these two existing target shape estimation methods is given below:

[0052] One type is the non-search type, which typically models target shape estimation as an optimization problem that minimizes a cost function. For example, in the paper H. Zhao, Q. Zhang, M. Chiba, R. Shibasaki, J. Cui, and H. Zhao, “Moving object classification using horizontal laser scan data,” in IEEE International Conference on Robotics and Automation, 2009, pp. 2424–2430, the PCA algorithm is used to extract the principal direction (the eigenvector corresponding to the largest eigenvalue) of the target point cloud distribution, and the principal direction is used as an estimate of the target orientation. In the paper R. MacLachlan and C. Mertz, “Tracking of moving objects from a moving vehicle using a scanning laser rangefinder,” in IEEE Intelligent Transportation Systems Conference, 2006, pp. 301–306, an iterative weighted least squares method is used for line fitting, and weighted least squares is used to remove outliers. Based on the line fitting, a right-angle shape fitting is performed on the target, and finally the target orientation and size can be obtained. In the paper X. Shen, S. Pendleton, and MH Ang, “Efficient L-shapefitting of laser scanner data for vehicle pose estimation,” in IEEE Conference on Robotics, Automation and Mechatronics, 2015, pp. 173–178, the complexity of corner search is reduced from exponential to linear by utilizing the scanning sequence information of LiDAR. By searching for corners, the target point set is divided into two subsets, each of which corresponds to one edge of the target. By fitting straight lines to the points of the two subsets, the two orthogonal edges of the target can be obtained.

[0053] In non-search-based target shape estimation methods, the principal direction of the target point cloud distribution extracted using the PCA algorithm is not equivalent to the target orientation. Using the principal direction as the target orientation estimate may result in significant errors. Therefore, non-search-based target shape estimation methods exhibit low robustness in target orientation estimation.

[0054] Another type involves a full-range search of the target's orientation angle, determining the target's orientation according to specific criteria, and selecting the orientation angle that minimizes the cost function or maximizes the matching function under those criteria. These criteria typically include those described in the literature Xiao Zhang, Wenda Xu, Chiyu Dong, and John M. Dolan; "Efficient L-shape fitting for vehicle detection using laser scanners," in IEEE Intelligent Vehicles Symposium (IV), 2017: minimizing the rectangle area, minimizing the sum of distances from a point to the rectangle's sides, and minimizing the variance of the distance distribution from a point to the rectangle's sides.

[0055] The following section provides a detailed introduction to existing search-based methods, specifically techniques that estimate the target's orientation by searching the entire range of target orientation angles according to specific criteria. For example... Figure 1 As shown, in this technical solution, the input data is first determined to be the clustered target point cloud data, and then the search parameters are set. The search parameters include the target orientation angle search range, such as [0, 2π], and the angle search interval, such as △θ. In each search, the projection of the point cloud data onto the direction vector is calculated first, and then the optimization function value is calculated. When calculating the projection of the point cloud data onto the direction vector, the unit direction vectors of the two sides of the L-shape corresponding to the current search angle are first calculated, as shown in Equation (1).

[0056] e1=[cos(θ),sin(θ)], e2=[-sin(θ),cos(θ)] (1)

[0057] Where θ is the current search angle, cos() is the cosine function, and sin() is the sine function. Then, the projections of the data points in the target point cloud data onto the directions e1 and e2 are calculated, as shown in equation (2).

[0058]

[0059] Where X is the target point cloud data matrix, each row of X represents the coordinates of a data point in the target point cloud data on the x-axis and y-axis, C1 is the projection vector of all target point cloud data in the e1 direction, C2 is the projection vector of all target point cloud data in the e2 direction, (·) TThis is a transpose operation. The optimization function value is calculated based on the projection and recorded. After the search is complete, i.e., after the cyclic search process ends, the optimal value of the optimization function, i.e., the search angle corresponding to the maximum / minimum optimization function value, is selected as the estimate of the target orientation. Based on this angle, the line equation parameters of the four sides corresponding to the target shape are calculated, ultimately obtaining the four vertices of the target shape. Commonly used optimization functions include minimizing the rectangle area, minimizing the sum of distances from the target point cloud to the edges, and minimizing the variance of the distance distribution from the target point cloud to the edges.

[0060] In search-based target shape estimation methods, the target point cloud data must also satisfy an L-shaped distribution. This L-shaped distribution is highly sensitive to the distribution of the point cloud data. If the point cloud data is sparse, or if its distribution changes over time or relative to the lidar's viewing angle, the data points inside the target will affect the rectangular or L-shaped distribution of the point cloud data. Therefore, the robustness of target orientation estimation is low. Furthermore, in search-based target shape estimation methods, due to the rationality of the optimization function and the limitation of the target orientation search interval, the accuracy of the obtained target shape information cannot be guaranteed when the distribution of the target point cloud data differs significantly from the rectangular or L-shaped model. Moreover, under the condition of a reasonable optimization function, the higher the accuracy of the target orientation estimation, the smaller the target orientation search interval, and the higher the computational complexity. The minimum rectangle area criterion is generally suitable for cases with high target point cloud density; however, its performance is poor when the target point cloud density is low or the target is large. Minimizing the distance from the target point cloud to the edge and minimizing the variance of the distance distribution from the target point cloud to the edge are more sensitive to the data points of specific components of the target, such as the data points corresponding to the rearview mirror of the vehicle and the internal structure of the vehicle itself (rear window, cabin, etc.).

[0061] Furthermore, in the aforementioned existing technical solutions, when the observation angle of the lidar is approximately parallel or perpendicular to the target orientation, the existing technology has an ambiguity problem in estimating the target orientation.

[0062] In order to solve the above-mentioned technical problems existing in the prior art, this application proposes a target shape estimation method. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0063] The target shape estimation method provided in this application can be applied to vehicles with autonomous driving functions, or to other devices (such as cloud servers) that control autonomous driving functions. The vehicle can implement the target shape estimation method provided in this application through its components (including hardware and software). An initial orientation is determined based on the orientation of the visible edge of the target on the target envelope, wherein the target envelope is determined by target point cloud data. Then, one or more candidate target orientations are determined based on the initial orientation, the initial orientation is optimized, and one of the candidate target orientations is determined as the target orientation. Finally, a rectangle is fitted based on the target orientation to obtain the target shape information. Alternatively, other devices (such as servers) can implement the target shape estimation method of this application. An initial orientation is determined based on the orientation of the visible edge of the target on the target envelope, wherein the target envelope is determined by target point cloud data. Then, one or more candidate target orientations are determined based on the initial orientation, and the initial orientation is optimized by combining interpolation points on the visible edge of the target, and one of the candidate target orientations is determined as the target orientation. Finally, a rectangle is fitted based on the target orientation to obtain the target shape information. The other device can also send the obtained target shape information to the vehicle.

[0064] Figure 2 This is a functional block diagram of a vehicle 100 provided in an embodiment of this application. In one embodiment, the vehicle 100 is configured in an autonomous driving mode. For example, when in autonomous driving mode, the vehicle 100 can perform actions such as obstacle avoidance, following other vehicles, lane keeping, and automatic parking based on a driving strategy, wherein the driving strategy is determined according to the state of the vehicle and its surrounding environment. Alternatively, when in autonomous driving mode, the vehicle 100 can perform actions such as obstacle avoidance, following other vehicles, lane keeping, and automatic parking based on a driving strategy.

[0065] Vehicle 100 may include various subsystems, such as a mobility system 110, a sensor system 120, a control system 130, one or more peripheral devices 140, a power supply 150, a computer system 160, and a user interface 170. Optionally, vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of vehicle 100 may be interconnected via wired or wireless means.

[0066] The propulsion system 110 may include components that provide powered motion to the vehicle 100. In one embodiment, the propulsion system 110 may include an engine 111, a transmission 112, an energy source 113, and wheels 114. The engine 111 may be an internal combustion engine, an electric motor, an air-compressed engine, or a combination of other types of engines, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air-compressed engine. The engine 111 can convert the energy source 113 into mechanical energy.

[0067] There are many types of energy sources 113, such as gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries and other power sources. Energy source 113 can also provide energy for other subsystems of vehicle 100.

[0068] The transmission 112 can transmit mechanical power from the engine 111 to the wheels 114 to change the rotational speed of the wheels 114, etc. The transmission 112 may include at least one of the following devices: a gearbox, a differential, and a drive shaft, wherein the drive shaft may be coupled to one or more axles of one or more wheels 114. In one embodiment, the transmission 112 may also include other devices, such as a clutch.

[0069] Sensor system 120 may include several sensors that sense information about the surrounding environment of vehicle 100. For example, sensor system 120 may include at least one of the following sensors: a positioning system 121 (e.g., GPS, BeiDou, or other positioning systems), an inertial measurement unit (IMU) 122, a lidar 123, a rangefinder 124, and a camera 125. Sensor system 120 may also include sensors from the vehicle 100's internal systems (e.g., an in-vehicle air quality monitor, fuel gauge, oil temperature gauge, etc.). One or more sensor data collected by these sensors can be used to detect objects and their corresponding characteristics (position, shape, orientation, speed, etc.), and this detection and identification is crucial for the safe operation of vehicle 100.

[0070] The positioning system 121 can be used to estimate the geographical location of the vehicle 100. The IMU 122 is used to sense changes in the position and orientation of the vehicle 100 based on inertial acceleration. In one embodiment, the IMU 122 can be a combination of an accelerometer and a gyroscope.

[0071] The lidar 123 can use lasers to sense objects in the surrounding environment of the vehicle 100. In some embodiments, in addition to sensing objects, the lidar 123 can also be used to sense the position, speed, and / or direction of travel of objects.

[0072] The rangefinder 124 can be used to sense the distance between the vehicle 100 and objects in the surrounding environment.

[0073] Camera 125 can be used to capture multiple images of the surrounding environment of vehicle 100. Camera 125 can be a still camera or a video camera.

[0074] The control system 130 can control the operation of the vehicle 100 and its components. The control system 130 may include at least one of various elements, such as a steering system 131, a throttle 132, a braking unit 133, a computer vision system 134, a route control system 135, and an obstacle avoidance system 136.

[0075] The steering system 131 is used to adjust the forward direction of the vehicle 100. For example, the steering system 131 can be a steering wheel system.

[0076] The throttle 132 further controls the speed of the vehicle 100 by controlling the operating speed of the engine 111.

[0077] The braking unit 133 is used to control the deceleration of the vehicle 100. The braking unit 133 can use friction to reduce the rotational speed of the wheel 114. Optionally, the braking unit 133 can reduce the rotational speed of the wheel 114 by converting the kinetic energy of the wheel 114 into an electric current. The braking unit 133 can also take other forms to reduce the rotational speed of the wheel 114, thereby controlling the speed of the vehicle 100.

[0078] The computer vision system 134 can process and analyze images captured by the camera 125 to identify objects and / or object features in the environment surrounding the vehicle 100. These objects and / or object features include traffic signals, road boundaries, and obstacles. In some embodiments, the computer vision system 134 can use at least one of object recognition algorithms, structure from motion (SFM) algorithms, video tracking, or other computer vision techniques to perform operations such as mapping the environment, tracking objects, and estimating object speeds.

[0079] The route control system 135 is used to determine the driving route of the vehicle 100. In some embodiments, the route control system 135 may combine sensor data from the sensor system 120 and one or more predetermined map data to determine the driving route for the vehicle 100.

[0080] The obstacle avoidance system 136 is used to identify and assess obstacles and plan ways to overcome potential obstacles in the surrounding environment of the vehicle 100, such as by avoiding or detouring.

[0081] Of course, optionally, the control system 130 may add other components besides those mentioned above, or reduce some of the aforementioned components, or replace the aforementioned components with other components.

[0082] Vehicle 100 interacts with peripheral devices 140, such as external sensors, other vehicles, and other computer systems, via user interface 170. Peripheral devices 140 may include at least one of a wireless communication system 141, an on-board computer 142, a microphone 143, a speaker 144, and / or other peripheral devices. Exemplarily, the on-board computer 142 may provide information to vehicle 100 or its user via user interface 170, and receive information from vehicle 100 or its user. In some embodiments, the on-board computer 142 may be operated via a touchscreen. In other cases, user interface 170 may also provide a means for vehicle 100 to communicate with other devices located within vehicle 100. For example, microphone 143 may receive audio (e.g., voice commands or other audio input) from a user of vehicle 100 via user interface 170. Similarly, speaker 144 may output audio to a user of vehicle 100 via user interface 170.

[0083] The wireless communication system 141 can be used to wirelessly communicate with one or more devices via a communication network or directly. For example, the wireless communication system 141 can use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as LTE, or 5G cellular communication. The wireless communication system 141 can also communicate with a wireless local area network (WLAN) using WiFi or other wireless protocols. In some embodiments, the wireless communication system 141 can communicate directly with devices, such as various vehicle communication systems, using infrared links, Bluetooth, or ZigBee. Optionally, the wireless communication system 141 may include one or more dedicated short-range communications (DSRC) devices.

[0084] The power source 150 can provide power to various components of the vehicle 100. In one embodiment, the power source 150 can be a rechargeable lithium-ion or lead-acid battery, with one or more such battery packs configured as a power source to provide power to various components of the vehicle 100. In some embodiments, such as some fully electric vehicles, the power source 150 and the energy source 113 can be implemented together.

[0085] Some or all of the functions of vehicle 100 are controlled by computer system 160. Computer system 160 may include at least one processor 161, which executes instructions 1621 stored in a non-transitory computer-readable medium such as data storage device 162. Computer system 160 may also employ a distributed approach to control multiple computer devices in individual components or subsystems of vehicle 100. The processor 161 can be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor can be a dedicated device such as an application-specific integrated circuit (ASIC) or other hardware-based processor. Although... Figure 2 The processor, memory, and other components within the same physical housing are functionally illustrated; however, those skilled in the art will understand that the processor, computer system, or memory may actually include multiple processors, computer systems, or memories that may be stored within the same physical housing, or multiple processors, computer systems, or memories that may not be stored within the same physical housing. For example, memory may be a hard disk drive, or other storage media located in a different physical housing. Therefore, references to processors or computer systems will be understood to include references to collections of processors or computer systems or memories that may operate in parallel, or collections of processors or computer systems or memories that may not operate in parallel. Unlike using a single processor to perform the steps described herein, some components, such as steering and deceleration components, may each have their own processor, performing only calculations relevant to the function of that particular component.

[0086] In the various aspects described herein, the processor can be located remotely from and wirelessly communicate with the vehicle. In other aspects, some of the processes described herein are executed by a processor located within the vehicle, while others are executed by a remote processor, including taking the necessary steps to perform a single operation.

[0087] In some embodiments, the data storage device 162 may include instructions 1621 (e.g., program logic) that can be executed by the processor 161 to perform various functions of the vehicle 100, including those described above. The data storage device 162 may also include additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the mobility system 110, sensor system 120, control system 130, and peripheral devices 140.

[0088] In addition to instruction 1621, data storage device 162 may also store data such as road maps, route information, the position and / or direction and / or speed of vehicle 100, data of other vehicles, and other information. When vehicle 100 is in autonomous, semi-autonomous, and / or manual driving modes, the above data and related information can be used by vehicle 100 and computer system 160.

[0089] For example, in one possible embodiment, the data storage device 162 can acquire information such as the distance between itself and other vehicles, the shape information of other vehicles, and the speed of other vehicles obtained by the vehicle based on the speed and distance measurement functions of the LiDAR 123. The data storage device 162 can also acquire environmental information from the sensor system 120 or other components of the vehicle 100. Environmental information may include, for example, whether there are green belts, lanes, pedestrians, etc., in the vicinity of the vehicle's current environment, or whether the vehicle calculates the presence of green belts, pedestrians, etc., using machine learning algorithms. In addition to the above, the data storage device 162 can also store the vehicle's own state information and the state information of other vehicles interacting with it. The vehicle's state information includes, but is not limited to, the vehicle's speed, acceleration, and heading angle. Thus, the processor 161 can acquire this information from the data storage device 162 and, based on the environmental information of the vehicle's environment, the vehicle's own state information, the state information of other vehicles, and traditional rule-based driving strategies, derive a final driving strategy to control the vehicle 100 for autonomous driving.

[0090] User interface 170 is used to provide information to or receive information from a user of vehicle 100. Optionally, user interface 170 can interact with one or more input / output devices within a set of peripheral devices 140, such as one or more of wireless communication system 141, vehicle computer 142, microphone 143, and speaker 144.

[0091] Computer system 160 can control vehicle 100 based on information acquired from various subsystems (e.g., driving system 110, sensor system 120, and control system 130) and information received from user interface 170. For example, computer system 160 can control steering system 131 to change the vehicle's direction of travel based on information from control system 130, thereby avoiding obstacles detected by sensor system 120 and obstacle avoidance system 136. In some embodiments, computer system 160 can control many aspects of vehicle 100 and its subsystems.

[0092] Optionally, one or more of these components may be installed separately from or associated with vehicle 100. For example, data storage device 162 may exist partially or completely separate from vehicle 100. The components may be coupled together for communication via wired and / or wireless means.

[0093] Optionally, the components described above are merely examples. In actual applications, components in each of the above modules may be added or removed as needed. Figure 2 This should not be construed as a limitation on the embodiments of this application.

[0094] An autonomous vehicle traveling on a road, such as vehicle 100 above, can determine its speed adjustment instructions based on other vehicles in its surrounding environment. Objects in the surrounding environment of vehicle 100 can be traffic control equipment, green belts, or other types of objects. In some examples, each object in the surrounding environment can be considered independently, and the speed adjustment instructions for vehicle 100 can be determined based on the object's individual characteristics, such as its current speed, acceleration, and distance from the vehicle.

[0095] Optionally, the vehicle 100, which is an autonomous vehicle, or the computer equipment associated with it (such as...) Figure 2 The computer system 160, computer vision system 134, and data storage device 162 can predict the behavior of the identified objects based on the shape information of the identified objects and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of each other; therefore, the behavior of a single identified object can also be predicted by considering all identified objects holistically. The vehicle 100 can adjust its driving strategy based on the predicted behavior of the identified objects. In other words, the autonomous vehicle can determine what stable state the vehicle needs to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the objects. In this process, other factors can also be considered in determining the speed adjustment command of the vehicle 100, such as the lateral position of the vehicle 100 in the road, the curvature of the road, the proximity of static and dynamic objects, etc.

[0096] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computer device can also provide instructions to modify the steering angle of the vehicle 100 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance between the autonomous vehicle and nearby objects (such as cars in adjacent lanes).

[0097] The aforementioned vehicle 100 can be a car, truck, motorcycle, bus, ship, airplane, helicopter, lawnmower, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, and handcart, etc., and this application embodiment does not impose any special limitations.

[0098] In other embodiments of this application, the autonomous vehicle may further include hardware structures and / or software modules to implement the above functions in the form of hardware structures, software modules, or a combination of hardware structures and software modules. Whether a particular function is implemented in the form of hardware structures, software modules, or a combination of hardware structures and software modules depends on the specific application and design constraints of the technical solution.

[0099] See Figure 3 For example, a vehicle may include the following modules:

[0100] The environmental perception module 201 is used to acquire information about vehicles, pedestrians, and road objects in the road environment through roadside sensors and on-board sensors. During autonomous driving, the vehicle needs to fully perceive its surrounding environment, especially other vehicle targets nearby. Roadside sensors and on-board sensors can be cameras, LiDAR, millimeter-wave radar, etc. The data acquired by the environmental perception module 201 can be raw video streams or radar point cloud data. For the raw video stream data and radar point cloud data, the environmental perception module 201 can process this data into identifiable, structured data such as the position, speed, turning angle, and size of people, vehicles, and objects, determine the target category, and perform target shape estimation to obtain target shape information, thereby achieving target tracking. For example, a camera acquires visual information about the target, which is then processed and analyzed by the computer vision system 134 to identify the target category. Similarly, after the LiDAR acquires the target's point cloud data, the computer vision system 134 processes and analyzes the target's point cloud data to perform target shape estimation, thereby obtaining target shape information and achieving target tracking. The environmental perception module 201 performs target tracking based on the obtained target category and target shape information, acquires target tracking information, and transmits the target category, target shape information, and target tracking information to the decision planning module 202, providing more refined and accurate information for the decision planning of the autonomous vehicle, so as to generate driving strategies.

[0101] Decision planning module 202: This module is a traditional control module in autonomous vehicles. Its function is to generate a driving strategy based on data such as target category, target shape information and target tracking information received from the environmental perception module 201, output the action command corresponding to the driving strategy, and send the action command to the vehicle control module. The action command is used to instruct the vehicle control module to control the vehicle to perform autonomous driving.

[0102] Vehicle control module ( Figure 3 (Not shown in the image): Used to receive action instructions from the decision planning module 202 to control the vehicle to complete the autonomous driving operation.

[0103] Vehicle communication module ( Figure 3(Not shown in the image): Used for information exchange between the vehicle and other vehicles.

[0104] Storage components ( Figure 3 (Not shown in the image), used to store the executable code of each of the above modules. Running this executable code can implement some or all of the method flows of the embodiments of this application.

[0105] In one possible implementation of the embodiments of this application, such as Figure 4 As shown, Figure 2 The computer system 160 shown includes a processor 301 coupled to a system bus 302. The processor 301 can be one or more processors, each of which can include one or more processor cores. A video adapter 303 drives a display 324, which is coupled to the system bus 302. The system bus 302 is coupled to an input / output (I / O) bus 305 via a bus bridge 304. An I / O interface 306 is coupled to the I / O bus 305 and communicates with various I / O devices, such as input devices 307 (e.g., keyboard, mouse, touchscreen), a media tray 308 (e.g., CD-ROM, multimedia interface), a transceiver 309 (capable of sending and / or receiving radio communication signals), a camera 310 (capable of capturing still and moving digital video images), and an external Universal Serial Bus (USB) port 311. Optionally, the interface connected to the I / O interface 306 can be a USB interface.

[0106] The processor 301 can be any conventional processor, including a Reduced Instruction Set Computer (RISC) processor, a Complex Instruction Set Computer (CISC) processor, or a combination thereof. Optionally, the processor 301 can also be a special-purpose device such as an Application-Specific Integrated Circuit (ASIC). Optionally, the processor 301 can also be a neural network processor or a combination of a neural network processor and the aforementioned conventional processors.

[0107] Optionally, in the various embodiments described in this application, the computer system 160 may be located remotely from the autonomous vehicle and wirelessly communicate with the autonomous vehicle 100. In other aspects, some processes described in this application may be set to execute on a processor within the autonomous vehicle, while others may be executed by a remote processor, including taking actions necessary to perform a single manipulation.

[0108] Computer system 160 can communicate with software deployment server 313 via network interface 312. Optionally, network interface 312 can be a hardware network interface, such as a network interface card (NIC). Network 314 can be an external network, such as the Internet, or an internal network, such as Ethernet or a Virtual Private Network (VPN). Optionally, network 314 can also be a wireless network, such as a WiFi network or a cellular network.

[0109] Hard disk drive interface 315 is coupled to system bus 302. Hard disk drive interface 315 is connected to hard disk drive 316. System memory 317 is coupled to system bus 302. Data running in system memory 317 may include the operating system (OS) 318 and applications 319 of computer system 160.

[0110] The operating system (OS) 318 includes, but is not limited to, Shell 320 and kernel 321. Shell 320 is an interface between the user and the kernel 321 of the operating system 318. Shell 320 is the outermost layer of the operating system 318. Shell manages the interaction between the user and the operating system 318: waiting for user input, interpreting user input for the operating system 318, and processing various outputs of the operating system 318.

[0111] Kernel 321 consists of the portion of operating system 318 that manages memory, files, peripherals, and system resources, and interacts directly with the hardware. Kernel 321 of operating system 318 typically runs processes and provides inter-process communication, as well as functions such as CPU time slice management, interrupts, memory management, and I / O management.

[0112] Application 319 includes autonomous driving-related programs 323, such as programs managing the interaction between the autonomous vehicle and obstacles on the road, programs controlling the autonomous vehicle's driving route or speed, and programs controlling the interaction between the autonomous vehicle and other vehicles / autonomous vehicles on the road. Application 319 also exists on the system of deploying server 313. In one embodiment, when application 319 needs to be executed, computer system 160 can download application 319 from deploying server 313.

[0113] For example, application 319 could be an application that controls the vehicle to determine a driving strategy based on the aforementioned target category, target shape information, target tracking information, and conventional control modules. The processor 301 of computer system 160 calls application 319 to obtain the driving strategy.

[0114] Sensor 322 is associated with computer system 160. Sensor 322 is used to detect the environment surrounding computer system 160. For example, sensor 322 can detect animals, cars, obstacles, and / or pedestrian crossings. Furthermore, sensor 322 can also detect the environment around the aforementioned animals, cars, obstacles, and / or pedestrian crossings. For example, the environment around an animal, such as other animals present in the vicinity, weather conditions, and the brightness of the surrounding light. Optionally, if computer system 160 is located on an autonomous vehicle, sensor 322 can be at least one of the following devices: a camera, an infrared sensor, a chemical detector, a microphone, etc.

[0115] In other embodiments of this application, the target shape estimation method of this application can also be executed by a chip system. See also Figure 5 This is a structural diagram of a chip system provided in an embodiment of this application.

[0116] The neural network processing unit (NPU) 40 is mounted as a coprocessor on the host CPU, which assigns tasks to the NPU 40. The core of the NPU 40 is the arithmetic circuit 403. For example, the arithmetic circuit 403 is controlled by the controller 404, so that the arithmetic circuit 403 performs multiplication operations using matrix data retrieved from memory.

[0117] In some implementations, the arithmetic circuit 403 internally includes multiple process engines (PEs). In some implementations, the arithmetic circuit 403 is a two-dimensional pulsating array. Optionally, the arithmetic circuit 403 can also be a one-dimensional pulsating array, or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 403 is a general-purpose matrix processor.

[0118] For example, suppose there is an input matrix A, a weight matrix B, and an output matrix C. The arithmetic circuit 403 retrieves the corresponding data for the weight matrix B from the weight memory 402 and caches it in each PE (Execution Module) of the arithmetic circuit 403. The arithmetic circuit 403 also retrieves the corresponding data for the input matrix A from the input memory 401, and then performs matrix operations based on the input matrix A and the weight matrix B, storing part of the result or the final result of the matrix operations in the accumulator 408.

[0119] For example, the arithmetic circuit 403 can be used to implement a feature extraction model (such as a convolutional neural network model), and input image data into the convolutional neural network model. Through the operation of this model, the features of the image are obtained. Then, the image features are output to a classifier, which outputs the classification probability of objects in the image.

[0120] Unified memory 406 is used to store input and output data. Weight data in external memory is directly transferred to weight memory 402 via direct memory access controller (DMAC) 405. Input data in external memory can be transferred to unified memory 406 or input memory 401 via DMAC.

[0121] The bus interface unit (BIU) 410 is used for interaction between the advanced extensible interface (AXI) bus and the DMAC and instruction fetch buffer 409. It is also used by the instruction fetch buffer 409 to fetch instructions from external memory, and by the memory access controller 405 to fetch the original data of input matrix A or weight matrix B from external memory.

[0122] The DMAC is mainly used to move input data from external memory (DDR) to unified memory 406, or to weight data to weight memory 402, or to input data to input memory 401.

[0123] For example, in this embodiment of the application, when using the DQN model to calculate data such as target category, target shape information, and target tracking information corresponding to a target, the input data can be the input data of the DQN model, that is, the lidar point cloud data of the target object in the vehicle's surrounding environment (such as other vehicles that interact with the vehicle). The output data is the output data of the DQN model, that is, the target category, target shape information, and target tracking information of the target in the vehicle's surrounding environment.

[0124] The vector computation unit 407 may include multiple computation processing units. It is used to further process the output of the computation circuit 403 when needed, such as vector multiplication, vector addition, exponentiation, logarithmic operations, size comparisons, etc. It is mainly used for computation in non-convolutional / FC layer networks of neural networks, such as pooling, batch normalization, and local response normalization.

[0125] In some implementations, vector computation unit 407 stores the processed output vector in unified memory 406. For example, vector computation unit 407 can apply a nonlinear function to the output of arithmetic circuit 403, such as a vector of accumulated values, to generate activation values. In some implementations, vector computation unit 407 generates normalized values, merged values, or both. In some implementations, the processed output vector can also be used as activation input to arithmetic circuit 403, for example, for use in subsequent layers of a neural network.

[0126] The controller 404 is connected to the instruction fetch buffer 409, and the instructions used by the controller 404 can be stored in the instruction fetch buffer 409.

[0127] As one possible implementation, the unified memory 406, input memory 401, weight memory 402, and instruction fetch memory 409 are all on-chip memories. External memory is proprietary to this NPU 40 hardware architecture.

[0128] Combination Figures 2 to 4 The main CPU and NPU 40 work together to achieve Figure 2 The corresponding algorithms for the functions required by vehicle 100 can also be implemented. Figure 3 The corresponding algorithms for the functions required by the vehicle shown can also be implemented. Figure 4 The corresponding algorithms for the functions required by the computer system 160 shown.

[0129] In other embodiments of this application, computer system 160 may also receive information from or transfer information to other computer systems. Alternatively, sensor data collected from sensor system 120 of vehicle 100 may be transferred to another computer for processing. Figure 6 As shown, data from computer system 160 can be transmitted via a network to cloud-side computer system 510 for further processing. The network and intermediate nodes can include various configurations and protocols, including the Internet, World Wide Web, Intranet, Virtual Private Network, Wide Area Network, Local Area Network, Private Network using proprietary communication protocols of one or more companies, Ethernet, WiFi, and HTTP, as well as various combinations thereof. This communication can be performed by any device capable of transmitting data to and from other computers, such as modems and wireless interfaces.

[0130] In one example, computer system 510 may include a server with multiple computers, such as a load balancing server cluster. To receive, process, and transmit data from computer system 160, server 520 exchanges information with different nodes in the network. Computer system 510 may have a configuration similar to computer system 160 and includes a processor 530, memory 540, instruction set 550, and data storage 560.

[0131] In one example, the data 560 of server 520 may include weather-related information. For example, server 520 may receive, monitor, store, update, and transmit various information related to target objects in the surrounding environment. This information may include, for example, target category, target shape information, and target tracking information in the form of reports, radar information, forecasts, etc.

[0132] See Figure 7 This is an example of interaction between an autonomous vehicle and a cloud service center (cloud server). The cloud service center can receive information (such as data collected by vehicle sensors or other information) from the autonomous vehicles 613 and 612 within its environment 600 via a network 611, such as a wireless communication network.

[0133] Based on the received data, the cloud service center 620 runs the stored programs related to autonomous driving to control the autonomous vehicles 613 and 612. These programs can be: programs that manage the interaction between the autonomous vehicle and obstacles on the road, programs that control the autonomous vehicle's route or speed, or programs that control the interaction between the autonomous vehicle and other autonomous vehicles on the road.

[0134] For example, cloud service center 620 can provide portions of a map to vehicles 613 and 612 via network 611. In other examples, operations can be divided among different locations. For instance, multiple cloud service centers can receive, verify, combine, and / or send information reports. In some examples, information reports and / or sensor data can also be sent between vehicles. Other configurations are also possible.

[0135] In some examples, the cloud service center 620 sends suggested solutions to possible driving situations in the environment to the autonomous vehicle (e.g., informing it of obstacles ahead and how to avoid them). For example, the cloud service center 620 can assist the vehicle in determining how to proceed when facing a specific obstacle in the environment. The cloud service center 620 sends responses to the autonomous vehicle instructing it on how to proceed in a given scenario. For example, based on collected sensor data, the cloud service center 620 can confirm the presence of a temporary stop sign ahead, or, based on a "lane closure" sign and sensor data from construction vehicles, determine that a lane is closed due to construction. Accordingly, the cloud service center 620 sends suggested operating modes for the vehicle to pass through the obstacle (e.g., instructing the vehicle to change lanes onto another road). Once the cloud service center 620 observes the video stream within its operating environment 600 and confirms that the autonomous vehicle can safely and successfully pass through the obstacle, the operating steps taken by the autonomous vehicle can be added to the driving information map. Accordingly, this information can be sent to other vehicles in the area that may encounter the same obstacle, assisting them not only in identifying closed lanes but also in knowing how to pass through them.

[0136] In some embodiments, the disclosed method may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of art. Figure 8 A conceptual partial view of an example computer program product arranged according to at least some embodiments shown herein is schematically illustrated. The example computer program product includes a computer program for executing computer processes on a computing device. In one embodiment, the example computer program product 700 is provided using a signal carrying medium 701. The signal carrying medium 701 may include one or more program instructions 702 that, when executed by one or more processors, can provide the above-described instructions for… Figures 2 to 7 The description may include all or part of the functions, or may provide all or part of the functions described in subsequent embodiments. For example, refer to Figure 9 In the embodiment shown, one or more features in S801 to S805 can be fulfilled by one or more instructions associated with the signal carrying medium 701. Furthermore, Figure 8 The program instruction 702 in the document also describes example instructions.

[0137] In some examples, the signal-bearing medium 701 may comprise a computer-readable medium 703, such as, but not limited to, a hard disk drive, a compact disc (CD), a digital video optical disc (DVD), a digital magnetic tape, a memory, read-only memory (ROM), or random access memory (RAM), etc. In some embodiments, the signal-bearing medium 701 may comprise a computer-recordable medium 704, such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, etc. In some embodiments, the signal-bearing medium 701 may comprise a communication medium 705, such as, but not limited to, digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, etc.). Therefore, for example, the signal-bearing medium 701 may be conveyed by a wireless communication medium 705 (e.g., a wireless communication medium conforming to the IEEE 802.11 standard or other transmission protocols). One or more program instructions 702 may be, for example, computer-executable instructions or logical implementation instructions. In some examples, such as for... Figures 2 to 7 The described computing device can be configured to provide various operations, functions, or actions in response to program instructions 702 transmitted to the computing device via one or more of computer-readable media 703, and / or computer-recordable media 704, and / or communication media 705. It should be understood that the arrangements described herein are merely illustrative. Therefore, those skilled in the art will understand that other arrangements and other elements (e.g., machines, interfaces, functions, sequences, and functional groups, etc.) can be used instead, and some elements can be omitted depending on the desired result. Furthermore, many of the described elements are functional entities that can be implemented as discrete or distributed components, or in any suitable combination and location in conjunction with other components.

[0138] In existing technologies, estimating target orientation using target point cloud data considers all points in the data, including those inside the target, and requires the points in the target point cloud to have a rectangular or L-shaped distribution. However, the distribution of points in the target point cloud data may be sparse or change with time or relative to the lidar viewpoint. In this case, the data points inside the target will disrupt the rectangular or L-shaped distribution of the target point cloud data, resulting in low robustness and low accuracy of target orientation estimation in existing technologies. Regarding target orientation, taking a target vehicle as an example, the target orientation is explained as follows: a vector is constructed starting from the rear of the target vehicle and ending at the front; the direction of this vector is the target orientation of the vehicle.

[0139] To improve the accuracy of target shape estimation, embodiments of this application provide a target shape estimation method, such as... Figure 9 As shown, the method includes steps S801-S805:

[0140] S801, Determine the target outer envelope.

[0141] Optionally, a sensor is installed at the observation point. This sensor can be a roadside sensor or a vehicle-mounted sensor, and can be, for example, a camera, LiDAR, or millimeter-wave radar. The technical solution of this application is described below using an example where LiDAR is installed at the observation point. First, the LiDAR detects the surrounding environment and obtains raw point cloud data. Then, the raw point cloud data detected by the LiDAR is preprocessed, that is, the three-dimensional raw point cloud data is converted into two-dimensional point cloud data. Next, clustering methods such as density clustering and mean clustering are used to perform cluster analysis on the two-dimensional point cloud data, resulting in at least one cluster. Finally, the point cloud data in a cluster is determined to be a set of target point cloud data, and a set of target point cloud data corresponds to a target, such as a vehicle. At this point, the point cloud data in the cluster, i.e., the target point cloud data, is two-dimensional point cloud data. The plane containing the two-dimensional point cloud data is the horizontal plane where the LiDAR is located. For each set of target point cloud data, the following processing is performed: Algorithms such as Graham scanning and Jarvis stepping are used to analyze the target point cloud data and determine the target outer envelope corresponding to that target point cloud data. The target outer envelope includes multiple edges, each edge containing a two-dimensional data point corresponding to the point observed at the observation point (i.e., a two-dimensional point corresponding to the three-dimensional point cloud that the radar can observe at the observation point).

[0142] For example, consider a raw point cloud data received by a lidar located at an observation point, containing four data points: A, B, C, and D. A three-dimensional coordinate system is established with the observation point as the origin O, where the horizontal plane where the lidar is located is the xoy plane. The coordinates of the data points in the raw point cloud data are determined within this three-dimensional coordinate system based on their positions. The coordinates of points A, B, C, and D in the raw point cloud data are (1, 2, 1), (1, 1, 2), (1, 2, 0), and (1, 3, 1), respectively. Points A, B, C, and D are then converted to points on the xoy plane. Their two-dimensional coordinates are then determined to be (1, 2), (1, 1), (1, 2), and (1, 3), respectively. Since the coordinates of points A and C coincide, A and C need to be processed, for example, by removing one of their coordinates. Therefore, the preprocessed original point cloud data includes 3 points, namely A (or C), B, and D, with position coordinates of (1, 2), (1, 1), and (1, 3), respectively.

[0143] For example, the target point cloud data is input into the convex hull function ConvexHull() of the open-source computer vision library OpenCV to obtain the target outer envelope corresponding to the target point cloud data. This target outer envelope includes multiple edges. The algorithm used by the convex hull function ConvexHull() is either the Graham scan method or the Jarvis step method, etc.

[0144] For example, taking the Graham scan algorithm used in the convex hull function ConvexHull() as an example, such as... Figure 10 As shown in (a), the observation point with the lidar is set as the origin O. After preprocessing the raw point cloud data observed by the lidar at the origin O, two-dimensional data A is obtained. Density clustering algorithm is used to cluster the data points of two-dimensional data A, resulting in multiple clusters. The two-dimensional point cloud data in a cluster is a set of target point cloud data, and a set of target point cloud data corresponds to a target, such as a vehicle. Specifically, target data B is the two-dimensional data corresponding to target vehicle 1, target data C is the two-dimensional data corresponding to target vehicle 2, and target data D is the two-dimensional data corresponding to target vehicle 3. The target point cloud data B, C, and D of target vehicles 1, 2, and 3 are input into the convex hull function ConvexHull(), respectively, to obtain the target outer envelopes corresponding to target point cloud data B, C, and D, as shown by line segments b, c, and d in the figure. Taking the outer envelope of vehicle 1 as an example... Figure 10 As shown in (a), the outer envelope b of vehicle 1 includes 6 edges.

[0145] Optionally, in another possible implementation, a lidar is installed at the observation point. The lidar detects the surrounding environment and obtains raw point cloud data. First, clustering algorithms such as density clustering and mean clustering are used to cluster the raw point cloud data detected by the lidar, resulting in at least one cluster. Each cluster includes at least one data point from the raw point cloud data. The point cloud data within a cluster constitutes a target point cloud data set, and a set of target point cloud data corresponds to a target, such as a vehicle. At this point, both the target point cloud data and the raw point cloud data are three-dimensional point cloud data. Subsequently, the three-dimensional raw point cloud data is converted into two-dimensional point cloud data. Each set of target point cloud data is processed as follows: First, a two-dimensional convex hull function is used to analyze the target point cloud data, determining its outer envelope on a two-dimensional plane. This target outer envelope is a two-dimensional outer envelope containing multiple edges, each edge including the two-dimensional data point corresponding to the point observed at the observation point.

[0146] It should be noted that the points mentioned in steps S802-S805 below are all two-dimensional data points obtained after preprocessing the original point cloud data, and the target envelope mentioned below refers to the outer envelope on the plane. This is a unified explanation here and will not be repeated below.

[0147] S802. Based on the target envelope, determine at least one visible edge on the target envelope.

[0148] As one possible implementation, S802 can be implemented as follows: determining the visible edges of the target outer envelope based on the vertices on the target outer envelope. The target outer envelope includes multiple edges, among which at least one is a visible edge, and each visible edge includes a point observed by a sensor (e.g., a lidar) at the observation point. The target outer envelope also includes invisible edges. Invisible edges refer to edges containing points that cannot be observed by the sensor at the observation point. The unobservable points mentioned in this embodiment can refer to points in a 3D point cloud or to 2D points transformed from a 3D point cloud. When a point cloud data is unobservable from the observation point, the corresponding 2D point is considered unobservable from the observation point. A description of the vertices on the target outer envelope can be found above and will not be repeated here. Taking an observation point with a camera as an example, the edges on the outer envelope of the target that can be monitored by the camera are the visible edges of the target. That is, the two-dimensional points on the visible edges of the target are observable to the observation point. This also means that the edges on the outer envelope of the target that cannot be monitored by the camera are the invisible edges of the target.

[0149] Specifically, first, determine the maximum and minimum azimuth angles of the vertices on the target's outer envelope relative to the observation point. The difference between the maximum and minimum azimuth angles is the field of view (FOV) of the target's outer envelope. Then, determine the line connecting the vertex corresponding to the maximum azimuth angle and the vertex corresponding to the minimum azimuth angle; this line serves as the dividing line for determining at least one visible edge and at least one invisible edge. Finally, among the multiple edges on the target's outer envelope, if a vertex of any one of the edges is located on the same side of the dividing line as the observation point, then that edge is a visible edge of the target. If a vertex of any one of the edges is located on a different side of the dividing line from the observation point, then that edge is an invisible edge of the target.

[0150] For example, in a two-dimensional coordinate system with the observation point as the origin, the coordinates of the vertices corresponding to the maximum and minimum azimuth angles of the target's outer envelope can be represented by V. max and V min This is represented by the coordinates V of the vertex corresponding to the maximum azimuth angle of the target's outer envelope. max=arg max(atan2(y i x i The coordinates V of the vertex corresponding to the minimum azimuth angle on the outer envelope of the target. min =arg min(atan2(y i x i )). (x i y i ) represents the coordinates of the i-th vertex on the outer envelope of the target, atan2(y i x i ) represents the direction pointing from the origin to the i-th vertex (x) on the outer envelope of the target. i y i The angle between the ray formed between the origin and the target, and the positive x-axis, is the azimuth angle of the i-th vertex on the target's outer envelope relative to the observation point. arg max(atan2(y i x i )) represents V max The value of is atan2(y i x i The coordinates of the vertex at the maximum value, arg min(atan2(y i x i )) represents V min The value of is atan2(y i x i The coordinates of the vertex at the minimum value.

[0151] For example, in Figure 10 In (b), the target envelope includes five edges: L1, L2, L3, L4, and L5. These five edges are connected end-to-end through vertices A, B, C, D, and E. The vertex corresponding to the maximum azimuth angle α is A, and the vertex corresponding to the minimum azimuth angle β is D. Therefore, the line connecting AD can be determined as the dividing line. At least one vertex on the visible edges L1, L2, and L3 of the target envelope is located on the same side of the dividing line as the origin O. At least one vertex on the invisible edges L4 and L5 of the target envelope is located on a different side of the dividing line from the origin O. The field of view angle γ of this target envelope is the difference between the maximum azimuth angle α and the minimum azimuth angle β.

[0152] It should be noted that if the two vertices corresponding to the maximum and minimum azimuth angles of the target's outer envelope are the two endpoints of a certain side of the target's outer envelope, then this side can be used as the dividing line between the visible and invisible edges of the target. If there are other visible edges among the other edges of the target's outer envelope, then the edge serving as the dividing line is an invisible edge of the target's outer envelope, and in this case, there is only one invisible edge. If there are no visible edges among the other edges of the target's outer envelope, but there are invisible edges, then the edge serving as the dividing line is a visible edge of the target's outer envelope, and in this case, there is only one visible edge.

[0153] If there are multiple vertices corresponding to the maximum azimuth angle on the target's outer envelope, then the line connecting the vertex farther from the observation point to the vertex corresponding to the minimum azimuth angle is the dividing line. If there are multiple vertices corresponding to both the maximum and minimum azimuth angles on the target's outer envelope, then the line connecting the vertex farther from the observation point to the vertex corresponding to the maximum azimuth angle is the dividing line.

[0154] It should be noted that by using the target's outer envelope determined by the point cloud data collected by the lidar through the above process, the influence of the point cloud data inside the target on the target orientation estimation can be reduced, thereby improving the robustness of the target orientation estimation.

[0155] S803. Determine one or more initial orientations based on the orientation of at least one visible edge of the target.

[0156] Optionally, before determining the initial orientation, it is also necessary to determine the first target visible edge from at least one target visible edge.

[0157] In one possible implementation, the average length of all visible edges corresponding to the same target envelope is first calculated. Then, based on the calculated average length, at least one visible edge exceeding the average length is identified as the first visible edge. In this case, there may be one or more first visible edges. For example, there are four visible edges corresponding to the same target envelope, namely L1, L2, L3, and L4, with lengths of 9, 5, 6, and 8 respectively, and an average length of 7. Therefore, L1 and L4, whose lengths exceed the average length, are identified as the first visible edges.

[0158] In another possible implementation, the lengths of all visible edges corresponding to the same target envelope are directly determined, and the longest visible edge is identified as the first visible edge. For example, there are four visible edges corresponding to the same target envelope: L1, L2, L3, and L4, with lengths of 9, 5, 6, and 8 respectively. Therefore, the longest visible edge, L1, is identified as the first visible edge.

[0159] Optionally, after determining the first visible edge of the target, the two endpoints of the first visible edge are analyzed according to a first preset algorithm to determine the orientation of the first visible edge as its initial orientation. As described above, for the same target envelope, there may be one or more first visible edges. The first preset algorithm can be... S represents a set of one or more initial orientations. Indicates the orientation of the visible edge of the first target. Indicates the orientation of the visible edge of the first target. The initial orientation θ corresponding to the visible edge of the first target i Establish a two-dimensional coordinate system with the observation point as the origin. and Let represent the coordinates of the two endpoints of the first visible edge in the two-dimensional coordinate system, n be the number of the first visible edges, and i be an integer greater than 0.

[0160] For example, consider two visible edges corresponding to the same target's outer envelope, L1 and L2. In this case, n = 2, and i = 1 or 2. The coordinates of the endpoints of L1 are... and The initial orientation corresponding to L1 is The orientation of L1 will be used as the initial orientation of L1. The coordinates of the endpoints of L2 are respectively... and The initial orientation corresponding to L2 is The orientation of L2 will be used as the initial orientation corresponding to L2.

[0161] For example, such as Figure 11 As shown in (a), the two endpoints of the visible edge L1 of the first target are A and B, and the coordinates of A and B are respectively... and The orientation of L1 is θ1, which can be determined based on the coordinates of A and B and the first preset algorithm, i.e. To determine this. Optionally, the orientation of L1 can also be θ2 = π - θ1, where θ1 can be determined based on the coordinates of A and B and the first preset algorithm. In this case, the first preset algorithm is...

[0162] As can be seen from the above example, in determining the initial orientation, the initial orientation can be either θ or θ-π. Therefore, there may be a problem of ambiguity in the initial orientation. Optionally, either θ or θ-π can be selected as the initial orientation to perform the following step S804 to determine the target orientation.

[0163] It should be noted that by estimating the target orientation based on the visible edge information on the target's outer envelope through the above process, the influence of the point cloud data inside the target on the target orientation estimation can be reduced, thus improving the accuracy of the target orientation estimation. Furthermore, by selecting the orientation of the first visible edge of the target as the initial orientation based on the visible edge information on the target's outer envelope, and estimating the target orientation based on the initial orientation, the feasible region of the target orientation can be constrained. This improves the accuracy of determining the target orientation while reducing the computational complexity of the target orientation estimation process. For an introduction to the feasible region of the target orientation, please refer to the following content, which will not be elaborated upon here.

[0164] S804. Determine the target orientation based on one or more initial orientations.

[0165] Optionally, within a first preset angle range corresponding to the first visible edge of the first target, the orientation of the first visible edge of the first target, i.e., the initial orientation corresponding to the first visible edge of the first target, is rotated by different angles to obtain multiple candidate target orientations corresponding to the first visible edge of the first target. By processing the orientation of each first visible edge of the first target in the same manner as described above, all candidate target orientations corresponding to the first visible edge of the first target can be obtained. For an introduction to the first visible edge of the first target, please refer to step S802 above, which will not be repeated here.

[0166] In one possible implementation, the first preset angle range is [θ]. i -△θ,θ i +△θ], where θ i The orientation of the first visible edge of the target is the initial orientation, where Δθ is set by the user or predetermined by the system. After determining the first preset angle range corresponding to each first visible edge of the target, the orientation of each first visible edge of the target is rotated by different angles at preset angle intervals to obtain multiple candidate target orientations corresponding to each first visible edge of the target.

[0167] For example, such as Figure 11As shown in (b), the orientation of the visible edge L1 of the first target is σ. If the first preset angle range corresponding to L1 is [σ-△δ, σ+△δ], the preset angle interval is △σ, and △δ=2△σ, then the candidate target orientations corresponding to L1 include α=σ-2△σ, β=σ-△σ, σ, χ=σ+△σ, and λ=σ+2△σ. If the orientation of L1 is σ=2°, the preset angle interval is △σ=0.5°, and △δ=2△σ=1°, then the first preset angle range is [1°, 3°], and the candidate target orientations corresponding to L1 include 1°, 1.5°, 2°, 2.5°, and 3°.

[0168] In another possible implementation, the first preset angle range [θ] corresponding to each visible edge of the first target is determined. i -△θ,θ i After adding [Δθ], the union of these first preset angle ranges is taken to obtain the range of candidate target orientation values. The candidate target orientation is determined using preset angle intervals. Regarding [θ]... i -△θ,θ i The explanation of +△θ] can be found above and will not be repeated here. Since the target orientation is determined from the candidate target orientations, the range of values ​​for the candidate target orientations is also the range of values ​​for the target orientation, which can also be called the feasible region of the target orientation.

[0169] For example, there are two visible edges corresponding to the outer envelope of the same target, namely L1 and L2, with orientations of 2° and 3° respectively, i.e., θ1 = 2° and θ2 = 3°. If Δθ = 1°, then the first preset angle range corresponding to L1 is [1°, 3°], and the first preset angle range corresponding to L2 is [2°, 4°]. Taking the union of the first preset angle ranges corresponding to L1 and L2 yields the feasible region of target orientation, which is the range of candidate target orientation values ​​[1°, 4°]. If the angles in [1°, 4°] are determined as candidate target orientations at preset angle intervals of 0.5°, then the set of candidate target orientations is {1°, 1.5°, 2°, 2.5°, 3°, 3.5°, 4°}.

[0170] In another possible implementation, a first preset angle range corresponding to the visible edge of the first target is determined. Within this first preset angle range, the angular interval of the first initial orientation rotation can be random, but the number of candidate target orientations obtained after the initial orientation rotation is fixed. The number of candidate target orientations can be preset or set by the user.

[0171] For example, the orientation of the first visible edge L1, i.e., the initial orientation corresponding to the first visible edge L1, is 2°, and the first preset angle range corresponding to L1 is [1°, 3°]. Taking the rotation angle interval of the initial orientation corresponding to the first visible edge as random, and taking 11 candidate target orientations corresponding to the first visible edge L1 as an example, the set of candidate target orientations can be {1°, 1.3°, 1.4°, 1.5°, 1.6°, 1.8°, 2.1°, 2.4°, 2.7°, 2.9°, 3°}.

[0172] Optionally, for each visible edge of the first target, one or more points on each visible edge of the first target are determined as interpolation points. Then, one or more target distances corresponding to these interpolation points are determined, and these multiple target distances are summed to obtain the sum of target distances corresponding to the candidate target orientation. The target distance corresponding to an interpolation point is the minimum distance between the interpolation point and two adjacent edges of the first rectangle, which is obtained by rectangle fitting of the candidate target orientations. The specific process of rectangle fitting can be found in step S805 below, and will not be elaborated here. Finally, the candidate target orientation corresponding to the minimum sum of target distances is determined as the first target orientation, and this first target orientation is determined as the target orientation.

[0173] For example, consider five candidate target orientations: A, B, C, D, and E. There are two visible edges corresponding to the outer envelope of the same target: L1 and L2. L1 has one interpolation point a, and L2 has two interpolation points b and c. The target distances of interpolation points a, b, and c to candidate target orientation A are Aa, Ab, and Ac, respectively; the target distances of interpolation points a, b, and c to candidate target orientation B are Ba, Bb, and Bc, respectively; the target distances of interpolation points a, b, and c to candidate target orientation C are Ca, Cb, and Cc, respectively; the target distances of interpolation points a, b, and c to candidate target orientation D are Da, Db, and Dc, respectively; and the target distances of interpolation points a, b, and c to candidate target orientation E are Ea, Eb, and Ec. Aa + Ab + Ac > Ca + Cb + Cc > Da + Db + Dc = Ba + Bb + Bc > Ea + Eb + Ec. Therefore, candidate target orientation E is the target orientation.

[0174] In one possible implementation, the interpolation points on the first visible edge of the target are determined at equal intervals. For example, if the length of the first visible edge of the target is 1, and the interpolation points are determined at preset intervals of 0.2, then there are 4 interpolation points on the first visible edge of the target, located at distances of 0.2, 0.4, 0.6, and 0.8 from the endpoints of the first visible edge of the target, respectively. If the length of the first visible edge of the target is 1.1, and the interpolation points are determined at preset intervals of 0.2, then there are 5 interpolation points on the first visible edge of the target, located at distances of 0.2, 0.4, 0.6, 0.8, and 1 from the endpoints of the first visible edge of the target, respectively.

[0175] In another possible implementation, the number of interpolation points on the first visible edge of the target is predetermined. In this case, the interpolation points on the same first visible edge are determined at equal intervals, or they can be randomly determined. For example, there are two first visible edges, L1 and L2. L1 has a length of 1, and L2 has a length of 2. If the number of interpolation points on the first visible edge of the target is determined to be 4, and the interpolation points are determined at equal intervals, then for L1, these 4 interpolation points are located at distances of 0.2, 0.4, 0.6, and 0.8 from the endpoint of L1, respectively. For L2, these 4 interpolation points are located at distances of 0.4, 0.8, 1.2, and 1.6 from the endpoint of L2, respectively. Alternatively, these 4 interpolation points are randomly distributed on the first visible edge of the target; for example, for L1, these 4 interpolation points are located at distances of 0.1, 0.5, 0.7, and 0.8 from the endpoint of L1, respectively. For L2, these four interpolation points are located at distances of 0.5, 0.7, 1.1, and 1.8 from the endpoint of L2, respectively.

[0176] In one possible implementation, after determining the interpolation points on all visible edges of the first target, the position of each interpolation point is further determined. For each candidate target orientation, the following operations are performed: the projection values ​​of each interpolation point onto the two direction vectors corresponding to the candidate target orientation are calculated, and the maximum and minimum projection values ​​of all interpolation points onto these two direction vectors are determined. The two direction vectors corresponding to the candidate target orientation are the direction vectors corresponding to the two mutually perpendicular sides of the rectangle used to fit the candidate target orientation; these two direction vectors are obtained through analysis of the candidate target orientation. By combining the maximum and minimum projection values ​​of all interpolation points onto the aforementioned two direction vectors, the projection values ​​of each interpolation point onto these two direction vectors are calculated to determine the distance between each interpolation point and the two vectors corresponding to the candidate target orientation, which is the distance between each interpolation point and the two mutually perpendicular sides of the fitted rectangle of the candidate target orientation. The minimum distance between each interpolation point and the two vectors corresponding to the candidate target orientation is the target distance between the interpolation point and the candidate target orientation. Sum the target distances corresponding to the candidate target orientations to obtain the sum of the target distances corresponding to the candidate target orientations. Determine the candidate target orientation corresponding to the smallest sum of target distances as the first target orientation, and determine the first target, North Korea, as the desired target orientation.

[0177] For example, the interpolation point P on the visible edge of the first target i The position can be represented by (x) i y i ) means, (x i y i ) is P i The coordinates in a two-dimensional coordinate system with the observation point as the origin. For the candidate target orientation α, the direction vectors corresponding to the two mutually perpendicular sides of the rectangle used to fit the candidate target orientation are e1 and e2, respectively. Here, e1 and e2 are determined according to the angle of the candidate target orientation, i.e., α, where e1 = [cos(α), sin(α)] and e2 = [-sin(α), cos(α)]. The interpolation point P on the visible side of the first target. i The maximum projection value in the e1 direction is Minimum projection value is Among them, P i T The interpolation point P on the visible edge of the first target i The transpose of the matrix, P i The matrix is ​​[x i y i ]. P i The maximum projection value in the e2 direction is Minimum projection value is P iThe projection values ​​in the directions e1 and e2 are c1 and c2, respectively, where, according to And c1 = P i T By analyzing e1, P can be determined. i The distance to the direction vector e1 corresponding to the candidate target's orientation α is according to And c2 = P i T By analyzing e2, P can be determined. i The distance to the direction vector e2 corresponding to the candidate target's orientation α is Interpolation point P i The distance to the target being oriented towards the candidate target is d0, where d0 is... and The minimum value in can be used This indicates that the sum of the distances from the same candidate target to its corresponding target is d. α According to θ = argmin α d α The minimum sum of target distances d is determined by filtering all candidate targets by their corresponding target distances. α The corresponding candidate target orientation is the target orientation θ.

[0178] For example, let's take the case where there is only one visible edge corresponding to the outer envelope of the same target, namely L1. Figure 12As shown, there are four interpolation points on the visible edge L1 of the first target, including the endpoints of L1. These four interpolation points are a, b, c, and d. The orientation of the visible edge L1 of the first target, i.e., the initial orientation corresponding to L1, is θ1. Generally, according to the implementation method given in the above example, multiple candidate target orientations corresponding to L1 can be determined. For example, there are three candidate target orientations, including θ2, θ3, and the orientation θ1 of L1. To more clearly describe the technical solution of this application, this example takes the candidate target orientation θ1 as an example to explain in detail how to determine the target distance corresponding to the candidate target orientation. Based on the angle θ1 of the candidate target orientation, two direction vectors e1 and e2 corresponding to the candidate target orientation θ1 can be determined, where e1 = [cos(θ1), sin(θ1)], e2 = [-sin(θ1), cos(θ1)]. Based on the coordinates of e1, e2, and interpolation points a, b, c, and d, the projection values ​​of these four interpolation points on e1 and e2 can be determined. The projection values ​​of interpolation point a in the directions of e1 and e2 are l1 and l2, respectively; the projection values ​​of interpolation point b in the directions of e1 and e2 are l3 and l4, respectively; the projection values ​​of interpolation point c in the directions of e1 and e2 are l5 and l6, respectively; and the projection values ​​of interpolation point d in the directions of e1 and e2 are l7 and l8, respectively. By comparing l1, l3, l5, and l7, the maximum projection value of the interpolation point in the direction of e1 is l1, and the minimum projection value is l7; the maximum projection value of the interpolation point in the direction of e2 is l2, and the minimum projection value is l8. The distances of interpolation point a to the direction vectors e1 and e2 corresponding to the candidate target orientation θ1 are respectively... Therefore, the distance between interpolation point a and the candidate target facing θ1 is 0. The distances between interpolation point b and the direction vectors e1 and e2 corresponding to the candidate target facing θ1 are respectively... Therefore, the distance between interpolation point b and the candidate target facing θ1 is l1-l3. The distances between interpolation point c and the direction vectors e1 and e2 corresponding to the candidate target facing θ1 are respectively... Therefore, the distance between interpolation point c and the candidate target facing θ1 is l6-l8. The distances between interpolation point d and the direction vectors e1 and e2 corresponding to the candidate target facing θ1 are respectively... Therefore, the target distance between the interpolation point d and the candidate target orientation θ1 is 0. In summary, the sum of the target distances corresponding to the candidate target orientation θ1 is M1 = l1 - l3 + l6 - l8. The sum of the target distances corresponding to the candidate target orientations θ2 and θ3 can be determined in the same way, and the sums of the target distances corresponding to θ2 and θ3 are M2 and M3, respectively. If M1 > M2 > M3, then the candidate target orientation θ3 corresponding to M3 is determined to be the desired target orientation θ.

[0179] For example, consider two visible edges corresponding to the same target envelope, L1 and L2. Edge L1 has three interpolation points, including its endpoints: a, b, and c. Edge L2 has three interpolation points, including its endpoints: d, e, and f. The orientation of edge L1, i.e., its initial orientation, is θ1. There are three candidate target orientations for L1, including its initial orientation: θ1, θ2, and θ3. The orientation of edge L2, i.e., its initial orientation, is θ4. There are three candidate target orientations for L2, including its initial orientation: θ4, θ5, and θ6. The direction vectors corresponding to θ1 are e1 and e2, those corresponding to θ2 are e3 and e4, those corresponding to θ3 are e5 and e6, those corresponding to θ4 are e7 and e8, and those corresponding to θ5 are e9 and e6. 10 The direction vector corresponding to θ6 is e 11 and e 12 The maximum and minimum projection values ​​of interpolation points a, b, c, d, e, f in directions e1 and e2 are t1 and t2, respectively; the maximum and minimum projection values ​​of interpolation points a, b, c, d, e, f in directions e3 and e4 are t3 and t4, respectively; the maximum and minimum projection values ​​of interpolation points a, b, c, d, e, f in directions e5 and e6 are t5 and t6, respectively; the maximum and minimum projection values ​​of interpolation points a, b, c, d, e, f in directions e7 and e8 are t7 and t8, respectively; and the maximum and minimum projection values ​​of interpolation points a, b, c, d, e, f in directions e9 and e2 are t1 and t2, respectively. 10 The maximum and minimum projection values ​​in the direction are t9 and t 10 Interpolation points a, b, c, d, e, f are at e 11 and e 12 The maximum and minimum projection values ​​in the direction are t 11 and t 12 Based on the projection values ​​of interpolation points a, b, c, d, e, and f onto e1 and e2, and t1 and t2, determine the target distances between interpolation points a, b, c, d, e, and f and the candidate target orientation θ1, respectively. Sum these six target distances to obtain the sum of target distances S1 corresponding to candidate target orientation θ1. Similarly, determine the sums of target distances S2, S3, S4, S5, and S6 corresponding to candidate target orientations θ2, θ3, θ4, θ5, and θ6, respectively. If S1 > S2 > S3 > S4 > S5 > S6, then the candidate target orientation θ6 corresponding to the smallest sum of target distances S6 is determined as the desired target orientation θ.

[0180] Optionally, in another possible implementation, after determining the target distance corresponding to the interpolation point, multiple target distances corresponding to the same candidate target orientation are determined. Then, the variance of these multiple target distances is calculated to obtain the target distance distribution variance corresponding to the candidate target orientation. Finally, among the multiple candidate target orientations corresponding to each visible edge of the first target (i.e., all candidate target orientations), the candidate target orientation corresponding to the smallest target distance distribution variance is determined as the first target orientation, and this first target orientation is determined as the desired target orientation.

[0181] It should be noted that, through the above process, multiple candidate target orientations are determined based on the initial orientation on the visible edge of the first target. These candidate orientations are then filtered using interpolation points on the visible edge of the first target to determine the final target orientation. Narrowing the range of target orientation values ​​using the orientation of the visible edge of the first target and filtering from multiple candidate orientations improves the accuracy of target orientation determination. Furthermore, using interpolation points on the visible edge of the first target to filter candidate orientations reduces the influence of data points within the target on the target orientation estimation, further improving the robustness and accuracy of the target orientation estimation.

[0182] Optionally, in another possible implementation, the interpolation points on the first visible edge of the target may also include vertices on that first visible edge, that is, points in the target point cloud data located on that first visible edge. In this case, the target orientation determined based on the interpolation points and the candidate target orientation is more accurate.

[0183] Alternatively, in another possible implementation, the interpolation points on the first visible edge of the target may include only the points in the target point cloud data located on the first visible edge of the target.

[0184] S805. Perform rectangle fitting based on the target orientation to determine the target shape information.

[0185] The target shape information includes the target's location and shape.

[0186] Optionally, after determining the target orientation, the second preset algorithm L1 = max(P) is used. i T ·e1)-min(P i T ·e1), L2=max(P i T ·e2)-min(P i T ·e2), V1=[min(P i T ·e1),min(P i T·e2)]·A,V2=[max(P i T ·e1),min(P i T ·e2)]·A,V3=[max(P i T ·e1),max(P i T ·e2)]·A,V4=[min(P i T ·e1),max(P i T ·e2)]·A, e1 = [cos(θ), sin(θ)], e2 = [-sin(θ), cos(θ)], and a rectangle is fitted to the target orientation to determine the target shape information. In the second preset algorithm, L1 and L2 are the lengths of two mutually perpendicular sides in the fitted rectangle corresponding to the target orientation. These two side lengths are based on the maximum and minimum projection values ​​of the interpolation points on the visible side of the first target in the e1 direction, the maximum and minimum projection values ​​of the interpolation points on the visible side of the first target in the e2 direction, and L1 = max(P i T ·e1)-min(P i T ·e1) and L2=max(P i T ·e2)-min(P i T ·e2) to determine. P i T Let L1 be the transpose of the matrix representing the interpolation points on the visible edge of the target, and let e1 and e2 represent the direction vectors corresponding to L1 and L2, respectively. e1 and e2 are determined based on the target orientation θ, and e1 = [cos(θ), sin(θ)] and e2 = [-sin(θ), cos(θ)]. V1, V2, V3, and V4 are the vertices of the rectangle. These four vertices are determined based on the maximum and minimum projection values ​​of the interpolation points on the first visible edge of the target in the e1 direction, the maximum and minimum projection values ​​of the interpolation points on the first visible edge of the target in the e2 direction, and V1 = [min(P...]. i T ·e1),min(P i T ·e2)]·A,V2=[max(P i T ·e1),min(P i T ·e2)]·A,V3=[max(P i T·e1),max(P i T ·e2)]·A,V4=[min(P i T ·e1),max(P i T ·e2)]·A, The transformation matrix is ​​determined based on the target orientation θ.

[0187] For example, such as Figure 13 As shown in (a), the origin O is the observation point where the lidar is installed. The polygon formed by the line segments connected end to end in the figure is the target outer envelope determined based on a set of target point cloud data. The points in this target outer envelope are the target point cloud data used to determine the target outer envelope. L1 is the visible edge on the target outer envelope, and the vertices of L1 are A and B. Taking the determination of the orientation of the first visible edge L1 of the target as the target orientation after the above steps S801-S803 as an example, the target shape information is obtained by calculating based on the orientation of L1, the interpolation points on L1, and the second preset algorithm. Figure 13 As shown in (b). Figure 13 In (b), quadrilateral CDEF represents the outline of the target. CDEF can be a rectangle or a regular parallelogram. The points within quadrilateral CDEF are a set of target point cloud data obtained by processing the raw point cloud data collected by the lidar. The side lengths of the quadrilateral are L2, L3, L4, and L5, and the vertices are C, D, E, and F. Where L2 = L3 and L4 = L5.

[0188] In another possible implementation, during the process of determining the target shape information using the second preset algorithm, it is possible to first directly determine the target shape information based on V1 = [min(P i T ·e1),min(P i T ·e2)]A,V2=[max(P i T ·e1),min(P i T ·e2)]·A,V3=[max(P i T ·e1),max(P i T ·e2)]·A,V4=[min(P i T ·e1),max(P i T ·e2)]·A, e1 = [cos(θ), sin(θ)], e2 = [-sin(θ), cos(θ)] are used to determine the coordinates of the four vertices in the target contour, and then the side length information in the target contour is determined based on the obtained coordinates of the four vertices.

[0189] In one possible implementation, after determining the target visible edge in step S801, the target orientation can be determined directly based on the orientation of the target visible edge. Specifically, if there is only one target visible edge, the target orientation is determined to be the orientation of that target visible edge, and then step S804 is executed. Alternatively, if there are multiple target visible edges, the target orientation can be determined to be the average of the orientations of these multiple target visible edges, and then step S804 is executed. In this case, the points in the target point cloud data located on the target visible edge can be used as P. i The target's shape information is determined by performing rectangle fitting on the target's orientation using a second preset algorithm. In this process, the determination of the target's shape information primarily relies on the target's visible edge information, i.e., points located on the target's outer envelope in the target point cloud data, rather than points located inside the target in the target point cloud data. Therefore, compared to existing technologies, the technical solution of this embodiment can reduce the influence of points inside the target on determining the target's orientation and other target shape information, thus improving the robustness of the target orientation determination.

[0190] In another possible implementation, after determining the visible edge of the target in step S801, the target orientation can be directly determined based on the orientation of the visible edge. Specifically, after step S801, step S803 is executed, and one of the candidate target orientations is determined as the target orientation based on the candidate target orientations corresponding to all visible edges. Subsequently, step S805 is executed to perform rectangle fitting on the target orientation to determine the target shape information. In the above process, the determination of the target shape information mainly depends on the visible edge information, that is, the points located on the outer envelope of the target in the target point cloud data, and does not depend on the points located inside the target in the target point cloud data. Therefore, compared with the prior art, the technical solution of this embodiment can reduce the influence of points inside the target on the determination of target orientation and other target shape information, and improve the robustness of the target orientation determination. After determining the visible edge of the target, multiple candidate target orientations are determined based on the orientation of the visible edge of the target, and one of the candidate target orientations is determined as the target orientation, which can further optimize the target orientation and improve the target orientation estimation accuracy.

[0191] Through the above process, after acquiring target point cloud data at the observation point, at least one visible edge of the target on the target's outer envelope, determined based on the target point cloud data, is first identified. Then, based on the orientation of this visible edge, one or more initial orientations are determined. These initial orientations are then optimized to obtain the target orientation. Finally, a rectangle fit is performed on the target orientation to determine the target shape information. First, using edge point data from the target point cloud data to determine the target's outer envelope and relying on the visible edge information on the outer envelope for target orientation estimation reduces the influence of data points inside the target on the estimation, improving its robustness. Second, filtering the visible edges on the target's outer envelope to obtain the first visible edge, and determining the initial orientation based on this first visible edge, reduces computational complexity. Third, using the visible edge information to narrow down the range of target orientation values ​​for estimation reduces computational complexity and improves accuracy. Finally, optimizing these initial orientations to obtain the target orientation further improves the accuracy of target orientation estimation and the accuracy of determining target shape information.

[0192] It should be noted that, as shown in the example in step S803 above, the initial orientation cannot be distinguished between θ and θ-π during the determination of the initial orientation, thus leading to an issue of initial orientation ambiguity. To address this problem, this application proposes a possible implementation method: selecting any one of θ and θ-π as the initial orientation, and proceeding to step S804 to determine the first target orientation as the target orientation. Specifically, the process of determining the first target orientation as the target orientation is as follows: acquiring target tracking information, which includes the target velocity direction; and determining the target orientation as the one with the smallest difference from the target velocity direction among one or more preset target orientations corresponding to the first target orientation. Here, the preset target orientations are θ and θ-π, and the target velocity direction is the velocity direction of the target object before the current moment.

[0193] For example, if the target velocity direction is α, and the difference between θ and α is greater than the difference between θ-π and α, then the preset target orientation θ-π is determined to be the target orientation. If the difference between θ and α is less than or equal to the difference between θ-π and α, then the target orientation is still determined to be θ.

[0194] Optionally, in one possible implementation, the target tracking information includes one or more target velocity directions of the target object prior to the current time. The mean of these one or more target velocity directions is determined, and then, among one or more preset target directions corresponding to the first target direction, the preset target direction with the smallest difference from the obtained mean is determined as the target direction. Here, the preset target directions are θ and θ-π, and the preset time period is located prior to the current time; it can be pre-set or user-defined.

[0195] For example, there are two target velocity directions within a preset time period, α1 and α2. The average of α1 and α2 is calculated, resulting in α = (α1 + α2) / 2. If the difference between θ and α is greater than the difference between θ-π and α, then the preset target orientation θ-π is determined to be the target orientation. If the difference between θ and α is less than or equal to the difference between θ-π and α, then the target orientation remains the preset target orientation θ.

[0196] Optionally, in another possible implementation, the target tracking information also includes the target's historical trajectory within a preset time period. First, based on the target's historical trajectory within the preset time period, multiple target historical trajectory directions are determined. Then, the average of these multiple target historical trajectory directions is determined, and the difference between each preset target orientation corresponding to the first target orientation θ (i.e., θ and θ-π) and the average is determined. The preset target orientation corresponding to the smallest difference is determined as the target orientation. Here, the target historical trajectory is a line connecting the coordinates of the target object's position before the current moment. The volume of the target object can be ignored, and the coordinates of the target object's center position are taken as the coordinates of the target object's current position.

[0197] For example, if the current time is t, based on the target's historical trajectory within a preset time period, the coordinates of the target object at times t-1 and t-2 are determined as (x, y, y) respectively. t-1 y t-1 ) and (x t-2 y t-2 ), and the coordinates of the target object at the current time t are (x, y). t y t Therefore, the two historical trajectory directions of the target object within a preset time period can be determined, and these two historical trajectory directions are β1 = atan2(y). t-1 -y t-2 x t-1 -x t-2 ) and β2=atan2(y t -y t-1 x t -x t-1The average of the historical trajectories of these two targets is calculated to obtain β = (β1 + β2) / 2. The target orientation is θ, and the preset target orientations are θ and θ-π. The differences between θ and θ-π are calculated to obtain α1 and α2. If |α1| > |α2|, then the preset target orientation θ-π is determined to be the target orientation; if |α1| ≤ |α2|, then the target orientation remains θ.

[0198] To address the aforementioned problem of ambiguous initial orientation, this application also provides a possible implementation method: based on target tracking information within a preset time period, determine one of θ and θ-π as the initial orientation, and then execute steps S804-S805 to determine the target orientation and target shape information.

[0199] Furthermore, when the observation line of the lidar's observation point is approximately perpendicular or parallel to the visible edge corresponding to the initial orientation θ, the initial orientation cannot be distinguished at the four angles θ, θ±π / 2, and θ-π, resulting in an ambiguous initial orientation. If the angle between the visible edge of the target corresponding to the initial orientation and the connecting line is approximately 0° or 90°, and the connecting line is the line connecting the observation point and the midpoint of the visible edge of the target corresponding to the target orientation, then it is determined that the observation line of the lidar's observation point is approximately perpendicular or parallel to the visible edge corresponding to the initial orientation. To solve the above problem, this application also proposes an embodiment in which any one of θ, θ±π / 2, and θ-π is selected as the initial orientation, and step S804 is performed to determine the target orientation. Determining the first target orientation as the target orientation specifically includes: determining the target orientation from one or more preset target orientations corresponding to the first target orientation based on target tracking information. The preset target orientations are θ, θ±π / 2, and θ-π, etc. For a detailed description of this embodiment, please refer to the above method for determining the target orientation from θ and θ-π, which will not be repeated here.

[0200] Of course, this application can also determine one of θ, θ±π / 2 and θ-π as the initial orientation based on the target tracking information within a preset time period, and then execute steps S804-S805 to determine the target orientation and target shape information.

[0201] In this application embodiment, the target shape estimation device can be divided into functional modules according to the above method example. When each functional module is divided according to its corresponding function, Figure 14 A possible structural schematic diagram of the target shape estimation involved in the above embodiments is shown. For example... Figure 14 As shown, the target shape estimation device includes a processing unit 1401 and a determining unit 1402. Of course, the target shape estimation device may also include other modules, or the target shape estimation device may include fewer modules.

[0202] The processing unit 1401 is used to determine at least one visible edge of the target on the target outer envelope based on the target outer envelope, wherein the target outer envelope is determined by the target point cloud data, and each visible edge of the target includes points that can be observed by the sensor at the observation point.

[0203] Specifically, processing unit 1401 is used to determine a dividing line between at least one visible edge and at least one invisible edge of the target based on the target's outer envelope, so as to determine at least one visible edge of the target from among multiple edges on the target's outer envelope based on the dividing line. The dividing line is the line connecting the vertex corresponding to the maximum azimuth angle and the vertex corresponding to the minimum azimuth angle in the target's outer envelope. Invisible edges include points that cannot be observed by the sensor at the observation point. Among the multiple edges, if at least one vertex of a certain edge is located on a different side of the dividing line from the observation point, then that edge is an invisible edge. Among the multiple edges, if at least one vertex of a certain edge is located on the same side of the dividing line as the observation point, then that edge is a visible edge.

[0204] The processing unit 1401 is also configured to determine one or more initial orientations based on the orientation of at least one target visible edge.

[0205] Specifically, the processing unit 1401 is further configured to, for each of the at least one target visible edges, analyze the two endpoints of the first target visible edge according to a first preset algorithm, and determine the orientation of the first target visible edge as the initial orientation corresponding to the first target visible edge. Wherein, the first target visible edge is either a target visible edge whose length exceeds the average length among the at least one target visible edges, or the first target visible edge is the longest target visible edge among the at least one target visible edges.

[0206] The first preset algorithm is: Where S represents a set of one or more initial orientations, Indicates the orientation of the visible edge of the first target. Indicates the orientation of the visible edge of the first target. Let θ be the initial orientation corresponding to the visible edge of the first target. i . and Let n and i represent the coordinates of the two endpoints of the visible edge of the first target in a two-dimensional coordinate system with the observation point as the origin, n being the number of visible edges of the first target, and i being an integer greater than 0.

[0207] The processing unit 1401 is also configured to determine the target orientation based on the one or more initial orientations.

[0208] Specifically, the processing unit 1401 is further configured to, for each visible edge of the first target, rotate the initial orientation corresponding to the visible edge of the first target by different angles within a first preset angle range to obtain multiple candidate target orientations corresponding to the visible edge of the first target. Then, for each candidate target orientation, one or more points on the visible edge of the first target are determined as interpolation points, and then one or more target distances corresponding to these interpolation points are determined. The target distance corresponding to the interpolation point is the minimum distance between the interpolation point and two adjacent sides of the first rectangle, and the first shape is obtained by rectangle fitting of the candidate target orientations. The multiple target distances are summed to determine the sum of target distances corresponding to the candidate target orientations. Among the multiple candidate target orientations corresponding to each visible edge of the first target, the candidate target orientation corresponding to the minimum sum of target distances is determined as the first target orientation, and this first target orientation is determined as the desired target orientation.

[0209] Alternatively, specifically, processing unit 1401 is further configured to, for each visible edge of the first target, rotate the initial orientation corresponding to the visible edge of the first target by different angles within a first preset angle range to obtain multiple candidate target orientations corresponding to the visible edge of the first target. Then, for each candidate target orientation, one or more points on the visible edge of the first target are determined as interpolation points, and one or more target distances corresponding to these interpolation points are determined. The target distance corresponding to the interpolation point is the minimum distance between the interpolation point and two adjacent sides of a first rectangle, which is obtained by rectangle fitting of the candidate target orientations. The variance of the obtained multiple target distances is calculated to determine the target distance distribution variance corresponding to the candidate target orientation. Among the multiple candidate target orientations corresponding to each visible edge of the first target, the candidate target orientation corresponding to the smallest target distance distribution variance is determined as the first target orientation, and this first target orientation is determined as the desired target orientation.

[0210] After determining the target orientation, the determining unit 1402 is used to determine the preset target orientation with the smallest difference from the target velocity direction among one or more preset target orientations corresponding to the first target orientation.

[0211] In another possible implementation, the determining unit 1402 is used to determine the average value of the historical trajectory direction of the target within a preset time period, and then calculate the difference between each preset target orientation corresponding to the target orientation and the average value, and redetermine the preset target orientation corresponding to the smallest difference as the target orientation.

[0212] Finally, the processing unit 1401 is also used to perform rectangle fitting based on the target orientation to determine the target shape information.

[0213] Specifically, the processing unit 1401 is also used to perform rectangle fitting on the target orientation using a second preset algorithm to determine the target shape information. The second preset algorithm is L1 = max(P i T ·e1)-min(P i T ·e1), L2=max(P i T ·e2)-min(P i T ·e2), V1=[min(P i T ·e1),min(P i T ·e2)]·A,V2=[max(P i T ·e1),min(P i T ·e2)]·A,V3=[max(P i T ·e1),max(P i T ·e2)]·A,V4=[min(P i T ·e1),max(P i T ·e2)]·A, e1 = [cos(θ), sin(θ)], e2 = [-sin(θ), cos(θ)]. L1 and L2 are the lengths of the two perpendicular sides of the fitted rectangle corresponding to the target orientation, respectively. P i T Let θ be the transpose of the matrix of interpolation points on the visible edge of the target corresponding to the target orientation, e1 and e2 represent the direction vectors corresponding to L1 and L2 respectively, V1, V2, V3 and V4 are the vertices of the rectangle, A is the transformation matrix, and θ is the target orientation.

[0214] Reference Figure 15 As shown, this application also provides a target shape estimation device, which includes a memory 1501, a processor 1502, a communication interface 1503, and a bus 1504. The processor 1502 is used to manage and control the operation of the device, and / or to execute other processes of the techniques described in the text. The communication interface 1503 is used to support communication between the device and other network entities. The memory 1501 is used to store the device's program code and data.

[0215] The processor 1502 (or described as a controller) described herein can implement or execute various exemplary logic blocks, unit modules, and circuits as described in conjunction with the disclosure of this application. The processor or controller can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, unit modules, and circuits as described in conjunction with the disclosure of this application. The processor 1502 can also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0216] Communication interface 1503 can be a transceiver circuit.

[0217] The memory 1501 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk or solid-state drive; the memory may also include combinations of the above types of memory.

[0218] The 1504 bus can be an extended industry standard architecture (EISA) bus, etc. The 1504 bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 15 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0219] The specific working process of the server and device described above can be referred to the corresponding process in the following method embodiments, and will not be repeated here.

[0220] This application provides a computer-readable storage medium for storing one or more programs, the one or more programs including instructions that, when executed by a computer, cause the computer to perform the target shape estimation method described in steps S801-S805 of the above embodiments.

[0221] This application also provides a computer program product containing instructions that, when executed on a computer, cause the computer to perform the method executed by the target shape estimation device in steps S801-S805 of the above embodiments.

[0222] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0223] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented using software programs, the above embodiments can appear, in whole or in part, as a computer program product, which includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of this application are generated.

[0224] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. This available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0225] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0226] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0227] The units described as separate components may or may not be physically separate. A component shown as a unit may be one or more physical units, meaning it may be located in one place or distributed in multiple different locations. In application, some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0228] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0229] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, or parts of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a device (which may be a personal computer, server, network device, microcontroller, or chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0230] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be covered within the scope of protection of this application.

Claims

1. A target shape estimation method, characterized in that, include: Based on the target envelope, at least one visible edge on the target envelope is determined; The target envelope is determined by the target point cloud data; each visible edge of the target includes points in the target point cloud data that can be observed by the sensor at the observation point; The target point cloud data is two-dimensional point cloud data; Determine one or more initial orientations based on the orientation of at least one visible edge of the target; For each of the at least one target visible edge, within a first preset angle range, the initial orientation corresponding to the first target visible edge is rotated by different angles to obtain multiple candidate target orientations corresponding to the first target visible edge; For each candidate target orientation, one or more points on the visible edge of the first target are determined as interpolation points. The multiple candidate target orientations are then filtered based on the interpolation points to determine the target orientation. Based on the target orientation, a rectangle is fitted to determine the target shape information.

2. The target shape estimation method according to claim 1, characterized in that, The step of determining at least one visible edge on the target's outer envelope based on the target's outer envelope specifically includes: Based on the target envelope, determine the dividing line between the at least one visible edge and the at least one invisible edge of the target; The at least one visible edge and the at least one invisible edge of the target are determined based on the dividing line, wherein the dividing line is the line connecting the vertex corresponding to the maximum azimuth angle and the vertex corresponding to the minimum azimuth angle in the outer envelope of the target; the invisible edge of the target includes points that cannot be observed by the sensor at the observation point; wherein at least one vertex of the invisible edge of the target is located on a different side of the dividing line from the observation point; and at least one vertex of the visible edge of the target is located on the same side of the dividing line as the observation point.

3. The target shape estimation method according to claim 1 or 2, characterized in that, The step of determining one or more initial orientations based on the orientation of the at least one target visible edge specifically includes: For each of the first target visible edges in the at least one target visible edge, perform the following operation: According to the first preset algorithm, the two endpoints of the first target visible edge are analyzed to determine the orientation of the first target visible edge as the initial orientation corresponding to the first target visible edge; The first target visible edge is a target visible edge whose length exceeds the average length among the at least one target visible edges, or the first target visible edge is the longest target visible edge among the at least one target visible edges.

4. The target shape estimation method according to claim 3, characterized in that, The first preset algorithm is Where S represents the set of one or more initial orientations, Indicates the orientation of the visible edge of the first target. Indicates the orientation of the visible edge of the first target. The initial orientation θ corresponding to the visible edge of the first target i , and These represent the coordinates of the two endpoints of the first target's visible edge in a two-dimensional coordinate system, with the observation point as the origin, n being the number of the first target's visible edges, and i being an integer greater than 0.

5. The target shape estimation method according to claim 1 or 2, characterized in that, The step of filtering the multiple candidate target orientations based on the interpolation points to determine the target orientation specifically includes: Determine one or more target distances corresponding to one or more interpolation points. The target distance corresponding to an interpolation point is the minimum distance between the interpolation point and two adjacent sides of a first rectangle. The first rectangle is obtained by rectangle fitting of the candidate target orientation. The sum of the distances to the multiple targets is obtained to obtain the sum of the distances to the targets corresponding to the orientations of the candidate targets; Among the multiple candidate target orientations corresponding to each visible edge of the first target, the candidate target orientation corresponding to the minimum sum of target distances is determined as the first target orientation; The first target orientation is determined to be the target orientation.

6. The target shape estimation method according to claim 1 or 2, characterized in that, The target orientation is determined by filtering the multiple candidate target orientations based on the interpolation points, specifically including: Determine one or more target distances corresponding to one or more interpolation points. The target distance corresponding to an interpolation point is the minimum distance between the interpolation point and two adjacent sides of a first rectangle. The first rectangle is obtained by rectangle fitting of the candidate target orientation. The variance of the multiple target distances is calculated to obtain the variance of the target distance distribution corresponding to the candidate target orientation; Among the multiple candidate target orientations corresponding to each visible edge of the first target, the candidate target orientation corresponding to the smallest target distance distribution variance is determined as the first target orientation; The first target orientation is determined to be the target orientation.

7. The target shape estimation method according to claim 5, characterized in that, Determining the first target orientation as the target orientation specifically includes: Among one or more preset target orientations corresponding to the first target orientation, the preset target orientation with the smallest difference from the target velocity direction is determined as the target orientation.

8. The target shape estimation method according to claim 5, characterized in that, Determining the first target orientation as the target orientation specifically includes: Determine the average value of the historical trajectory direction of the target within a preset time period; Determine the difference between each preset target orientation corresponding to the first target orientation and the mean value; The preset target orientation corresponding to the smallest difference is determined as the target orientation.

9. The target shape estimation method according to claim 1 or 2, characterized in that, The step of performing rectangle fitting based on the target orientation to determine the target shape information specifically includes: Using a second preset algorithm, a rectangle is fitted to the target orientation to determine the target shape information; The second preset algorithm is e1=[cos(θ),sin(θ)], e2=[-sin(θ),cos(θ)]; Where L1 and L2 are the side lengths of two mutually perpendicular sides in the fitted rectangle corresponding to the target orientation, respectively. Let θ be the transpose of the matrix of interpolation points on the visible edge of the target corresponding to the target orientation, e1 and e2 represent the direction vectors corresponding to L1 and L2 respectively, V1, V2, V3 and V4 are the vertices of the rectangle, A is the transformation matrix, and θ is the target orientation.

10. A target shape estimation device, characterized in that, The device includes: The processing unit is configured to determine at least one visible edge of the target on the target outer envelope based on the target outer envelope; the target outer envelope is determined by target point cloud data; each visible edge of the target includes points in the target point cloud data that can be observed by the sensor at the observation point; the target point cloud data is two-dimensional point cloud data. The processing unit is further configured to determine one or more initial orientations based on the orientation of the at least one visible target edge; The processing unit is further configured to, for each of the at least one target visible edges, rotate the initial orientation corresponding to the first target visible edge by different angles within a first preset angle range to obtain multiple candidate target orientations corresponding to the first target visible edge; The processing unit is further configured to, for each candidate target orientation, determine one or more points on the visible edge of the first target as interpolation points, and filter the plurality of candidate target orientations according to the interpolation points to determine the target orientation; The processing unit is also used to perform rectangle fitting based on the target orientation to determine the target shape information.

11. The target shape estimation device according to claim 10, characterized in that, The processing unit is configured to determine at least one visible edge on the target's outer envelope based on the target's outer envelope, specifically including: Based on the target envelope, determine the dividing line between the at least one visible edge and the at least one invisible edge of the target; The at least one visible edge and the at least one invisible edge of the target are determined based on the dividing line, wherein the dividing line is the line connecting the vertex corresponding to the maximum azimuth angle and the vertex corresponding to the minimum azimuth angle in the outer envelope of the target; the invisible edge of the target includes points that cannot be observed by the sensor at the observation point; wherein at least one vertex of the invisible edge of the target is located on a different side of the dividing line from the observation point; and at least one vertex of the visible edge of the target is located on the same side of the dividing line as the observation point.

12. The target shape estimation device according to claim 10 or 11, characterized in that, The processing unit is further configured to determine one or more initial orientations based on the orientation of the at least one target visible edge, specifically including: For each of the first target visible edges in the at least one target visible edge, perform the following operation: According to the first preset algorithm, the two endpoints of the first target visible edge are analyzed to determine the orientation of the first target visible edge as the initial orientation corresponding to the first target visible edge; The first target visible edge is a target visible edge whose length exceeds the average length among the at least one target visible edges, or the first target visible edge is the longest target visible edge among the at least one target visible edges.

13. The target shape estimation device according to claim 12, characterized in that, The first preset algorithm is Where S represents the set of one or more initial orientations, Indicates the orientation of the visible edge of the first target. Indicates the orientation of the visible edge of the first target. The initial orientation θ corresponding to the visible edge of the first target i , and These represent the coordinates of the two endpoints of the first target's visible edge in a two-dimensional coordinate system, with the observation point as the origin, n being the number of the first target's visible edges, and i being an integer greater than 0.

14. The target shape estimation device according to claim 10 or 11, characterized in that, The processing unit is further configured to filter the multiple candidate target orientations based on the interpolation points to determine the target orientation, specifically including: Determine one or more target distances corresponding to one or more interpolation points. The target distance corresponding to an interpolation point is the minimum distance between the interpolation point and two adjacent sides of a first rectangle. The first rectangle is obtained by rectangle fitting of the candidate target orientation. The sum of the distances to the multiple targets is obtained to obtain the sum of the distances to the targets corresponding to the orientations of the candidate targets; Among the multiple candidate target orientations corresponding to each visible edge of the first target, the candidate target orientation corresponding to the minimum sum of target distances is determined as the first target orientation; The first target orientation is determined to be the target orientation.

15. The target shape estimation device according to claim 10 or 11, characterized in that, The processing unit is further configured to filter the multiple candidate target orientations based on the interpolation points to determine the target orientation, specifically including: Determine one or more target distances corresponding to one or more interpolation points. The target distance corresponding to an interpolation point is the minimum distance between the interpolation point and two adjacent sides of a first rectangle. The first rectangle is obtained by rectangle fitting of the candidate target orientation. The variance of the multiple target distances is calculated to obtain the variance of the target distance distribution corresponding to the candidate target orientation; Among the multiple candidate target orientations corresponding to each visible edge of the first target, the candidate target orientation corresponding to the smallest target distance distribution variance is determined as the first target orientation; The first target orientation is determined to be the target orientation.

16. The target shape estimation device according to claim 14, characterized in that, After determining the target orientation, the device further includes: The determining unit is configured to determine, among one or more preset target orientations corresponding to the first target orientation, the preset target orientation with the smallest difference from the target velocity direction as the target orientation.

17. The target shape estimation device according to claim 14, characterized in that, After determining the target orientation, the device further includes: The determining unit is used to determine the average value of the historical trajectory direction of the target within a preset time period; The determining unit is further configured to determine the difference between each preset target orientation corresponding to the first target orientation and the mean value; The determining unit is further configured to determine the preset target orientation corresponding to the smallest difference as the target orientation.

18. The target shape estimation device according to claim 10 or 11, characterized in that, The processing unit is further configured to perform rectangle fitting based on the target orientation to determine the target shape information, specifically including: Using a second preset algorithm, a rectangle is fitted to the target orientation to determine the target shape information; The second preset algorithm is e1=[cos(θ),sin(θ)], e2=[-sin(θ),cos(θ)]; Where L1 and L2 are the side lengths of two mutually perpendicular sides in the fitted rectangle corresponding to the target orientation, respectively. Let θ be the transpose of the matrix of interpolation points on the visible edge of the target corresponding to the target orientation, e1 and e2 represent the direction vectors corresponding to L1 and L2 respectively, V1, V2, V3 and V4 are the vertices of the rectangle, A is the transformation matrix, and θ is the target orientation.

19. A target shape estimation device, characterized in that, include: The device includes a processor, a memory, and a communication interface; wherein the communication interface is used to communicate with other devices or communication networks, and the memory is used to store one or more programs, the one or more programs including computer-executable instructions, wherein when the device is running, the processor executes the computer-executable instructions stored in the memory to cause the device to perform the target shape estimation method as described in any one of claims 1-9.

20. A computer-readable storage medium, characterized in that, It includes programs and instructions, which, when run on a computer, implement the target shape estimation method as described in any one of claims 1-9.

21. A computer program product containing instructions, characterized in that, When the computer program product is run on a computer, the computer performs the target shape estimation method as described in any one of claims 1-9.

Images