Trajectory planning method, device, controller and intelligent vehicle

Abstract

A trajectory planning method, device, controller and intelligent vehicle are used to effectively detect obstacles and plan the driving trajectory of the intelligent vehicle. The method comprises the following steps: the controller of the intelligent vehicle obtains obstacle data collected by multiple sensors, performs deduplication processing on the obstacle data collected by the multiple sensors to obtain a detection result of a target obstacle, and finally performs trajectory planning of the vehicle based on the detection result of the target obstacle. By combining the obstacle data of multiple sensors, the obstacles around the intelligent vehicle can be comprehensively detected, and repeated obstacles can be effectively removed. The method can be used to reasonably plan the driving trajectory of the intelligent vehicle and improve the safety of automatic driving.

Description

[0001] This application is a divisional application. The original application has the application number 201911417562.5 and the original application date is December 31, 2019. The entire contents of the original application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of smart / inteligent car, and more particularly to a method, apparatus, controller, and smart car for trajectory planning during automatic driving (ADS) process. Background Technology

[0003] Obstacle perception in autonomous driving refers to the detection of obstacles around the vehicle by sensors during the autonomous driving process, and it is the foundation for achieving autonomous driving. Current methods for obstacle detection mainly rely on onboard sensors. Typically, each intelligent vehicle uses a combination of LiDAR and high-precision maps to confirm obstacle information and then plan its driving trajectory. However, LiDAR is greatly affected by weather, and its application scenarios and accuracy are limited. Another approach is to use cameras as detection devices. Since cameras can only capture two-dimensional images, complex processing is required to identify the categories of objects in the images. However, they cannot distinguish different individuals of the same type of object. Therefore, cameras cannot meet the obstacle detection needs of intelligent vehicles in various complex scenarios during autonomous driving, and cannot reasonably plan the vehicle's driving trajectory, affecting the safety of autonomous driving. Therefore, how to effectively detect obstacles and plan the driving trajectory of intelligent vehicles has become an urgent technical problem to be solved. Summary of the Invention

[0004] This application provides a trajectory planning method, apparatus, controller, and intelligent vehicle to effectively detect obstacles and plan the driving trajectory of the intelligent vehicle.

[0005] Firstly, this application provides a trajectory planning method, which can be implemented by a controller in an intelligent vehicle. After acquiring obstacle data from multiple sensors, the controller performs deduplication processing on the obstacle data to obtain the detection result of the target obstacle. Finally, based on the detection result of the target obstacle, the controller performs trajectory planning for the vehicle. The multiple sensors include at least two types of sensors.

[0006] The above method addresses the current inability to adequately detect obstacles in intelligent vehicles, thus hindering the proper planning of their driving trajectories. This method combines obstacle data from multiple sensors to comprehensively detect obstacles around the intelligent vehicle and effectively remove duplicate obstacles. This allows for the rational planning of the intelligent vehicle's trajectory, improving the safety of autonomous driving.

[0007] In one possible design, when the controller performs deduplication processing on obstacle data collected by multiple sensors to obtain the detection result of the target obstacle, the specific method can be as follows: The controller divides multiple sensors into multiple perception groups according to the perception range of multiple sensors, and further divides the multiple perception groups into multiple calculation groups according to the maximum range formed by the perception ranges of the multiple perception groups; then, after performing a first deduplication processing on the obstacle data collected by the sensors in each calculation group, a second deduplication processing is performed on the first deduplication results corresponding to the multiple calculation groups to obtain a second deduplication result; wherein any calculation group contains at least two perception groups, and the maximum ranges of at least two perception groups overlap; any perception group contains at least two sensors, and the at least two sensors are of at least two different types; the perception ranges of the sensors in any perception group overlap; the total range of the perception ranges of all sensors in all perception groups covers the panoramic range of the intelligent vehicle.

[0008] Using the above method, the controller can first divide multiple sensors into sensing groups, then divide the sensing groups into calculation groups, then simultaneously deduplicate the data in several calculation groups, and finally deduplicate the data between calculation groups. This allows multiple deduplication processes to be executed simultaneously, thereby improving the accuracy and efficiency of obstacle deduplication.

[0009] In another possible design, when the controller performs the first deduplication process on the obstacle data collected by the sensors in any computing group, the specific method can be as follows: the controller performs the first deduplication process on the obstacle data collected by the sensors in each sensing group within any computing group, and then performs a second deduplication process on the first deduplication results corresponding to at least two sensing groups within any computing group, obtaining a second deduplication result corresponding to any computing group. The second deduplication result includes the detection result of the target obstacle. Using this method, obstacle data in several sensing groups can be deduplicated simultaneously, and then data between sensing groups can be deduplicated. This allows for simultaneous execution of multiple deduplication processes, improving deduplication efficiency, and deduplication within groups followed by deduplication between groups can improve deduplication accuracy.

[0010] In another possible design, when the controller performs the first deduplication process on the obstacle data collected by the sensors in any perception group, the specific method can be as follows: The controller generates an obstacle attribute list corresponding to each sensor based on the obstacle data collected by each sensor in any perception group. Then, it merges the obstacle attribute lists corresponding to all sensors in the perception group to obtain the obstacle attribute list corresponding to any perception group. Each sensor's obstacle attribute list contains the attribute of at least one obstacle. This method merges the attribute data of the same obstacle identified by all sensors in a perception group, ensuring that there are no duplicate obstacles in the obstacle attribute list corresponding to the perception group, thereby improving the accuracy of obstacle detection.

[0011] In another possible design, when the controller generates an obstacle attribute list for each sensor based on the obstacle data collected by each sensor, the specific method can be as follows: When the sensor is a LiDAR, the obstacle data collected by the LiDAR is point cloud data. The controller can first project the point cloud data onto a 2D RGB image based on preset virtual camera parameters, and then perform instance segmentation on the resulting 2D RGB image to obtain the obstacle attribute list corresponding to the LiDAR. When the sensor is a camera, the obstacle data collected by the camera is a 2D RGB image. The controller can directly perform instance segmentation on the 2D RGB image to obtain the obstacle attribute list corresponding to the camera. This method allows for the generation of corresponding obstacle attribute lists for different types of sensors, facilitating subsequent fusion of obstacle attribute data.

[0012] In another possible design, when the controller performs instance segmentation on any 2D RGB image to obtain a list of obstacle attributes, the specific method could be as follows: the controller inputs the 2D RGB image into a preset instance segmentation model, obtains the instance segmentation results, and then generates a list of obstacle attributes based on the instance segmentation results. Using this method, the 2D RGB image corresponding to the sensor can be analyzed through the instance segmentation model to determine the attributes of obstacles, thereby obtaining a list of obstacle attributes corresponding to the sensor.

[0013] In another possible design, when the controller fuses the obstacle attribute lists corresponding to all sensors in any perception group, the specific method can be as follows: starting from the first unmarked obstacle in the first obstacle attribute list until all obstacles in the first obstacle attribute list are marked as "fused," match them with obstacles in the second obstacle attribute list; when a matching obstacle exists in the second obstacle attribute list, merge the attribute data of the obstacle in the first and second obstacle lists, and merge the attribute data of obstacles near the obstacle in the first and second obstacle lists. The attribute data in the list are merged, and obstacles and obstacles near the obstacles are marked as "fused". When there is no matching obstacle in the second obstacle attribute list, the obstacle is marked as "fused". The first obstacle attribute list is the fused obstacle attribute list of A obstacle attribute lists from the obstacle attribute lists corresponding to N sensors in any perception group, and the second obstacle attribute list is the fused obstacle attribute list of B obstacle attribute lists. N is an integer greater than or equal to 2, A is an integer greater than or equal to 1, B is an integer greater than or equal to 1, and A+B=N. This method can merge the attribute data of the same obstacle detected by all sensors in a perception group, ensuring that there are no duplicate obstacles in the obstacle attribute list corresponding to the perception group, thereby improving the accuracy of obstacle detection. Furthermore, combining the relationship between the obstacle and nearby obstacles during the fusion process to identify duplicate obstacles can improve deduplication efficiency.

[0014] In another possible design, sensor A can be the same or different types of sensors; sensor B can be the same or different types of sensors.

[0015] In another possible design, the controller performs a second deduplication process on the first deduplication results corresponding to at least two perception groups within any given calculation group. Specifically, to obtain the second deduplication result for any given calculation group, the controller determines the perception range for each perception group based on the obstacle positions in the first deduplication results, and then determines the overlap range between at least two perception groups based on the perception range of each group. Finally, the second deduplication process is performed on the first deduplication results corresponding to at least two perception groups with overlapping ranges. This method removes duplicate obstacles between different perception groups, ensuring that no duplicate obstacles exist within a given calculation group, thereby improving the accuracy of obstacle detection.

[0016] In another possible design, when the controller performs a second deduplication process on the first deduplication results corresponding to at least two perception groups with overlapping ranges, the specific method can be as follows: the controller starts from the first unmarked fused obstacle in the third obstacle attribute list and continues until all obstacles in the third obstacle attribute list are marked as "fused," matching them with obstacles in the fourth obstacle attribute list; when a matching obstacle exists in the fourth obstacle attribute list, the attribute data of the obstacle in the third obstacle list and the fourth obstacle list are merged, and the attribute data of obstacles near the obstacle are merged in the third obstacle list and the fourth obstacle list. The attribute data in the obstacle list are merged, and the obstacle and the obstacles near the obstacle are marked as "merged". When there is no matching obstacle in the fourth obstacle attribute list, the obstacle is marked as "merged". The third obstacle attribute list is the obstacle attribute list after deduplication and fusion of the C perception groups in M ​​perception groups in any calculation group, and the fourth obstacle attribute list is the obstacle attribute list after deduplication and fusion of the D perception groups. M is an integer greater than or equal to 2, C is an integer greater than or equal to 1, D is an integer greater than or equal to 1, and C+D=M.

[0017] Using the above method, the attribute data of the same obstacle corresponding to all perception groups in a calculation group can be merged together, so that there are no duplicate obstacles in the obstacle attribute list corresponding to a calculation group, thereby improving the accuracy of obstacle detection; and in the process of fusion, combining the relationship between the obstacle and nearby obstacles to identify duplicate obstacles can improve the deduplication efficiency.

[0018] In another possible design, when the controller performs a second deduplication process on the first deduplication results corresponding to multiple calculation groups, the specific method can be as follows: The controller starts from the first unmarked fused obstacle in the fifth obstacle attribute list and continues until all obstacles in the fifth obstacle attribute list are marked as "fused," matching them with obstacles in the sixth obstacle attribute list. When a matching obstacle exists in the sixth obstacle attribute list, the attribute data of the obstacle in both the fifth and sixth obstacle lists are merged, and the attribute data of obstacles near the original obstacle are also merged in both lists. The attribute data is merged, and obstacles and their nearby obstacles are marked as "merged". When there is no matching obstacle in the sixth obstacle attribute list, the obstacle is marked as "merged". The fifth obstacle attribute list is the merged list of E obstacle attribute lists from the L calculation groups, and the sixth obstacle attribute list is the merged list of F obstacle attribute lists. L is the total number of calculation groups, L is an integer greater than or equal to 2, E is an integer greater than or equal to 1, F is an integer greater than or equal to 1, and E+F=L. This method merges the attribute data of the same obstacles corresponding to all calculation groups, ensuring that there are no duplicate obstacles in the final obstacle detection result's obstacle attribute list, thus improving the accuracy of obstacle detection. Furthermore, combining the relationship between the obstacle and nearby obstacles during the fusion process to identify duplicate obstacles improves deduplication efficiency.

[0019] In another possible design, a matching obstacle is considered to exist when the following conditions are met simultaneously: the difference between the timestamps of the two obstacles is less than a preset range, the two obstacles belong to the same obstacle type, the positional deviation of the two obstacles is within a preset range, and the border areas of the two obstacles overlap. Using this method, it is possible to accurately determine that two obstacles are the same, and then merge the attribute data of the same obstacles to achieve obstacle deduplication.

[0020] In a second aspect, this application provides a trajectory planning apparatus, which includes various modules or units for executing the trajectory planning method in the first aspect or any possible design of the first aspect, such as a processing unit and an acquisition unit.

[0021] Thirdly, this application provides a controller, which includes a processor and a memory. When the controller is in operation, the processor executes computer programs or instructions stored in the memory to cause the controller to perform the corresponding methods as shown in the first aspect or any possible design of the first aspect above.

[0022] Fourthly, this application provides an intelligent vehicle, which may include the trajectory planning device described in any of the second aspects above or the intelligent driving domain controller described in any of the third aspects.

[0023] Fifthly, this application provides an in-vehicle system, which may include the controller and multiple sensors mentioned above.

[0024] In a sixth aspect, this application provides a computer-readable storage medium storing a program or instructions that, when executed on a computer, cause the computer to perform the method described in the first aspect or any possible design of the first aspect.

[0025] In a seventh aspect, this application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the method described in the first aspect or any of the possible designs of the first aspect.

[0026] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0027] Figure 1 A schematic diagram of the architecture of an in-vehicle system to which the trajectory planning method provided in this application is applicable; Figure 2 A schematic diagram of the architecture of an in-vehicle system to which the alternative trajectory planning method provided in this application is applicable; Figure 3 A schematic diagram of the architecture of an in-vehicle system to which the alternative trajectory planning method provided in this application is applicable; Figure 4 A flowchart of a trajectory planning method provided in this application; Figure 5 A schematic diagram of a point cloud provided for this application; Figure 6 A schematic diagram of the structure of a Mask R-CNN model provided in this application; Figure 7 A schematic diagram illustrating an instance segmentation method provided in this application; Figure 8 A schematic diagram illustrating a list of obstacle attributes provided in this application; Figure 9 A schematic diagram illustrating the fusion of an obstacle attribute list provided in this application; Figure 10 A schematic diagram illustrating the overlapping range of two sensing groups provided in this application; Figure 11 A flowchart illustrating an example of a trajectory planning method provided in this application; Figure 12 A schematic diagram of the structure of a trajectory planning device provided in this application; Figure 13 This application provides a structural diagram of a controller. Detailed Implementation

[0028] The present application will now be described in further detail with reference to the accompanying drawings.

[0029] The trajectory planning method provided in this application is applicable to in-vehicle systems, which can be deployed in intelligent vehicles. The architecture of the in-vehicle system includes at least multiple sensors and controllers, wherein the multiple sensors may include one or more of LiDAR, cameras, and millimeter-wave radar. For example, Figure 1 This diagram illustrates the layout of multiple sensors in a smart car. For example, in Figure 1 The multiple sensors include five lidars: lidar a, lidar b, ... lidar e, and six cameras: cameras 0-5.

[0030] It should be understood that Figure 1 This application is not limited to including more or fewer sensors, or other types of sensors, as merely an example.

[0031] The multiple sensors collect environmental data and input it into the controller. The data collected by the lidar is point cloud data, and the data collected by the camera is image data. Each sensor has a corresponding sensing range, which is the detection range of each sensor for obstacles. The controller can divide the multiple sensors into multiple sensing groups based on preset principles. For example, the controller can group at least two sensors with overlapping sensing ranges into a sensing group based on their sensing ranges. Figure 1 For example, the controller can divide a camera and a LiDAR with overlapping sensing ranges into a sensing group, which can be divided into the following 6 groups: Perception group (1): Camera 0 and LiDAR a; Perception group (2): Camera 3 and LiDAR b; Perception group (3): Camera 1 and LiDAR a; Perception group (4): Camera 4 and LiDAR d; Perception group (5): Camera 2 and LiDAR c; Perception group (6): Camera 5 and LiDAR e.

[0032] The controller can determine the possibility of overlapping total sensing ranges between sensing groups based on the installation positions of the sensors within the sensing group. Then, it allocates the data collected by the sensors of two or more overlapping sensing groups to a sensing computing unit for deduplication. For example, the data of sensing group (1) and sensing group (2) are allocated to sensing computing unit A, the data of sensing group (3) and sensing group (5) are allocated to sensing computing unit B, and the data of sensing group (4) and sensing group (6) are allocated to sensing computing unit C.

[0033] Optionally, the same sensor can belong to one sensing group, or the same sensor can belong to multiple sensing groups.

[0034] Optionally, the architecture of the vehicle system may include multiple perception computing units, which are used to deduplicatize data from several perception groups within the divided perception groups. For example... Figure 1 The sensor computing units A, B, and C are shown. These sensor computing units can exist independently of the controller, in the form of boards or similar devices. Figure 2 The diagram showing the vehicle system architecture illustrates the deployment relationship between controller 101 and perception computing units A, B, and C; the perception computing units can also be deployed within the controller, for example, Figure 3 The diagram shows the deployment relationship between controller 101 and perception computing units A, B, and C in the vehicle system architecture.

[0035] in Figure 2 or Figure 3 The controller 101 can further deduplicate the results obtained by the sensing units A, B, and C to obtain the final obstacle detection result, so that the intelligent vehicle can be used for trajectory planning based on the final obstacle detection result.

[0036] Optionally, the controller 101 is specifically implemented by a processor, which includes a central processing unit (CPU) or a device or module with processing capabilities. For example, the controller 101 may be a mobile data center (MDC).

[0037] It should be noted that, Figure 1 , Figure 2 , Figure 3 The number and layout of the controllers, sensors, or sensing computing units are merely examples; in practice, there can be various implementation methods, which are not specifically limited herein. Figure 1 Only the layout of multiple sensors in the smart car is shown. Figure 2 and Figure 3 They are in Figure 1This diagram illustrates the layout of the controller and sensing computing unit. Figure 2 The sensing and computing unit exists independently of the controller; that is, the sensing and computing unit and the controller are deployed independently. Figure 3 This means that the perception computing unit is integrated into the controller, i.e., the perception computing unit is deployed within the controller. In the following introduction to the trajectory planning scheme, we will use the deployment of the perception computing unit within the controller as an example, and the controller as the main execution entity.

[0038] Based on the above description, this application provides a trajectory planning method, applicable to... Figure 1 Or the system shown in Figure 3. The method can be implemented by a controller in the intelligent vehicle. See also... Figure 4 As shown, the specific process of the method may include: Step 401: The controller acquires obstacle data collected by multiple sensors; the multiple sensors include at least two types of sensors.

[0039] Obstacles are a general term for terrain features, landforms, and engineering facilities that can impede or delay the movement of vehicles. In this application, obstacles can include people, vehicles, road infrastructure, and road signs, etc.

[0040] The multiple sensors can include various types of sensors such as lidar, cameras, and millimeter-wave radar. The obstacle data collected by lidar is a point cloud, also known as a point cloud image. The point cloud image can be a three-dimensional image, for example... Figure 5 The point cloud image shown is illustrated. The obstacle data acquired by the camera is an image, specifically a two-dimensional red (R), green (G), and blue (B) image. The obstacle data acquired by the millimeter-wave radar is also a point cloud.

[0041] Step 402: The controller performs deduplication processing on the obstacle data collected by the multiple sensors to obtain the detection result of the target obstacle.

[0042] In one specific implementation, when the controller performs deduplication processing on the obstacle data collected by the multiple sensors to obtain the detection result of the target obstacle, the specific method may be as follows: The controller divides the multiple sensors into multiple sensing groups based on their sensing ranges; each sensing group contains at least two sensors of at least two different types, and the sensing ranges of the sensors in any sensing group overlap; the total sensing range of all sensors in all sensing groups covers the panoramic range of the intelligent vehicle. The controller divides the multiple sensing groups into multiple calculation groups based on the maximum range formed by the sensing ranges of the multiple sensing groups; wherein any calculation group contains at least two sensing groups, and the maximum ranges of the at least two sensing groups overlap. The controller performs a first deduplication process on the obstacle data collected by the sensors in each computing group; The controller performs a second deduplication process on the first deduplication results corresponding to the multiple calculation groups to obtain the second deduplication result, which is the detection result of the target obstacle.

[0043] Specifically, based on the parameters of each sensor, the sensing range of that sensor, i.e., the detection range of obstacles in the surrounding environment, is determined. When the controller divides the multiple sensors into multiple sensing groups based on their sensing ranges, sensors with overlapping sensing ranges can be grouped into one sensing group. This overlapping range refers to two sensors with partial overlap being grouped into one sensor group. The extent of partial overlap can be set with different thresholds according to specific implementation requirements to control the size of the overlapping area, and further subdividing the sensor groups based on this. Optionally, multiple sensors with an overlap range greater than a preset overlap rate can be grouped into one sensing group, where the preset overlap rate refers to the percentage of the overlap range between the sensors relative to the total sensing range of the overlapping sensors. Of course, in practice, sensors with the largest overlap rate should be grouped into one sensing group whenever possible. The specific division by the controller depends on the sensor's installation location in the intelligent vehicle; generally, sensors with adjacent installation locations are more likely to have overlapping ranges. For example... Figure 1 The example of sensor grouping in the system includes two types of sensors: a camera and a LiDAR. The sensors in each group are installed close to each other, with the aim of maximizing the overlap range and ensuring that the total sensing range of all sensing groups covers the panoramic range of the intelligent vehicle, that is, covering all 360-degree angles of the intelligent vehicle.

[0044] In one exemplary embodiment, based on the installation locations of the sensors that make up different sensing groups, the maximum range of the sensing range of each sensing group can be determined, and at least two sensing groups that overlap can be grouped into a calculation group. Of course, the overlap range between sensing groups can also be based on a preset overlap rate. For example, still using... Figure 1 For example, the pre-divided perception groups (1) and (2) are divided into a calculation group, perception groups (3) and (5) are divided into a calculation group, and perception groups (4) and (6) are divided into a calculation group.

[0045] Furthermore, after being divided into multiple calculation groups, the controller can simultaneously perform deduplication processing on the obstacle data in all calculation groups. In an optional implementation, the controller performs deduplication processing on the obstacle data collected by the sensors in any calculation group. Specifically, the controller performs a first deduplication process on the obstacle data collected by the sensors in each sensing group of the calculation group, and then performs a second deduplication process on the first deduplication results corresponding to at least two sensing groups in the calculation group to obtain the second deduplication result corresponding to the calculation group.

[0046] It should be noted that the first deduplication process refers to deduplicating obstacle data from sensors within the same sensing group, while the second deduplication process refers to deduplicating data between sensing groups. The deduplication of data between sensing groups below is also referred to as the second deduplication process.

[0047] It should be noted that the controller may contain multiple units or modules, such as a perception computing unit, and each unit or module performs the above deduplication process on one of the computing groups.

[0048] In one optional implementation, the controller performs a first deduplication process on the obstacle data collected by the sensors in any one perception group. Specifically, the controller generates an obstacle attribute list corresponding to each sensor based on the obstacle data collected by each sensor in the any one perception group. The obstacle attribute list corresponding to each sensor contains the attribute of at least one obstacle. The controller then fuses the obstacle attribute lists corresponding to all sensors in the any one perception group to obtain the obstacle attribute list corresponding to the any one perception group, which is the first deduplication result corresponding to the any one perception group.

[0049] In one optional implementation, when the controller processes obstacle data collected by each sensor in a perception group, the specific method for generating an obstacle attribute list corresponding to each sensor based on the obstacle data collected by each sensor can be as follows: when the sensor is a LiDAR, the obstacle data collected by the LiDAR is point cloud data, and the point cloud data is projected onto a two-dimensional RGB image based on preset virtual camera parameters; the two-dimensional RGB image obtained after projection is segmented into instances to obtain the obstacle attribute list corresponding to the LiDAR; when the sensor is a camera, the obstacle data collected by the camera is a two-dimensional RGB image, and the two-dimensional RGB image is segmented into instances to obtain the obstacle attribute list corresponding to the camera.

[0050] In one example, when the sensor is a LiDAR, the process by which the controller projects the point cloud data onto a two-dimensional RGB image based on preset virtual camera parameters can be as follows: (1) Preset virtual camera parameters A camera is defined as C=K[RT], where R represents the rotational extrinsic parameter of the camera and is a 3x3 matrix. For example, R can be represented by the following matrix:

[0051] in These represent the camera's rotation angles around the x, y, and z axes, respectively. T is a 3x1 matrix representing the camera's translation parameters; K is the camera's intrinsic parameter, used for calibration and adjustment of camera intrinsic parameters, including parameters such as focal length and distortion. For example, K is shown in the matrix below: , in, , This is the camera's focal length; generally, the two are equal. This is the coordinate axis tilt parameter, which is ideally 0; , Principal point coordinates (relative to the imaging plane).

[0052] (2) Project the point cloud data onto a two-dimensional RGB image based on preset virtual camera parameters. The Katz projection method can be used here, or other methods can be used. This application does not make any specific limitations.

[0053] During projection, it is crucial for the controller to determine which points in the point cloud are visible to the camera. Taking the Katz projection algorithm as an example, this algorithm not only removes all points in the point cloud that are not visible to the camera, but also eliminates points that are too close to each other to accelerate the process. Points that are too close to each other can be defined as those with a distance less than a preset threshold. This is first achieved by defining a sphere of radius R around all points in the point cloud and placing the camera C at the origin. The spherical flipping of the points in the point cloud is calculated using the following formula: +2(R- ) ; in, These are the coordinates of the points after the flip. These are the coordinates of the point before the flip.

[0054] When the controller determines which points are visible, it projects the 3D points into pixels based on the following formula: C ,in, C represents the coordinates of the points projected onto the image, and C represents the defined camera parameters.

[0055] Because multiple points can be projected onto the same pixel, the controller calculates the weights of different point cloud projections onto the corresponding image pixels using the following Gaussian function:

[0056] Where dist represents the distance between the pixel and the point position.

[0057] In one optional implementation, the controller performs instance segmentation on the 2D RGB image obtained by projecting the point cloud data from the LiDAR to obtain a list of obstacle attributes corresponding to the LiDAR, which is the same method as performing instance segmentation on the 2D RGB image acquired by the camera to obtain a list of obstacle attributes corresponding to the camera. Specifically, when performing instance segmentation on any 2D RGB image, the Mask R-CNN model can be used for instance segmentation. The structure of the Mask R-CNN model can be as follows... Figure 6 As shown, where: Convolutional neural network (CNN): A series of convolutional layers are used to extract feature maps of an image. For example, it can be VGG16, VGG19, GooLeNet, ResNet50, ResNet101, etc. The ResNet101 structure is commonly used. The region proposal network (RPN) is used to help the network recommend regions of interest and extract feature maps of those regions. Fully connected layers identify obstacle categories and bounding box locations on each fixed-size feature map; The Mask branch is used for semantic segmentation.

[0058] Specifically, the controller can input a 2D RGB image into the Mask R-CNN model to obtain instance segmentation results. For example, the instance segmentation results may include the obstacle category, obstacle outline, obstacle bounding box, etc.

[0059] For example, Figure 7 A schematic diagram of instance segmentation is shown. Figure 7 Image (a) is a two-dimensional RGB image. Figure 7 (b) shows a schematic of the Mask R-CNN model. Figure 7 (c) represents the instance segmentation result.

[0060] Furthermore, the controller can obtain a list of obstacle attributes corresponding to the sensor based on the instance segmentation results. For example, any obstacle attribute list may include the obstacle's identity (ID), timestamp, obstacle category (e.g., person or vehicle), obstacle position, and obstacle size (length and width). Optionally, any obstacle list may also include at least one of the following: obstacle outline, relationship with nearby obstacles, relative velocity, and obstacle pose (e.g., vehicle orientation). The obstacle's position can be the coordinates of its center point, which can be represented by two-dimensional or three-dimensional coordinates; the relationship with nearby obstacles can be represented by a vector pointing from the center point of the obstacle to the center point of nearby obstacles.

[0061] For example, with Figure 1 Taking the perception group (2) consisting of camera 1 and lidar b as an example, see [reference] Figure 8 As shown, the area within the triangle represents the perception range of camera 3, which includes two people, P1 and P2, and one car, car1. LiDAR b has a 360-degree perception range; excluding the car portion, LiDAR b detects two people, P1 and P2, and two cars, car1 and car2, as obstacles. The obstacle attribute lists generated by processing the data collected by LiDAR b and camera a are shown below. Figure 8 The list is shown below. The obstacle attribute list only shows a portion of the attributes and does not represent all attributes. The relationship between an obstacle and nearby obstacles is represented by a vector from the center of the obstacle to the centers of nearby obstacles. For example, in this case, taking person P1 as an example, draw a circle with P1 as the center and radius R. This circle contains obstacles P2 and car1 (excluding P1 itself). The relationship between P1 and nearby obstacles can be represented as a vector with the center of P1 as the origin pointing to the centers of P2 and car1.

[0062] In one optional implementation, the controller fuses the obstacle attribute lists corresponding to all sensors in any given perception group. Specifically, the controller starts from the first unmarked obstacle in the first obstacle attribute list and continues until all obstacles in the first obstacle attribute list are marked as "fused," matching them with obstacles in the second obstacle attribute list. When a matching obstacle exists in the second obstacle attribute list, the attribute data of that obstacle in both the first and second obstacle lists are merged, and the attribute data of obstacles near that obstacle are merged in both the first and second obstacle lists. The attribute data in the two obstacle lists are merged, and the obstacle and the obstacles near the obstacle are marked as "merged". When there is no matching obstacle in the second obstacle attribute list, the obstacle is marked as "merged". The first obstacle attribute list is the obstacle attribute list after merging A obstacle attribute lists corresponding to N sensors in any perception group, and the second obstacle attribute list is the obstacle attribute list after merging B obstacle attribute lists. N is an integer greater than or equal to 2, A is an integer greater than or equal to 1, B is an integer greater than or equal to 1, and A+B=N.

[0063] Specifically, when N equals 2, meaning there are only two sensors in a perception group, such as a LiDAR and a camera, the first obstacle attribute list is either the obstacle attribute list obtained from the LiDAR point cloud processing or the obstacle attribute list obtained from the camera image. The second obstacle attribute list is the other one. Taking the first obstacle list as an example (generated from the camera image): When matching the first unmarked fused obstacle in the obstacle attribute list generated from the image with an obstacle in the obstacle attribute list generated from the point cloud, the following principles can be followed: First, determine if the timestamp difference between the two obstacles is within a preset range. If it is not within the preset range, it indicates no match, and the corresponding obstacle in the obstacle attribute list obtained from the point cloud is directly marked as "fused," indicating that this obstacle has no duplicate obstacles. If it is within the preset range, further determine if the types of the two obstacles are the same. If the types are different, it indicates no match, and the corresponding obstacle in the obstacle attribute list obtained from the point cloud is marked as "fused," indicating that this obstacle has no duplicate obstacles. If the obstacles are of the same type, further determine whether the positional deviation of the two obstacles is within a preset range. If it is not within the preset range, it is considered that there is no match, and the corresponding obstacle in the obstacle attribute list obtained from the point cloud is marked as "fused", indicating that there is no duplicate obstacle with it. If it is within the preset range, further calculate whether the area of ​​the obstacle rectangle reaches the preset overlap degree based on the size (length and width) of the obstacle. If it does not reach the preset overlap degree, it is considered that there is no match, and the corresponding obstacle in the obstacle attribute list obtained from the point cloud is marked as "fused", indicating that there is no duplicate obstacle with it. If it reaches the preset overlap degree, it is considered that the two obstacles match, that is, a matching obstacle has been found, and both obstacles are marked as "fused".

[0064] Of course, optionally, in order to make the matching more accurate, in addition to the above conditions, other obstacle attributes can be used to determine whether a match is made, such as whether the relationship with the adjacent obstacles is consistent, etc. The specific matching methods are similar and will not be described in detail here.

[0065] In an optional implementation, once two obstacles from the two obstacle lists match, to save on calculations, adjacent obstacles can be matched based on their relationship to nearby obstacles. For example, Figure 8 Taking the matching process of a1 and b3 in the equation as an example, the matching process of obstacle a2 near a1 can be described using the formulas... Calculate the error values ​​between a2 and b2, b1. and If the minimum of the two values ​​is less than a threshold d, then the two are considered a match. Let a1 be the vector relating to nearby obstacles. Let be the relationship vector between b3 and nearby obstacles. x is the value of the relationship vector along the x-axis, and y is the value of the relationship vector along the y-axis.

[0066] Based on the matching results, the attribute data of the two matched obstacles are merged, which is the process of merging attribute data. This can merge the attribute data of the same obstacle into the attribute data of one obstacle, so that there are no duplicate obstacles after merging, thus achieving the purpose of deduplication.

[0067] The above process is then repeated until all obstacles in the obstacle attribute list generated from the point cloud are marked as merged. This completes the merging of all data from the two lists into a single obstacle attribute list, where no duplicate obstacles are found. For example, Figure 9 The diagram illustrates the fusion process. After final fusion, a1 and b3 are merged, a2 and b2 are merged, a3 and b1 are merged, and a4 remains unchanged. This results in a fused obstacle attribute list containing attribute data for all four obstacles. This fused obstacle attribute list then becomes the attribute list corresponding to one perception group.

[0068] In one alternative implementation, before fusing the data of the two obstacles, the controller can first predict and compensate the position X of the obstacle with the later timestamp based on its velocity V according to the timestamp difference Δt, as shown in the following formula: Then, when merging the positions, the compensated positions are merged.

[0069] The above describes the case when N equals 2, meaning there are only two sensors in a sensing group. When N is greater than or equal to 3, meaning there are 3 or more sensors in a sensing group, the obstacle attribute lists of any two sensors can be merged first. Then, the merged obstacle attribute list is merged with the obstacle attribute list of any of the remaining sensors until all obstacle attribute lists are merged into a single obstacle attribute list. Alternatively, the obstacle attribute lists of multiple sensors can be merged pairwise first, and then the merged obstacle attribute lists can be merged pairwise again until a single obstacle attribute list is obtained. Here, the two obstacle attribute lists merged in the last step can be used as the first obstacle attribute list and the second obstacle attribute list, respectively. Alternatively, the obstacle attribute lists of sensors of the same type from multiple sensors can be merged, and the two resulting obstacle attribute lists can then be merged. Of course, many other methods can be used; any method that can merge all obstacle attribute lists into a single obstacle attribute list is acceptable, and this application does not limit this approach. Specifically, the method for fusing any two obstacle attribute lists is the same as the method for fusing two obstacle attribute lists when N equals 2, as described above, and can be referred to accordingly; it will not be described in detail here. In an optional implementation, during the pairwise fusion process, obstacle attribute lists from sensors of the same type can be fused first, and then fused with obstacle attribute lists from sensors of different types.

[0070] Using the above method, a list of obstacle attributes corresponding to each perception group can be obtained, which is the first deduplication result for each perception group, achieving the purpose of deduplication within the perception group. In an optional implementation, the controller performs a second deduplication process on the first deduplication results corresponding to at least two perception groups in any calculation group. Specifically, the controller determines the perception range corresponding to each perception group based on the position of obstacles in the first deduplication results corresponding to each perception group; the controller determines the overlap range between the at least two perception groups based on the perception range corresponding to each perception group, and performs a second deduplication process on the first deduplication results corresponding to the at least two perception groups with overlapping ranges.

[0071] In one optional implementation, the controller determines the sensing range of each sensing group based on the position of the obstacle in the first deduplication result corresponding to each sensing group. Specifically, the controller projects the obstacle corresponding to each sensing group onto the vehicle's three-dimensional coordinate system and determines the obstacle sensing range of each sensing group based on the coordinates of the obstacle's center point. The vehicle's three-dimensional coordinate system can be one with the rear axle center of the vehicle as the origin, the z-axis perpendicular to the horizontal direction, the x-axis pointing positively towards the front of the vehicle, and the y-axis in the horizontal direction.

[0072] In one alternative implementation, the controller can determine the center point of the obstacle in each sensing group according to the following formula. Projected onto the vehicle's three-dimensional coordinate system :

[0073] in These represent the rotation angles along the x, y, and z axes from the original obstacle coordinate system to the vehicle coordinate system. This is the translation matrix of the xyz axes during coordinate system changes.

[0074] In one optional implementation, the controller determines the obstacle sensing range of each sensing group based on the coordinates of the obstacle's center point. This can be achieved by using the center point of each obstacle as the origin and a circular area with a preset radius as the overall coverage area of ​​each sensing group. Furthermore, the overlap range of the sensing groups is determined based on empirical values, taking into account the installation positions of the sensors within each sensing group. For example, the overlap range of two overlapping sensing groups can be determined as follows: Figure 10 As shown.

[0075] In one optional implementation, the controller performs a second deduplication process on the first deduplication results corresponding to at least two perception groups with overlapping ranges. Specifically, the controller starts from the first unmarked fused obstacle in the third obstacle attribute list and continues until all obstacles in the third obstacle attribute list are marked as "fused," matching them with obstacles in the fourth obstacle attribute list. When a matching obstacle exists in the fourth obstacle attribute list, the attribute data of the obstacle in both the third and fourth obstacle lists are merged, and the attribute data of obstacles near the obstacle are merged in both the third and fourth obstacle lists. The attribute data in the fourth obstacle list are merged, and the obstacle and obstacles near the obstacle are marked as "merged". When there is no matching obstacle in the fourth obstacle attribute list, the obstacle is marked as "merged". The third obstacle attribute list is the obstacle attribute list after deduplication and fusion of C perception groups in M ​​perception groups in any calculation group, and the fourth obstacle attribute list is the obstacle attribute list after deduplication and fusion of D perception groups. M is an integer greater than or equal to 2, C is an integer greater than or equal to 1, D is an integer greater than or equal to 1, and C+D=M.

[0076] Specifically, when M equals 2, meaning there are two perception groups within a single calculation group, the obstacle attribute list corresponding to one perception group is the third obstacle attribute list, and the obstacle attribute list corresponding to the other perception group is the fourth obstacle attribute list. For a detailed description of the process of fusing the two obstacle attribute lists, please refer to the above description of the process of fusing two obstacle attribute lists when there are two sensors within a perception group (i.e., N equals 2). The principle is similar and will not be described in detail here.

[0077] When M is greater than or equal to 3, that is, when there are 3 or more perception groups in a calculation group, the obstacle attribute lists corresponding to any two perception groups can be merged first. Then, the merged obstacle attribute list can be merged with the obstacle attribute list corresponding to any of the remaining perception groups until all the obstacle attribute lists corresponding to all perception groups are merged into a single obstacle attribute list. Alternatively, the obstacle attribute lists of multiple perception groups can be merged pairwise first, and then the merged obstacle attribute lists can be merged pairwise again until a single obstacle attribute list is obtained. Here, the two obstacle attribute lists merged in the last step can be used as the third and fourth obstacle attribute lists, respectively. Of course, other methods can also be used, as long as they can achieve the merging of all obstacle attribute lists into a single obstacle attribute list. This application does not limit this method. Specifically, the merging method of any two obstacle attribute lists is the same as the principle of merging two obstacle attribute lists when N equals 2, as described above. They can be referred to each other, and will not be described in detail here.

[0078] Based on the above method, an obstacle attribute list corresponding to each calculation group can be obtained, which is also the first deduplication result of each calculation group, achieving the purpose of deduplication between perception groups. In an optional implementation, the controller performs a second deduplication process on the first deduplication results corresponding to the multiple calculation groups. The specific method is as follows: the controller starts from the first unmarked fused obstacle in the fifth obstacle attribute list until all obstacles in the fifth obstacle attribute list are marked as "fused"; it matches them with obstacles in the sixth obstacle attribute list respectively; when there is a matching obstacle in the sixth obstacle attribute list, the attribute data of the obstacle in the fifth obstacle list and the sixth obstacle list are merged, and the obstacle data of the obstacles near the obstacle are merged in the fifth obstacle list and the sixth obstacle list. The attribute data in the object list are merged, and the obstacle and the obstacles near the obstacle are marked as "merged"; when there is no matching obstacle in the sixth obstacle attribute list, the obstacle is marked as "merged"; wherein, the fifth obstacle attribute list is the obstacle attribute list after merging E obstacle attribute lists corresponding to L calculation groups, the sixth obstacle attribute list is the obstacle attribute list after merging F obstacle attribute lists, L is the total number of calculation groups, L is an integer greater than or equal to 2, E is an integer greater than or equal to 1, F is an integer greater than or equal to 1, and E+F=L.

[0079] Specifically, when L equals 2, there are two calculation groups. One calculation group corresponds to the fifth obstacle attribute list, and the other calculation group corresponds to the sixth obstacle attribute list. For a detailed description of the fusion process of the two obstacle attribute lists, please refer to the above description of the fusion process of the two obstacle attribute lists when there are two sensors in the perception group (i.e., N equals 2). The principle is similar and will not be described in detail here.

[0080] When L is greater than or equal to 3, that is, when there are 3 or more calculation groups, the obstacle attribute lists corresponding to any two calculation groups can be merged first. Then, the merged obstacle attribute list is merged with the obstacle attribute list corresponding to any calculation group in the remaining perception groups, until the obstacle attribute lists corresponding to all calculation groups are merged into a single obstacle attribute list. Alternatively, the obstacle attribute lists from multiple calculation groups can be merged pairwise first, and then the merged obstacle attribute lists can be merged pairwise again until a single obstacle attribute list is obtained. Here, the two obstacle attribute lists merged in the last step can be used as the fifth and sixth obstacle attribute lists, respectively. Of course, other methods can also be used, as long as they can achieve the merging of all obstacle attribute lists into a single obstacle attribute list; this application does not limit this. Specifically, the method for merging any two obstacle attribute lists is the same as the principle for merging two obstacle attribute lists when N equals 2, as described above, and can be referred to accordingly; it will not be described in detail here.

[0081] Based on the above method, the purpose of deduplication between calculation groups is achieved, and a list of obstacle attributes corresponding to the vehicle can be obtained. Then, the detection results of the target obstacle can be obtained, so as to perform vehicle trajectory planning.

[0082] Step 403: The controller plans the vehicle's trajectory based on the detection results of the target obstacle.

[0083] Specifically, the detection results of the target obstacle include data on all obstacles around the intelligent vehicle, and no obstacles are repeated. This allows for more accurate obstacle data to be referenced when planning the vehicle's trajectory, resulting in more accurate trajectory planning and improved safety.

[0084] Using the above method, during the operation of the intelligent vehicle, the controller can deduplicate obstacle data collected by multiple sensors, thereby obtaining effective target obstacle detection results. This allows for trajectory planning based on the target obstacle detection results. This solves the current problem of ineffective obstacle detection. It also addresses the issue of inaccurate trajectory planning when multiple sensors collect overlapping obstacle data in multi-sensor scenarios. In the aforementioned deduplication process, a grouping method can be proposed based on the overlap between the sensing ranges of the sensors. The sensors are divided into sensing groups, which are then further subdivided. Simultaneously, data within each sensing group is deduplicated, followed by data deduplication between sensing groups, and finally, deduplication between groups is calculated. This deduplication process ensures a significant deduplication effect, improves deduplication accuracy, and makes the deduplication results more precise. This leads to more accurate trajectory planning based on the deduplication results, enhancing the driving safety of intelligent vehicles. Furthermore, the deduplication process can consider the relationship with nearby obstacles, reducing steps. Since multiple sensing groups are divided, obstacle deduplication accuracy can be achieved by first deduplicating data within each group and then between groups. Simultaneous processing of data from multiple sensing groups improves the efficiency of deduplication for multiple obstacles. Moreover, the method in this application considers obstacle data collected by all sensors inside the vehicle, and the comprehensive sensing range can cover the entire panoramic range of the intelligent vehicle, providing more comprehensive obstacle data and facilitating autonomous driving decisions.

[0085] It should be noted that the trajectory planning method of this application is applicable to... Figure 2 When the system is shown, the controller 101 executes the grouping operations in steps 401 and 403 and step 402 above. The perception and computing unit performs the first deduplication process on the obstacle data collected by the sensors in a computing group. Since the principle is the same, it will not be described in detail here.

[0086] Based on the above embodiments, the following describes a specific example of an obstacle detection method provided in this application. Figure 3 For example, suppose the types and distribution of multiple sensors in a smart car are as follows: Figure 3 As shown. See also Figure 11 As shown, the specific execution flow of the controller can be as follows: Step 1101: The controller acquires obstacle data collected by multiple sensors.

[0087] Specifically, in this example, the obstacle data acquired by the controller from multiple sensors includes images acquired by cameras 0-5 and point clouds acquired by lidar a-lidar e.

[0088] Step 1102: The controller, based on the sensing range of the multiple sensors, groups a camera whose sensing range overlaps with that of the LiDAR sensor into one sensing group, ultimately obtaining the following six sensing groups: Perception group (1): Camera 0 and LiDAR a; Perception group (2): Camera 3 and LiDAR b; Perception group (3): Camera 1 and LiDAR a; Perception group (4): Camera 4 and LiDAR d; Perception group (5): Camera 2 and LiDAR c; Perception group (6): Camera 5 and LiDAR e.

[0089] Step 1103: The controller divides the six sensing groups into three calculation groups according to their sensing range. The sensing group (1) and the sensing group (2) are divided into one calculation group, the sensing group (3) and the sensing group (5) are divided into one calculation group, and the sensing group (4) and the sensing group (6) are divided into one calculation group.

[0090] Then, the controller performs deduplication on the data in the calculation group composed of perception group (1) and perception group (2) through perception computing unit A, deduplication on the data in the calculation group composed of perception group (3) and perception group (5) through perception computing unit B, and deduplication on the data in the calculation group composed of perception group (4) and perception group (6) through perception computing unit C. The above three processes are performed simultaneously. The following explanation continues the above steps using the calculation process performed by the controller through perception computing unit A as an example: The controller performs deduplication on the point cloud of lidar a and the data of camera 0 in perception group (1) through perception computing unit A, specifically executing: Step 1104: The perception and calculation unit A of the controller determines whether the obstacle data is an image. If it is, proceed to step 1105; otherwise, proceed to step 1106.

[0091] Step 1105: The controller uses the Mask R-CNN model to perform instance segmentation on the image through the perception computing unit A, and then executes step 1108.

[0092] Step 1106: The controller projects the point cloud onto a two-dimensional RGB image through the perception computing unit A.

[0093] For a detailed description of the projection process, please refer to step 402 above, which describes the point cloud projection process. It will not be elaborated here.

[0094] Step 1107: The controller performs instance segmentation on the projected two-dimensional RGB image through the perception computing unit A.

[0095] Step 1108: The controller obtains the obstacle attribute list corresponding to the LiDAR or the obstacle attribute list corresponding to the camera based on the instance segmentation results through the perception computing unit A.

[0096] Step 1109: The controller selects the first unmarked fused obstacle in the obstacle attribute list corresponding to the camera through the perception computing unit A, and matches it with the obstacle in the obstacle attribute list corresponding to the lidar.

[0097] Step 1110: The controller determines whether the difference between the timestamps is within a preset range through the sensing and computing unit A, for example, whether it is less than or equal to 0.4 milliseconds (ms). If yes, proceed to step 1112; otherwise, proceed to step 1111.

[0098] Step 1111: The controller marks the object as fused by the perception computing unit A, or marks it as not matching an obstacle, or other marking forms, and then executes step 1115.

[0099] Step 1112: The controller determines whether the obstacle types are the same through the perception and calculation unit A. If so, it executes step 1113; otherwise, it executes step 1111.

[0100] Step 1113: The controller determines whether the obstacle position deviation is within a preset range through the sensing and computing unit A. If so, step 1114 is executed; otherwise, step 1111 is executed.

[0101] Step 1114: The controller calculates whether the area of ​​the obstacle rectangle reaches the preset overlap degree based on the size (length and width) of the obstacle through the sensing and computing unit A. If yes, then step 1115 is executed; otherwise, step 1111 is executed.

[0102] Step 1115: The controller fuses the attribute data of the matched obstacles and the nearby obstacles through the perception computing unit A, and marks them as fused.

[0103] For example, Figure 9 The system will merge the matched obstacles a1 and b3, as well as the attached obstacles a2 and b2.

[0104] Step 1116: The controller uses the perception and computing unit A to determine whether all obstacles in the obstacle attribute list corresponding to the camera are marked. If so, proceed to step 1117; otherwise, proceed to step 1109.

[0105] Step 1117: The controller adds the attribute data of unmarked obstacles from the obstacle attribute list corresponding to the lidar to the fused obstacle attribute list through the perception computing unit A.

[0106] The list of obstacles after fusion is the list of obstacle attributes corresponding to perception group (1).

[0107] Similarly, the process of the controller performing deduplication on the point cloud of the lidar b and the data of the camera 3 in the perception group (2) through the perception computing unit A to obtain the obstacle attribute list corresponding to the perception group (2) can also be referred to in the above steps 1104-1117, and will not be described in detail here.

[0108] Step 1118: The controller uses the perception computing unit A to determine whether the obstacle attribute lists in all perception groups (i.e., perception group (1) and perception group (2)) have been fused. If so, step 1119 is executed; otherwise, step 1109 is executed.

[0109] Step 1119: The controller projects the obstacles corresponding to each perception group onto the vehicle's three-dimensional coordinate system through the perception computing unit A, and determines the obstacle perception range of each perception group based on the coordinates of the obstacle's center point.

[0110] For details on the projection process, please refer to step 402 above, which will not be described in detail here.

[0111] Step 1120: The controller determines the overlap range of the two sensing groups through the sensing computing unit A.

[0112] Step 1121: The controller performs deduplication processing on the obstacle attribute lists corresponding to two overlapping perception groups through the perception computing unit A.

[0113] Specifically, the deduplication and fusion process of the obstacle attribute lists between the two perception groups is based on the same principle as the fusion process of the obstacle attribute lists corresponding to the two sensors described above. Please refer to steps 1109-1117 above. The difference is that the obstacle attribute lists corresponding to the camera and the LiDAR mentioned in the above steps are the obstacle attribute lists corresponding to the two perception groups, respectively. The result is the obstacle attribute list corresponding to the calculation group composed of the two perception groups, which will not be described in detail here.

[0114] Similarly, the method by which the controller performs deduplication processing on the data in the calculation group composed of the perception group (3) and the perception group (5) through the perception computing unit B, and on the method by which the controller performs deduplication processing on the data in the calculation group composed of the perception group (4) and the perception group (6) through the perception computing unit C, is the same as the method by which the controller performs deduplication processing on the point cloud of the lidar a and the data of the camera 0 in the perception group (1) through the perception computing unit A. They can be referred to each other, and will not be described in detail here.

[0115] Step 1122: The controller determines whether the obstacle attribute list fusion between all perception groups in all calculation groups has been completed. If yes, then proceed to step 1123; otherwise, proceed to step 1121.

[0116] Step 1123: The controller fuses the obstacle attribute list obtained by the perception computing unit A and the obstacle attribute list obtained by the perception computing unit B to obtain the seventh obstacle attribute list.

[0117] Step 1124: The controller fuses the seventh obstacle attribute list with the obstacle attribute list processed by the perception computing unit C to obtain the detection result of the target obstacle.

[0118] Specifically, the fusion process in steps 1123 and 1124 is based on the same principle as the process of deduplicating the obstacle attribute lists corresponding to the two sensors. Please refer to the process in steps 1109-1117 above, which will not be described in detail here.

[0119] In one optional implementation, the controller may first fuse the obstacle attribute list obtained by the perception computing unit A and the obstacle attribute list obtained by the perception computing unit C, and then fuse them with the obstacle attribute list obtained by the perception computing unit B; or the controller may first fuse the obstacle attribute list obtained by the perception computing unit B and the obstacle attribute list obtained by the perception computing unit C, and then fuse them with the obstacle attribute list obtained by the perception computing unit A. This application does not limit the scope of the implementation.

[0120] Step 1225: The controller plans the vehicle's trajectory based on the detection results of the target obstacle.

[0121] Using the above method, during the operation of an intelligent vehicle, the controller can deduplicate obstacle data collected by multiple sensors, thereby obtaining effective target obstacle detection results. This allows for trajectory planning based on the target obstacle detection results. This solves the current problem of ineffective obstacle detection. It also addresses the issue of inaccurate trajectory planning when multiple sensors collect overlapping obstacle data in multi-sensor scenarios. In the deduplication process, a grouping method can be proposed based on the overlap between sensor sensing ranges. The sensors are divided into sensing groups, and deduplication is performed simultaneously within each sensing group, then between sensing groups, and finally between groups. This deduplication process ensures significant deduplication effectiveness, improves deduplication accuracy, and makes the deduplication results more precise. This results in more accurate trajectory planning based on the deduplication results, improving the driving safety of the intelligent vehicle. Furthermore, because multiple sensing groups are used, the accuracy of obstacle deduplication can be achieved by first deduplicating data within each group and then between groups. Simultaneous processing of data from multiple sensing groups improves the efficiency of deduplication for multiple obstacles. Furthermore, the method of this application takes into account obstacle data collected by all sensors in the vehicle, and the comprehensive perception range can cover the panoramic range of the intelligent vehicle, thereby providing more comprehensive obstacle data and being more conducive to the autonomous driving decision-making of the intelligent vehicle.

[0122] The above text combines Figures 1 to 11 The method for trajectory planning provided in this application is described in detail below. Figure 12 and Figure 13 This application describes the trajectory planning apparatus, controller, and intelligent vehicle provided in this application.

[0123] Figure 12 A schematic diagram of a trajectory planning device provided in this application is shown in the figure. The trajectory planning device 1200 may include an acquisition unit 1201 and a detection unit 1202. Specifically: The acquisition unit 1201 is used to acquire obstacle data collected by multiple sensors; the multiple sensors include at least two types of sensors; the detection unit 1202 is used to perform deduplication processing on the obstacle data collected by the multiple sensors to obtain the detection result of the target obstacle, and to perform vehicle trajectory planning based on the detection result of the target obstacle.

[0124] Optionally, when the detection unit 1202 performs deduplication processing on the obstacle data collected by the plurality of sensors to obtain the detection result of the target obstacle, it specifically performs the following steps: Dividing the plurality of sensors into multiple sensing groups based on their sensing ranges; wherein each sensing group contains at least two sensors of at least two different types, and the sensing ranges of the sensors in any sensing group overlap; the total sensing range of all sensors in all sensing groups covers the panoramic range of the intelligent vehicle; dividing the plurality of sensing groups into multiple calculation groups based on the maximum range formed by the sensing ranges of the plurality of sensing groups; wherein each calculation group contains at least two sensing groups, and the maximum ranges of the at least two sensing groups overlap; performing a first deduplication processing on the obstacle data collected by the sensors in each calculation group; and performing a second deduplication processing on the first deduplication result corresponding to the plurality of calculation groups to obtain a second deduplication result.

[0125] Optionally, when the detection unit 1202 performs the first deduplication processing on the obstacle data collected by the sensors in any one calculation group, it is specifically used to: perform the first deduplication processing on the obstacle data collected by the sensors in each perception group in the any one calculation group; and perform the second deduplication processing on the first deduplication results corresponding to at least two perception groups in the any one calculation group to obtain the second deduplication result corresponding to the any one calculation group, wherein the second deduplication result includes the detection result of the target obstacle.

[0126] Optionally, when the detection unit 1202 performs the first deduplication process on the obstacle data collected by the sensors in any one perception group, it is specifically used to: generate an obstacle attribute list corresponding to each sensor based on the obstacle data collected by each sensor in the any one perception group, wherein the obstacle attribute list corresponding to each sensor contains the attribute of at least one obstacle; and fuse the obstacle attribute lists corresponding to all sensors in the any one perception group to obtain the obstacle attribute list corresponding to the any one perception group.

[0127] Optionally, when generating the obstacle attribute list corresponding to each sensor based on the obstacle data collected by each sensor, the detection unit 1202 is specifically configured to: when the sensor is a lidar, the obstacle data collected by the lidar is point cloud data, and the point cloud data is projected onto a two-dimensional RGB image based on preset virtual camera parameters; perform instance segmentation on the two-dimensional RGB image obtained after projection to obtain the obstacle attribute list corresponding to the lidar; when the sensor is a camera, the obstacle data collected by the camera is a two-dimensional RGB image, and perform instance segmentation on the two-dimensional RGB image to obtain the obstacle attribute list corresponding to the camera.

[0128] Optionally, when the detection unit 1202 performs instance segmentation on any two-dimensional RGB image to obtain an obstacle attribute list, it is specifically used to: input the two-dimensional RGB image into a preset instance segmentation model to obtain an instance segmentation result; and generate an obstacle attribute list based on the instance segmentation result.

[0129] Optionally, when the detection unit 1202 fuses the obstacle attribute lists corresponding to all sensors in any one of the perception groups, it is specifically configured to: starting from the first unmarked fused obstacle in the first obstacle attribute list, and continuing until all obstacles in the first obstacle attribute list are marked as "fused", match them with obstacles in the second obstacle attribute list; when a matching obstacle exists in the second obstacle attribute list, merge the attribute data of the obstacle in the first obstacle list and the second obstacle list, and merge the attribute data of obstacles near the obstacle in the first obstacle list and the second obstacle list. The attribute data in the object list are merged, and the obstacle and the obstacles near the obstacle are marked as "merged"; when there is no matching obstacle in the second obstacle attribute list, the obstacle is marked as "merged"; wherein, the first obstacle attribute list is the obstacle attribute list after merging A obstacle attribute lists corresponding to N sensors in any perception group, the second obstacle attribute list is the obstacle attribute list after merging B obstacle attribute lists, N is an integer greater than or equal to 2, A is an integer greater than or equal to 1, B is an integer greater than or equal to 1, and A+B=N.

[0130] Optionally, when the detection unit 1202 performs a second deduplication process on the first deduplication results corresponding to at least two perception groups in any one calculation group to obtain the second deduplication result corresponding to the any one calculation group, it is specifically used to: determine the perception range corresponding to each perception group based on the position of the obstacle in the first deduplication result corresponding to each perception group; determine the overlap range between the at least two perception groups based on the perception range corresponding to each perception group; and perform a second deduplication process on the first deduplication results corresponding to the at least two perception groups with overlapping ranges.

[0131] Optionally, when performing second deduplication processing on the first deduplication results corresponding to at least two perception groups with overlapping ranges, the detection unit 1202 is specifically configured to: starting from the first unmarked fused obstacle in the third obstacle attribute list, and continuing until all obstacles in the third obstacle attribute list are marked as "fused", match them with obstacles in the fourth obstacle attribute list; when there is a matching obstacle in the fourth obstacle attribute list, merge the attribute data of the obstacle in the third obstacle list and the fourth obstacle list, and merge the attribute data of obstacles near the obstacle in the third obstacle list and the fourth obstacle list. The attribute data in the obstacle list are merged, and the obstacle and the obstacles near the obstacle are marked as "merged". When there is no matching obstacle in the fourth obstacle attribute list, the obstacle is marked as "merged". The third obstacle attribute list is the obstacle attribute list after deduplication and fusion of the C perception groups in M ​​perception groups in any calculation group, and the fourth obstacle attribute list is the obstacle attribute list after deduplication and fusion of the D perception groups. M is an integer greater than or equal to 2, C is an integer greater than or equal to 1, D is an integer greater than or equal to 1, and C+D=M.

[0132] Optionally, when performing the second deduplication processing on the first deduplication results corresponding to the plurality of calculation groups, the detection unit 1202 is specifically configured to: starting from the first unmarked fused obstacle in the fifth obstacle attribute list, until all obstacles in the fifth obstacle attribute list are marked as "fused", match them with obstacles in the sixth obstacle attribute list respectively; when there is a matching obstacle in the sixth obstacle attribute list, merge the attribute data of the obstacle in the fifth obstacle list and the sixth obstacle list, and merge the attribute data of obstacles near the obstacle in the fifth obstacle list and the sixth obstacle list. The attribute data are merged, and the obstacle and obstacles near the obstacle are marked as "merged". When there is no matching obstacle in the sixth obstacle attribute list, the obstacle is marked as "merged". The fifth obstacle attribute list is the obstacle attribute list after merging the obstacle attribute lists of E obstacle attribute lists corresponding to L calculation groups, and the sixth obstacle attribute list is the obstacle attribute list after merging the obstacle attribute lists of F obstacle attribute lists. L is the total number of calculation groups, L is an integer greater than or equal to 2, E is an integer greater than or equal to 1, F is an integer greater than or equal to 1, and E+F=L.

[0133] Optionally, a matching obstacle exists if the following conditions are met: the difference between the timestamps of the two obstacles is less than a preset range, the two obstacles belong to the same obstacle type, the positional deviation of the two obstacles is within a preset range, and the border areas of the two obstacles overlap.

[0134] It should be understood that the trajectory planning device 1200 in this application embodiment can be implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. It can also be implemented by software. Figure 4 or Figure 11 When illustrating the trajectory planning method, the trajectory planning device 1200 and its various modules can also be software modules.

[0135] The trajectory planning apparatus 1200 according to the embodiments of this application can correspond to performing the method described in the embodiments of this application, and the above and other operations and / or functions of each unit in the trajectory planning apparatus 1200 are respectively for implementing Figure 4 or Figure 11 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0136] The trajectory planning device 1200 described above enables the deduplication of obstacle data collected by multiple sensors during the operation of an intelligent vehicle, thereby obtaining effective target obstacle detection results. This allows for trajectory planning of the intelligent vehicle based on the target obstacle detection results. This solves the current problem of ineffective obstacle detection. It also addresses the issue of inaccurate trajectory planning when obstacles collected by multiple sensors overlap in multi-sensor scenarios. In the aforementioned deduplication process, a grouping method can be proposed based on the overlap between the sensing ranges of the sensors. The sensors are divided into sensing groups, which are then further subdivided. Simultaneously, data within each sensing group is deduplicated, followed by data deduplication between sensing groups, and finally, deduplication between groups is calculated. This deduplication process ensures a significant deduplication effect, improves deduplication accuracy, and makes the deduplication results more precise. This leads to more accurate trajectory planning based on the deduplication results, enhancing the driving safety of intelligent vehicles. Furthermore, the deduplication process can consider the relationship with nearby obstacles, reducing steps. Since multiple sensing groups are divided, obstacle deduplication accuracy can be achieved by first deduplicating data within each group and then between groups. Simultaneous processing of data from multiple sensing groups improves the efficiency of deduplication for multiple obstacles. Moreover, the method in this application considers obstacle data collected by all sensors inside the vehicle, and the comprehensive sensing range can cover the entire panoramic range of the intelligent vehicle, providing more comprehensive obstacle data and facilitating autonomous driving decisions.

[0137] Figure 13 This is a schematic diagram of the structure of a controller provided in an embodiment of this application. The controller is applied to, for example... Figure 2 or Figure 3 The system shown is used to implement, for example Figure 4 or Figure 11 The trajectory planning method is shown. (See also...) Figure 13 As shown, the controller 1300 may include a processor 1301, a memory 1302, and a bus 1303. The processor 1301 and the memory 1302 communicate via the bus 1303, or via other means such as wireless transmission. The memory 1302 stores instructions, and the processor 1301 executes the instructions stored in the memory 1302. The memory 1302 stores program code, and the processor 1301 can call the program code stored in the memory 1302 to perform the following operations: Obstacle data collected by multiple sensors, including at least two types of sensors, is acquired; the obstacle data collected by the multiple sensors is deduplicated to obtain the detection results of the target obstacle; and vehicle trajectory planning is performed based on the detection results of the target obstacle.

[0138] Optionally, Figure 13The controller 1300 shown also includes memory 1304 and a communication interface 1303. Memory 1304 may be physically integrated with processor 1301, or exist within processor 1301, or as a separate unit. Computer program 13042 may be stored in memory 1304 or memory 1302. Optionally, computer program code 13042 stored in memory 1304 (e.g., a program to be debugged) is copied to memory 1304 and then executed by processor 1301. Kernel 13041 may also run in memory 1304.

[0139] It should be understood that, in the embodiments of this application, the processor 1301 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), or programmable logic devices (PLDs). The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Alternatively, the processor 1301 may be other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0140] Optionally, Figure 13 The controller shown may also include a processor 1306. Processors 1301 and 1306 may be processors of the same type or different types. The number of processor cores in each processor does not constitute a limitation on this application.

[0141] The memory 1302 may include read-only memory and random access memory, and provides instructions, programs, and data to the processor 1301. For example, the program may include program code that includes computer operation instructions. The memory 1302 may also include non-volatile random access memory. For example, the memory 1302 may also store device type information.

[0142] The memory 1302 can be volatile memory or non-volatile memory, or it can include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0143] In addition to the data bus, bus 1303 may also include an address bus, power bus, control bus, and status signal bus. Bus 1303 can be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, a Controller Area Network (CAN) bus, an automotive Ethernet bus, or other internal bus implementations. Figure 13 The connections of the various devices / equipment shown are illustrated. However, for clarity, [the following is omitted]. Figure 13 All buses are labeled as Bus 1303. For ease of representation, Figure 13 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0144] It should be understood that the controller 1300 according to the embodiments of this application may correspond to the trajectory planning device 1200 in the embodiments of this application, and may correspond to the execution Figure 4 or Figure 11The controller in the method shown is the main operation step, and the above and other operations and / or functions of each module in the controller 1300 are respectively for implementing Figure 4 or Figure 11 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0145] The controller 1300 described above enables the deduplication of obstacle data collected by multiple sensors during the operation of the intelligent vehicle, thereby obtaining effective target obstacle detection results. This allows for trajectory planning of the intelligent vehicle based on the target obstacle detection results. This solves the current problem of ineffective obstacle detection. It also addresses the issue of inaccurate trajectory planning when obstacles collected by multiple sensors overlap in multi-sensor scenarios. In the aforementioned deduplication process, a grouping method can be proposed based on the overlap between the sensing ranges of the sensors. The sensors are divided into sensing groups, which are then further subdivided. Simultaneously, data within each sensing group is deduplicated, followed by data deduplication between sensing groups, and finally, deduplication between groups is calculated. This deduplication process ensures a significant deduplication effect, improves deduplication accuracy, and makes the deduplication results more precise. This leads to more accurate trajectory planning based on the deduplication results, enhancing the driving safety of intelligent vehicles. Furthermore, the deduplication process can consider the relationship with nearby obstacles, reducing steps. Since multiple sensing groups are divided, obstacle deduplication accuracy can be achieved by first deduplicating data within each group and then between groups. Simultaneous processing of data from multiple sensing groups improves the efficiency of deduplication for multiple obstacles. Moreover, the method in this application considers obstacle data collected by all sensors inside the vehicle, and the comprehensive sensing range can cover the entire panoramic range of the intelligent vehicle, providing more comprehensive obstacle data and facilitating autonomous driving decisions.

[0146] This application also provides an intelligent vehicle, which may include the controller or trajectory planning device mentioned above.

[0147] This application also provides an in-vehicle system that includes multiple sensors and controllers, or multiple sensors, controllers and multiple perception computing units.

[0148] This application also provides a trajectory planning system, which includes at least one intelligent vehicle. The various components or devices described above are respectively used to perform the above... Figure 4 or Figure 11 The operational steps of the corresponding execution entity in the method shown are not repeated here for the sake of brevity.

[0149] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive (SSD).

[0150] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0151] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0152] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the technical objectives of this application, depending on actual needs.

[0153] The above description is merely a specific embodiment of this application. Any variations or substitutions conceived by those skilled in the art based on the specific embodiments provided in this application should be covered within the protection scope of this application.

Claims

1. A trajectory planning method, characterized in that, The method is implemented by the vehicle's controller, and the method includes: Obstacle data are acquired from multiple sensors; the multiple sensors include at least two types of sensors; The obstacle data collected by the multiple sensors is deduplicated to obtain the detection results of the target obstacle; The vehicle's trajectory is planned based on the detection results of the target obstacle.

2. The method as described in claim 1, characterized in that, The obstacle data collected by the multiple sensors is deduplicated to obtain the detection results of the target obstacle, including: Based on the sensing range of the multiple sensors, the multiple sensors are divided into multiple sensing groups; each sensing group contains at least two sensors, the at least two sensors are of at least two types, and the sensing ranges of the sensors in any sensing group overlap; the total sensing range of all sensors in all sensing groups covers the panoramic range of the intelligent vehicle. Based on the maximum range formed by the sensing ranges of the multiple sensing groups, the multiple sensing groups are divided into multiple calculation groups; wherein any calculation group contains at least two sensing groups, and the maximum ranges of the at least two sensing groups overlap. The obstacle data collected by the sensors in each computing group are subjected to the first deduplication process. The first deduplication results corresponding to the multiple calculation groups are subjected to a second deduplication process to obtain a second deduplication result.

3. The method as described in claim 1 or 2, characterized in that, The obstacle data collected by the sensors in any computing group undergoes a first deduplication process, including: The first deduplication process is performed on the obstacle data collected by the sensors in each perception group of any one of the calculation groups; The first deduplication result corresponding to at least two perception groups in any one calculation group is subjected to the second deduplication process to obtain the second deduplication result corresponding to the calculation group. The second deduplication result includes the detection result of the target obstacle.

4. The method according to any one of claims 1-3, characterized in that, The obstacle data collected by the sensors in any perception group undergoes a first deduplication process, including: Based on the obstacle data collected by each sensor in any of the perception groups, an obstacle attribute list corresponding to each sensor is generated, and the obstacle attribute list corresponding to each sensor contains at least one obstacle attribute. The obstacle attribute lists corresponding to all sensors in any given sensing group are fused to obtain the obstacle attribute list corresponding to any given sensing group.

5. The method according to any one of claims 1-4, characterized in that, Based on the obstacle data collected by each sensor, a list of obstacle attributes corresponding to each sensor is generated, including: When the sensor is a lidar, the obstacle data collected by the lidar is point cloud data. The point cloud data is projected onto a two-dimensional RGB image based on preset virtual camera parameters. The two-dimensional RGB image obtained after projection is segmented into instances to obtain the obstacle attribute list corresponding to the lidar. When the sensor is a camera, the obstacle data collected by the camera is a two-dimensional RGB image. The two-dimensional RGB image is segmented into instances to obtain a list of obstacle attributes corresponding to the camera.

6. The method according to any one of claims 1-5, characterized in that, Perform instance segmentation on any 2D RGB image to obtain a list of obstacle attributes, including: Input a 2D RGB image into a preset instance segmentation model to obtain instance segmentation results; An obstacle attribute list is generated based on the instance segmentation results.

7. The method according to any one of claims 1-6, characterized in that, The obstacle attribute lists corresponding to all sensors in any given perception group are fused, including: Starting from the first unmarked fused obstacle in the first obstacle attribute list, and continuing until all obstacles in the first obstacle attribute list are marked as "fused", each obstacle is matched with an obstacle in the second obstacle attribute list. When a matching obstacle exists in the second obstacle attribute list, the attribute data of the obstacle in the first obstacle list and the second obstacle list are merged, and the attribute data of obstacles near the obstacle in the first obstacle list and the second obstacle list are merged, and the obstacle and the obstacles near the obstacle are marked as "merged". If no matching obstacle exists in the second obstacle attribute list, the obstacle is marked as "fused". Wherein, the first obstacle attribute list is the obstacle attribute list fused from the obstacle attribute lists of A obstacles corresponding to N sensors in any one perception group, and the second obstacle attribute list is the obstacle attribute list fused from the obstacle attribute lists of B obstacles. N is an integer greater than or equal to 2, A is an integer greater than or equal to 1, B is an integer greater than or equal to 1, and A+B=N.

8. The method according to any one of claims 1-7, characterized in that, Perform a second deduplication process on the first deduplication results corresponding to at least two perception groups in any given computation group to obtain the second deduplication result corresponding to any given computation group, including: The sensing range of each sensing group is determined based on the position of the obstacle in the first deduplication result corresponding to each sensing group. The overlap range between the at least two perception groups is determined based on the perception range corresponding to each perception group. A second deduplication process is performed on the first deduplication results corresponding to at least two sensing groups with overlapping ranges.

9. The method according to any one of claims 1-8, characterized in that, Perform a second deduplication process on the first deduplication results corresponding to at least two sensor groups with overlapping ranges, including: Starting from the first unmarked merged obstacle in the third obstacle attribute list, and continuing until all obstacles in the third obstacle attribute list are marked as "merged", each obstacle is matched with an obstacle in the fourth obstacle attribute list. When a matching obstacle exists in the fourth obstacle attribute list, the attribute data of the obstacle in the third obstacle list and the fourth obstacle list are merged, and the attribute data of obstacles near the obstacle in the third obstacle list and the fourth obstacle list are merged, and the obstacle and the obstacles near the obstacle are marked as "merged". When there is no matching obstacle in the fourth obstacle attribute list, the obstacle is marked as "fused". The third obstacle attribute list is the obstacle attribute list after deduplication and fusion of the C perception groups in the M perception groups of any calculation group, and the fourth obstacle attribute list is the obstacle attribute list after deduplication and fusion of the D perception groups, where M is an integer greater than or equal to 2, C is an integer greater than or equal to 1, D is an integer greater than or equal to 1, and C+D=M.

10. The method according to any one of claims 1-9, characterized in that, The second deduplication process is performed on the first deduplication results corresponding to the plurality of calculation groups, including: Starting from the first unmarked merged obstacle in the fifth obstacle attribute list, and continuing until all obstacles in the fifth obstacle attribute list are marked as "merged", each obstacle is matched with an obstacle in the sixth obstacle attribute list. When a matching obstacle exists in the sixth obstacle attribute list, the attribute data of the obstacle in the fifth obstacle list and the sixth obstacle list are merged, and the attribute data of obstacles near the obstacle in the fifth obstacle list and the sixth obstacle list are merged, and the obstacle and the obstacles near the obstacle are marked as "merged". When there is no matching obstacle in the sixth obstacle attribute list, the obstacle is marked as "fused". Wherein, the fifth obstacle attribute list is the obstacle attribute list after merging the obstacle attribute lists of E obstacles corresponding to L calculation groups, and the sixth obstacle attribute list is the obstacle attribute list after merging the obstacle attribute lists of F obstacles. L is the total number of calculation groups, L is an integer greater than or equal to 2, E is an integer greater than or equal to 1, F is an integer greater than or equal to 1, and E+F=L.

11. The method according to any one of claims 1-10, characterized in that, A matching obstacle exists if the following conditions are met: The difference between the timestamps of the two obstacles is less than a preset range, the two obstacles belong to the same obstacle type, the positional deviation of the two obstacles is within a preset range, and the border areas of the two obstacles overlap.

12. A trajectory planning device, characterized in that, A device applied to a vehicle includes an acquisition unit and a detection unit, wherein: The acquisition unit is used to acquire obstacle data collected by multiple sensors; the multiple sensors include at least two types of sensors; The detection unit is used to deduplicate the obstacle data collected by the multiple sensors to obtain the detection result of the target obstacle, and to plan the trajectory of the vehicle based on the detection result of the target obstacle.

13. A controller, characterized in that, The system includes a processor and a memory, wherein the memory stores computer program instructions, and when the controller is running, the processor executes the computer program instructions stored in the memory to implement the operational steps of the method described in any one of claims 1 to 11.

14. An intelligent vehicle, characterized in that, The intelligent vehicle includes a controller for performing the functions of the controller as described in claim 13.

Images