Target object detection method and device, vehicle and storage medium

Abstract

The present disclosure relates to the technical field of automatic driving, and particularly relates to a target object detection method and device, a vehicle and a storage medium. An environment image of an environment where a vehicle is currently located can be acquired. An image detection result of a target object in the environment image is determined according to the environment image through a target detection model. The image detection result includes a first relative position of the target object and the vehicle. Radar point cloud data acquired by the vehicle is subjected to data screening according to the image detection result, so as to obtain target point cloud data corresponding to the target object. A target detection result is determined according to the target point cloud data. The target detection result includes a second relative position of the target object and the vehicle. Target object detection according to the screened target point cloud data can reduce the calculation complexity of a laser point cloud target detection algorithm and improve the detection efficiency. At the same time, compared with a pure visual image perception scheme, the positioning accuracy of the target object can be significantly improved.

Description

Technical Field

[0001] This disclosure relates to the field of autonomous driving technology, and in particular to a target object detection method, device, vehicle, and storage medium. Background Technology

[0002] The implementation of autonomous driving technology relies heavily on advanced perception algorithms. One autonomous driving solution uses camera images as the input to the perception algorithm. Camera images are consistent with the world perceived by the human eye; theoretically, autonomous driving technology can be developed based solely on camera images, mimicking human driving. Furthermore, with the development of high-resolution, high-spectral-rate camera imaging technology, the environmental information captured by cameras will be richer, and cameras are relatively inexpensive, allowing for widespread deployment around the vehicle to ensure comprehensive perception. Another autonomous driving solution uses LiDAR point clouds as the input to the perception algorithm. LiDAR point clouds can reconstruct the three-dimensional contours of target objects and also acquire information such as the target's surface reflection characteristics, position, and speed, which can be used for target detection and tracking. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a target object detection method, apparatus, vehicle, and storage medium.

[0004] According to a first aspect of the present disclosure, a target object detection method is provided, comprising: acquiring an environmental image of the environment in which a vehicle is currently located; determining an image detection result of a target object in the environmental image using a target detection model based on the environmental image; the image detection result including a first relative position between the target object and the vehicle; filtering radar point cloud data acquired by the vehicle based on the image detection result to obtain target point cloud data corresponding to the target object; and determining a target detection result based on the target point cloud data, the target detection result including a second relative position between the target object and the vehicle.

[0005] Optionally, the first relative position includes the first orientation of the target object relative to the vehicle, and the image detection result also includes the target boundary of the target image region corresponding to the target object and the FOV (Field of View) corresponding to the target object; the step of determining the image detection result of the target object in the environment image based on the environment image through the target detection model includes: The environmental image is input into the target detection model to obtain the boundary position information of the target image region corresponding to the target object, and the grounding point position information of the target object; The boundary location information and the grounding point location information are projected into the BEV (Bird's Eye View) to obtain the target boundary corresponding to the target image area and the target grounding point location of the target object; The first orientation and the FOV are determined based on the location of the target grounding point.

[0006] Optionally, the step of filtering the radar point cloud data acquired by the vehicle based on the image detection result to obtain the target point cloud data corresponding to the target object includes: Based on the first azimuth and the FOV, determine the undetermined point cloud data from the radar point cloud data; and use the undetermined point cloud data located within the target boundary as the target point cloud data.

[0007] Optionally, the second relative position includes the second orientation of the target object relative to the vehicle, and the target distance between the target object and the vehicle, wherein determining the target detection result based on the target point cloud data includes: The second orientation and the target distance are determined based on the target point cloud data.

[0008] Optionally, the method further includes: The vehicle is controlled to move according to the second relative position.

[0009] Optionally, the image detection result further includes a first category of the target object, and controlling the vehicle's movement based on the second relative position includes: The vehicle is controlled to move according to the second relative position and the first category.

[0010] Optionally, the image detection result further includes a first confidence level for the first category; the target detection result further includes a second category for the target object and a second confidence level for the second category; the method further includes: The target category of the target object is determined based on the first confidence level and the second confidence level, wherein the target category is the category with higher confidence level between the first category and the second category; The step of controlling the vehicle's movement based on the second relative position and the first category includes: The vehicle is controlled to drive based on the second relative position and the target category.

[0011] According to a second aspect of the present disclosure, a target object detection apparatus is provided, comprising: The acquisition module is configured to acquire an environmental image of the vehicle's current location. The first determining module is configured to determine the image detection result of a target object in the environmental image based on the environmental image and a target detection model; the image detection result includes the first relative position of the target object and the vehicle; The data filtering module is configured to filter the radar point cloud data acquired by the vehicle based on the image detection results to obtain the target point cloud data corresponding to the target object. The second determining module is configured to determine a target detection result based on the target point cloud data, the target detection result including the second relative position of the target object and the vehicle.

[0012] According to a third aspect of the present disclosure, a vehicle is provided, comprising: processor; Memory used to store processor-executable instructions; The processor is configured as follows: Acquire an environmental image of the vehicle's current location; Based on the environmental image, an image detection model is used to determine the image detection result of the target object in the environmental image; the image detection result includes the first relative position of the target object and the vehicle; Based on the image detection results, the radar point cloud data acquired by the vehicle is filtered to obtain the target point cloud data corresponding to the target object; The target detection result is determined based on the target point cloud data, and the target detection result includes the second relative position of the target object and the vehicle.

[0013] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the target object detection method provided in the first aspect of the present disclosure.

[0014] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: When a vehicle detects target objects in a driving environment, it can combine radar point cloud data with image detection results of target objects in the environmental image. Based on the image detection results, the radar point cloud data is filtered to obtain target point cloud data with a smaller data volume. In this way, detecting target objects based on the filtered target point cloud data can reduce the computational complexity of the laser point cloud target detection algorithm and improve detection efficiency. At the same time, compared with a purely visual image perception scheme, it can significantly improve the positioning accuracy of target objects.

[0015] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0017] Figure 1 This is a flowchart illustrating a target object detection method according to an exemplary embodiment.

[0018] Figure 2 It is based on Figure 1 The illustrated embodiment shows a flowchart of a target object detection method.

[0019] Figure 3 It is based on Figure 1 The illustrated embodiment shows a flowchart of a target object detection method.

[0020] Figure 4 It is based on Figure 1 The illustrated embodiment shows a flowchart of a target object detection method.

[0021] Figure 5 This is a block diagram illustrating a target object detection device according to an exemplary embodiment.

[0022] Figure 6 It is based on Figure 5 The illustrated embodiment shows a block diagram of a target object detection device.

[0023] Figure 7 This is a block diagram illustrating a vehicle according to an exemplary embodiment. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0025] It should be noted that all actions involving the acquisition of signals, information, or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device.

[0026] This disclosure is primarily applied to scenarios where vehicles identify target objects (such as other vehicles, pedestrians, etc.) in their surrounding environment. Related technologies provide two perception algorithms for autonomous driving solutions. One is a purely visual image perception algorithm, which uses camera images as input. However, camera images are easily affected by lighting conditions and cannot accurately perceive the distance, orientation, and other information of target objects in three-dimensional space. In another autonomous driving solution, LiDAR point clouds can be used as input. However, LiDAR point clouds struggle to obtain information such as the color and texture of target object surfaces, and cannot provide sufficient semantic category information. During autonomous driving control, if the distance, orientation, and other information of target objects in three-dimensional space cannot be accurately perceived, or if the category of target objects cannot be correctly identified, it will affect the autonomous driving control strategy, thereby impacting the control accuracy of autonomous driving.

[0027] In addition, in related technologies, the target detection algorithm based on laser point cloud is usually quite complex due to the large range of point cloud data.

[0028] To address the aforementioned problems, this disclosure provides a target object detection method, apparatus, vehicle, and storage medium. The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0029] Figure 1 This is a flowchart illustrating a target object detection method according to an exemplary embodiment, such as... Figure 1 As shown, this method can be applied to vehicles (or described as "automobiles") and includes the following steps.

[0030] In step S11, an environmental image of the current environment of the vehicle is acquired.

[0031] In one implementation, multiple image acquisition devices (such as cameras) can be arranged around the vehicle body. This allows multiple image acquisition devices to collect environmental images from different locations around the vehicle, and then image recognition can be performed on the environmental images collected by each image acquisition device.

[0032] In step S12, the target object in the environmental image is determined by the target detection model based on the environmental image; the image detection result includes the first relative position of the target object and the vehicle.

[0033] The target detection model may include, for example, a Faster R-CNN model or a YOLO (you only look once) model. The target object may include, for example, objects on the sides of the road where the vehicle is currently located or on the road itself (such as other vehicles, buildings, trees), pedestrians, etc. The first relative position may include the first orientation of the target object relative to the vehicle, the distance between the target object and the vehicle, etc. The image detection result may also include the target image region corresponding to the target object in the environmental image, the target boundary of the target image region, the FOV of the target object, etc., where FOV refers to the field of view angle of the target object relative to the vehicle. The field of view angle may be, for example, the central angle of a fan-shaped region with the vehicle as the vertex, and the two sides of the fan-shaped region may be the lines connecting the vehicle to the left and right outermost boundary points of the target boundary, respectively.

[0034] In step S13, the radar point cloud data acquired by the vehicle is filtered according to the image detection results to obtain the target point cloud data corresponding to the target object.

[0035] Typically, radar point cloud data is acquired by a rotating lidar system. The lidar emits lasers (such as 128-line lasers) in all directions, and a single scan of the lidar system yields one frame of radar point cloud data. In autonomous driving scenarios, radar point cloud data allows for the precise determination of the distance and orientation of target objects. However, due to the large volume of radar point cloud data, determining the distance and orientation of target objects using lidar point cloud-based target detection algorithms is usually quite complex. Therefore, in this disclosure, to simplify the complexity of lidar point cloud target detection algorithms, the radar point cloud data acquired by the vehicle can be filtered to obtain target point cloud data with a smaller volume, thereby reducing the complexity of the lidar point cloud target detection algorithm.

[0036] In addition, the camera image-based perception algorithms corresponding to steps S11 and S12 can obtain the approximate orientation of the target object and the target boundary of the target image region corresponding to the target object. Therefore, in this step, the radar point cloud data acquired by the vehicle can be filtered according to the first orientation, the field of view of the target object relative to the vehicle, and the target boundary in the image detection results to obtain the target point cloud data.

[0037] In step S14, a target detection result is determined based on the target point cloud data, and the target detection result includes the second relative position of the target object and the vehicle.

[0038] The second relative position may include, for example, the second orientation of the target object relative to the vehicle, and the target distance between the target object and the vehicle.

[0039] In this step, the second orientation and the distance to the target can be determined based on the target point cloud data using a lidar point cloud target detection algorithm. The specific implementation method for determining the target detection result using the lidar point cloud target detection algorithm can be found in the descriptions in related technologies, and will not be specifically limited here.

[0040] Using the above method, when a vehicle detects target objects in its driving environment, it can combine radar point cloud data with image detection results of target objects in the environmental image. Based on the image detection results, the radar point cloud data is filtered to obtain target point cloud data with a smaller data volume. Thus, detecting target objects based on the filtered target point cloud data reduces the computational complexity of the laser point cloud target detection algorithm and improves detection efficiency. At the same time, compared to purely vision-based image perception schemes, it can significantly improve the localization accuracy of target objects.

[0041] Figure 2 It is based on Figure 1 The illustrated embodiment shows a flowchart of a target object detection method, as follows: Figure 2 As shown, step S12 includes the following sub-steps: In step S121, the environmental image is input into the target detection model to obtain the boundary position information of the target image region corresponding to the target object, and the grounding point position information of the target object.

[0042] Taking the target object as another vehicle relative to the vehicle itself as an example, the grounding point location information can include each grounding pixel on a line segment in the environmental image with the grounding points of the front and rear wheels (or left and right wheels) of the other vehicle as endpoints. For another example, if the target object is a building, the grounding point location information can be each grounding pixel on the grounding line between the building's wall and the ground in the environmental image.

[0043] The target image region can be the bounding rectangle of the target object's outline, and the boundary position information is the position information of each pixel on the border of the bounding rectangle in the environment image.

[0044] In this step, for each environmental image acquired by each image acquisition device, the environmental image can be input into a pre-trained target detection model (such as the Faster RCNN model), and then the model outputs the boundary position information of the target image region corresponding to the target object, as well as the grounding point position information of the target object.

[0045] In step S122, the boundary location information and the grounding point location information are projected into the bird's-eye view (BEV) to obtain the target boundary corresponding to the target image region and the target grounding point location of the target object.

[0046] In one possible implementation of this step, the detection results of the target object in the environmental image acquired by each camera (i.e., the boundary position information of the target image region and the grounding point position information of the target object) can be projected into the BEV based on the camera's calibration parameters (e.g., camera extrinsic parameters). In this way, in this step, each boundary position point of the target image region on the environmental image can be projected into the BEV, and the target boundary can be obtained based on each coordinate point after projection; each grounding pixel point on the environmental image can be projected into the BEV to obtain the target grounding point position of the target object.

[0047] In step S123, the first orientation and the FOV are determined based on the location of the target grounding point.

[0048] In this step, the outer border of the target object in the BEV view can be determined based on the target ground point location of the target object. Then, the first orientation can be determined based on the coordinates of the center of the outer border in the BEV view and the position coordinates of the vehicle in the BEV view. Furthermore, the FOV of the target object can be obtained based on the outer border of the target object.

[0049] It should be noted that, based on steps S121-S123, the target object's first direction relative to the vehicle and the target object's field of view (FOV) can be obtained from the target detection algorithm on the acquired environmental image. However, considering that when the boundary position information of the target image area and the grounding point position information of the target object in the environmental image are projected onto the BEV view through the camera calibration parameters, the orientation of the target object obtained by converting the two-dimensional grounding point in the environmental image to the BEV view is not accurate enough due to the inaccuracy of the camera calibration parameters; and the two-dimensional grounding point inferred by the target detection algorithm from the image is also not accurate enough, which will lead to a large error in the orientation of the target object under the determined BEV view.

[0050] Therefore, in order to improve the accuracy of the detection results of the relative position between the target object and the vehicle, the relative position (including orientation and distance) between the target object and the vehicle can be further identified based on the collected LiDAR point cloud data. However, since the point cloud data range is huge, the target detection algorithm based on LiDAR point cloud is usually quite complex. Therefore, while ensuring the accuracy of the detection results of the relative position between the target object and the vehicle, it is also necessary to reduce the computational complexity of the target detection algorithm.

[0051] Figure 3 It is based on Figure 1 The illustrated embodiment shows a flowchart of a target object detection method, as follows: Figure 3 As shown, step S13 includes the following sub-steps: In step S131, the point cloud data to be determined is determined from the radar point cloud data based on the first azimuth and the FOV.

[0052] As mentioned above, after the lidar emits a laser and scans around the perimeter, it can acquire a frame of radar point cloud data. In other words, a frame of radar point cloud data includes radar point cloud data covering 360 degrees around the vehicle. Therefore, in this step, in order to reduce the computational complexity of the lidar target detection algorithm, the radar point cloud data located within the first azimuth field of view (FOV) can be selected from the 360-degree radar point cloud data as the point cloud data to be determined.

[0053] Here, the first azimuth can be the azimuth pointed to by the center line of the fan-shaped area corresponding to the field of view (FOV). One end of the center line is the vertex of the fan, and the other end is the midpoint of the arc corresponding to the fan-shaped area. In this way, during the screening of radar point cloud data, the radar point cloud data within half the field of view range on the left and right sides can be taken as the point cloud data to be determined, with the first azimuth as the center azimuth.

[0054] In step S132, the undetermined point cloud data located within the target boundary is taken as the target point cloud data.

[0055] In this step, point cloud data located within the target boundary can be extracted from the undetermined point cloud data and used as the target point cloud data. It can be understood that this target point cloud data is the point cloud data corresponding to the target object.

[0056] based on Figure 3 The method shown can reduce the amount of radar point cloud data, thereby reducing the processing complexity of the lidar target detection algorithm. At the same time, it can accurately determine the position of the target object, and thus accurately identify the second orientation and target distance of the target object relative to the vehicle.

[0057] Figure 4 It is based on Figure 1 The illustrated embodiment shows a flowchart of a target object detection method, as follows: Figure 4 As shown, the method also includes the following steps: In step S15, the vehicle is controlled to move according to the second relative position.

[0058] The second relative position may include a second orientation and a target distance. After determining the second relative position of the target object relative to the vehicle, path planning and speed planning can be performed based on the second orientation and the target distance, thereby controlling the vehicle to travel according to the planned path and speed.

[0059] Furthermore, taking autonomous driving scenarios as an example, the autonomous driving strategies of vehicles may differ depending on the category of the target objects in the vehicle's driving environment. Therefore, it is necessary to identify the category of the target objects. However, radar point cloud data usually cannot obtain information such as the color and texture of the object's surface, and cannot provide sufficient semantic category information. Therefore, although LiDAR point cloud data can accurately determine the second location and distance of the target object, it cannot accurately perceive the category of the target object. In comparison, camera images are consistent with the environmental information perceived by the human eye. Therefore, the category of the target object identified by image perception algorithms is more accurate.

[0060] In other words, the image detection result may also include the first category of the target object, such as vehicles, buildings, pedestrians, etc.

[0061] Therefore, in another possible implementation of this step, the vehicle's movement can be controlled based on the second relative position and the first category. Specifically, a driving strategy (including driving path, driving speed, etc.) for the vehicle can be planned based on the second relative position and the first category, so as to control the vehicle's movement according to the driving strategy.

[0062] In another possible implementation of this disclosure, the image detection result may further include a first confidence level of the first category; the target detection result may further include a second category of the target object and a second confidence level of the second category; thus, the target category of the target object can be determined based on the first confidence level and the second confidence level, wherein the target category is the category with higher confidence level between the first category and the second category; thereby, the vehicle driving can be controlled based on the second relative position and the target category.

[0063] Using the above method, when a vehicle detects target objects in its driving environment, it can combine radar point cloud data with image detection results of target objects in the environmental image. Based on the image detection results, the radar point cloud data is filtered to obtain target point cloud data with a smaller data volume. This reduces the computational complexity of the laser point cloud target detection algorithm and improves detection efficiency when detecting target objects using the filtered target point cloud data. Simultaneously, compared to purely vision-based image perception schemes, it significantly improves the localization accuracy of target objects. Furthermore, compared to identifying target objects solely based on laser radar point cloud data, it also significantly improves the accuracy of target object category recognition.

[0064] Figure 5 This is a block diagram illustrating a target object detection device according to an exemplary embodiment, such as... Figure 5 As shown, the device includes: The acquisition module 501 is configured to acquire an environmental image of the current environment in which the vehicle is located; The first determining module 502 is configured to determine the image detection result of a target object in the environment image based on the environment image through a target detection model; the image detection result includes the first relative position of the target object and the vehicle; The data filtering module 503 is configured to filter the radar point cloud data acquired by the vehicle based on the image detection results to obtain the target point cloud data corresponding to the target object. The second determining module 504 is configured to determine a target detection result based on the target point cloud data, the target detection result including the second relative position of the target object and the vehicle.

[0065] Optionally, the first relative position includes the first orientation of the target object relative to the vehicle, and the image detection result also includes the target boundary of the target image region corresponding to the target object and the field of view (FOV) corresponding to the target object; the first determining module 502 is configured to input the environmental image into the target detection model to obtain the boundary position information of the target image region corresponding to the target object and the grounding point position information of the target object; project the boundary position information and the grounding point position information onto the bird's-eye view BEV to obtain the target boundary corresponding to the target image region and the target grounding point position of the target object; and determine the first orientation and the FOV based on the target grounding point position.

[0066] Optionally, the data filtering module 503 is configured to determine the undetermined point cloud data from the radar point cloud data based on the first azimuth and the FOV; and to use the undetermined point cloud data located within the target boundary as the target point cloud data.

[0067] Optionally, the second relative position includes the second orientation of the target object relative to the vehicle, and the target distance between the target object and the vehicle. The second determining module 504 is configured to determine the second orientation and the target distance based on the target point cloud data.

[0068] Optionally, Figure 6 It is based on Figure 5 The illustrated embodiment shows a block diagram of a target object detection device, as follows: Figure 6 As shown, the device also includes: The control module 505 is configured to control the movement of the vehicle based on the second relative position.

[0069] Optionally, the image detection result further includes a first category of the target object, and the control module 505 is configured to control the vehicle's movement based on the second relative position and the first category.

[0070] Optionally, the image detection result further includes a first confidence level of the first category; the target detection result further includes a second category of the target object and a second confidence level of the second category; the control module 505 is configured to determine the target category of the target object based on the first confidence level and the second confidence level, wherein the target category is the category with higher confidence level among the first category and the second category; and control the vehicle to drive based on the second relative position and the target category.

[0071] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0072] Using the aforementioned device, when a vehicle detects target objects in its driving environment, it can combine radar point cloud data with image detection results of target objects in the environmental image. Based on the image detection results, the radar point cloud data is filtered to obtain target point cloud data with a smaller data volume. Thus, detecting target objects based on the filtered target point cloud data reduces the computational complexity of the laser point cloud target detection algorithm and improves detection efficiency. Simultaneously, compared to purely vision-based image perception schemes, it can significantly improve the localization accuracy of target objects.

[0073] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the target object detection method provided in this disclosure.

[0074] Figure 7 This is a block diagram illustrating a vehicle according to an exemplary embodiment. For example, vehicle 700 can be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicles. Vehicle 700 can be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0075] Reference Figure 7 The vehicle 700 may include various subsystems, such as an infotainment system 710, a perception system 720, a decision control system 730, a drive system 740, and a computing platform 750. The vehicle 700 may also include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and each component of the vehicle 700 can be interconnected via wired or wireless means.

[0076] In some embodiments, the infotainment system 710 may include a communication system, an entertainment system, and a navigation system, etc.

[0077] The perception system 720 may include several sensors for sensing information about the environment surrounding the vehicle 700. For example, the perception system 720 may include a global positioning system (which may be GPS, BeiDou, or other positioning systems), an inertial measurement unit (IMU), lidar, millimeter-wave radar, ultrasonic radar, and a camera device.

[0078] The decision control system 730 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0079] The drive system 740 may include components that provide powered motion to the vehicle 700. In one embodiment, the drive system 740 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of internal combustion engines, electric motors, and compressed air engines. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0080] Some or all of the functions of vehicle 700 are controlled by computing platform 750. Computing platform 750 may include at least one processor 751 and memory 752, and processor 751 may execute instructions 753 stored in memory 752.

[0081] Processor 751 can be any conventional processor, such as a commercially available CPU. Processors may also include graphics processing units (GPUs), field-programmable gate arrays (FPGAs), systems-on-chips (SoCs), application-specific integrated circuits (ASICs), or combinations thereof.

[0082] The memory 752 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0083] In addition to instruction 753, memory 752 can also store data, such as road maps, route information, vehicle position, direction, speed, and other data. The data stored in memory 752 can be used by computing platform 750.

[0084] In this embodiment of the disclosure, the processor 751 may execute instructions 753 to complete all or part of the steps of the target object detection method described above.

[0085] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the target object detection method described above when executed by the programmable device.

[0086] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0087] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for detecting a target object, characterized in that, include: Acquire an environmental image of the vehicle's current location; Based on the environmental image, the target detection model is used to determine the image detection result of the target object in the environmental image; The image detection result includes the first relative position between the target object and the vehicle; Based on the image detection results, the radar point cloud data acquired by the vehicle is filtered to obtain the target point cloud data corresponding to the target object; The target detection result is determined based on the target point cloud data, and the target detection result includes the second relative position between the target object and the vehicle; The first relative position includes the first orientation of the target object relative to the vehicle, and the image detection result also includes the target boundary of the target image region corresponding to the target object and the field of view (FOV) corresponding to the target object; The step of filtering the radar point cloud data acquired by the vehicle based on the image detection results to obtain the target point cloud data corresponding to the target object includes: taking radar point cloud data within a 1 / 2 field of view range to the left and right of the first azimuth as the center azimuth as the undetermined point cloud data, where the first azimuth is the azimuth pointed to by the center line of the fan-shaped area corresponding to the field of view. The undetermined point cloud data located within the target boundary is taken as the target point cloud data.

2. The method according to claim 1, characterized in that, The step of determining the image detection result of the target object in the environmental image based on the target detection model includes: The environmental image is input into the target detection model to obtain the boundary position information of the target image region corresponding to the target object, and the grounding point position information of the target object; The boundary location information and the grounding point location information are projected into the bird's-eye view (BEV) to obtain the target boundary corresponding to the target image region and the target grounding point location of the target object. The first orientation and the FOV are determined based on the location of the target grounding point.

3. The method according to claim 1, characterized in that, The second relative position includes the second orientation of the target object relative to the vehicle, and the target distance between the target object and the vehicle. Determining the target detection result based on the target point cloud data includes: The second orientation and the target distance are determined based on the target point cloud data.

4. The method according to any one of claims 1-3, characterized in that, The method further includes controlling the vehicle's movement based on the second relative position.

5. The method according to claim 4, characterized in that, The image detection result also includes the first category of the target object, and controlling the vehicle driving according to the second relative position includes controlling the vehicle driving according to the second relative position and the first category.

6. The method according to claim 5, characterized in that, The image detection result further includes a first confidence level for the first category; the target detection result further includes a second category for the target object, and a second confidence level for the second category; the method further includes: The target category of the target object is determined based on the first confidence level and the second confidence level, wherein the target category is the category with higher confidence level between the first category and the second category; The step of controlling the vehicle's movement based on the second relative position and the first category includes: The vehicle is controlled to drive based on the second relative position and the target category.

7. A target object detection device, characterized in that, include: The acquisition module is configured to acquire an environmental image of the vehicle's current location. The first determining module is configured to determine the image detection result of a target object in the environmental image based on the environmental image and a target detection model; the image detection result includes the first relative position of the target object and the vehicle; The data filtering module is configured to filter the radar point cloud data acquired by the vehicle based on the image detection results to obtain the target point cloud data corresponding to the target object. The second determining module is configured to determine a target detection result based on the target point cloud data, the target detection result including a second relative position between the target object and the vehicle; The first relative position includes the first orientation of the target object relative to the vehicle, and the image detection result also includes the target boundary of the target image region corresponding to the target object and the field of view (FOV) corresponding to the target object; The data filtering module is configured to take radar point cloud data within a 1 / 2 field of view range on both the left and right sides with the first azimuth as the center azimuth as the point cloud data to be determined, where the first azimuth is the azimuth pointed to by the center line of the fan-shaped area corresponding to the field of view; and to take the point cloud data to be determined located within the target boundary as the target point cloud data.

8. A vehicle, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: Acquire an environmental image of the vehicle's current location; Based on the environmental image, an image detection model is used to determine the image detection result of the target object in the environmental image; the image detection result includes the first relative position of the target object and the vehicle; Based on the image detection results, the radar point cloud data acquired by the vehicle is filtered to obtain the target point cloud data corresponding to the target object; The target detection result is determined based on the target point cloud data, and the target detection result includes the second relative position between the target object and the vehicle; The first relative position includes the first orientation of the target object relative to the vehicle, and the image detection result also includes the target boundary of the target image region corresponding to the target object and the field of view (FOV) corresponding to the target object; the step of filtering the radar point cloud data acquired by the vehicle based on the image detection result to obtain the target point cloud data corresponding to the target object includes: With the first azimuth as the center, radar point cloud data within a 1 / 2 field of view range to the left and right are taken as point cloud data to be determined. The first azimuth is the azimuth pointed to by the center line of the fan-shaped area corresponding to the field of view. The undetermined point cloud data located within the target boundary is taken as the target point cloud data.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1-6.

Images