Target detection method and device, vehicle and computer readable storage medium

Abstract

The embodiment of the invention provides a target detection method and device, a vehicle and a computer readable medium, and relates to the technical field of intelligent driving. According to the invention, the region of interest in the vehicle sensing range can be determined based on the current driving intention of the vehicle; increasing the sampling weight of the region of interest, and sampling a target sampling point from the sensor data of the vehicle based on the sampling weight; and according to the target sampling point, detecting an obstacle in a vehicle sensing range. According to the method and the device, the sampling points can be more focused on the interested area needing attention, the target detection capability of the interested area is improved, the safety of vehicle driving is improved, the interested area needing attention can be automatically adjusted based on the driving intention of the vehicle, and the user experience is improved. And the target detection performance of the local area in different scenes is improved without increasing additional computing power.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving perception technology, and more specifically, to a target detection method, device, vehicle and computer-readable storage medium. Background Art

[0002] Intelligent driving technology is integrated into the vehicle, enabling the vehicle to achieve automatic driving or driver-assisted driving through the sensors and controllers integrated on it. Intelligent driving technology includes target detection technology, which refers to detecting obstacles around the vehicle based on data sensed by the sensors integrated on the vehicle.

[0003] Current target detection methods all use forward reasoning and equal weighting to identify targets, that is, the same sampling weight is used to sample data for all areas within the vehicle's perception range, so that obstacles within the vehicle's perception range are detected based on the sampled data.

[0004] However, the inventors found in their research that under different driving behavior modes, target detection needs to identify different areas with bias. For example, when a vehicle is about to change lanes to the left lane for high-speed overtaking, the vehicle needs to focus on the vehicle it is currently in and the obstacles on the left lane, and pay less attention to the information on the right lane. However, the current target detection method performs equal-weighted sampling of the entire vehicle perception range and detects obstacles, resulting in insufficient target detection efficiency and accuracy in areas that need to be focused on. Summary of the invention

[0005] The embodiments of the present application propose a target detection method, device, vehicle and computer-readable storage medium, which improve the target detection performance in the area of ​​interest by increasing the sampling weight of the area of ​​interest that needs to be focused on according to the driving intention, thereby solving the technical problem of insufficient target detection efficiency and accuracy in the above-mentioned area of ​​interest.

[0006] In a first aspect, an embodiment of the present application provides a target detection method, the method comprising: determining an area of ​​interest within a perception range of the vehicle based on the current driving intention of the vehicle; increasing the sampling weight of the area of ​​interest, and sampling target sampling points from the sensor data of the vehicle based on the sampling weight; and detecting obstacles within the perception range of the vehicle based on the target sampling points.

[0007] In a second aspect, an embodiment of the present application provides a target detection device, which includes: a perception module, used to determine an area of ​​interest within a perception range of a vehicle based on the vehicle's current driving intention; a sampling module, used to increase the sampling weight of the area of ​​interest, and sample target sampling points from the vehicle's sensor data based on the sampling weight; a detection module, used to detect obstacles within the perception range of the vehicle based on the target sampling points.

[0008] In a third aspect, an embodiment of the present application provides a vehicle, comprising: a memory and a processor, wherein an application is stored in the memory, and the application is used to enable the vehicle to execute the method provided by the embodiment of the present application when called by the processor.

[0009] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a program code stored thereon, wherein the program code is used to enable the processor to execute the method provided by the embodiment of the present application when called by the processor.

[0010] The target detection method provided in the embodiment of the present application has the following technical effects: determining the region of interest based on the current driving intention of the vehicle, increasing the sampling weight of the region of interest, so that the sampling points can focus more on the region of interest that needs attention, improving the target detection capability in the region of interest, and improving the safety of vehicle driving, thereby being able to automatically adjust the region of interest that needs attention based on the driving intention of the vehicle, and improving the target detection performance of local areas in different scenarios without increasing additional computing power. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments and drawings obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0012] Figure 1 A schematic diagram showing the structure of the hardware operating environment of the target detection method provided in the embodiment of the present application;

[0013] Figure 2 A schematic diagram of the flow of the signal preprocessing module provided in an embodiment of the present application is shown;

[0014] Figure 3 A schematic diagram of the process of the target detection module provided in the embodiment of the present application is shown;

[0015] Figure 4 A schematic diagram of the flow of the planning and control module provided in an embodiment of the present application is shown;

[0016] Figure 5 A schematic diagram of the process flow of the abstract module provided in the embodiment of the present application is shown;

[0017] Figure 6 A schematic diagram of a process flow of a target detection method provided by an embodiment of the present application is shown;

[0018] Figure 7 A schematic diagram showing initial sampling points extracted from sensor data provided by an exemplary embodiment of the present application is shown;

[0019] Figure 8 Shows the Figure 7 Schematic diagram of sampling points obtained after adding sampling points in the area of ​​interest;

[0020] Fig. 9 A schematic diagram of a process flow of a target detection method provided by another embodiment of the present application is shown;

[0021] Fig.10 A schematic diagram of a target detection process provided by an exemplary embodiment of the present application is shown;

[0022] Fig.11 A schematic diagram of the process of step S230 provided in an embodiment of the present application is shown;

[0023] Fig.12 A schematic diagram of the structure of a target detection device provided in an embodiment of the present application is shown;

[0024] Fig.13 A schematic structural diagram of a vehicle provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0026] See also Figure 1 , Figure 1 The schematic diagram of the structure of the hardware operating environment of the target detection method provided by the embodiment of the present application is shown. The vehicle 100 may include a sensor module 110 , a signal preprocessing module 120 , a target detection module 130 , a planning and control module 140 and an abstract module 150 .

[0027] The sensor module 110 may be integrated with different sensors, including but not limited to laser radar, millimeter wave radar, and camera, etc. The sensor module 110 is used to collect data acquired by different sensors and transmit the collected sensor data to the signal preprocessing module 120 .

[0028] The signal preprocessing module 120 can receive sensor data, preprocess the sensor data, and transmit the preprocessed sensor data to the target detection module 130. Preprocessing may include, but is not limited to, cropping, resizing, voxelization, and normalization. Cropping is used to crop an image or video by selecting a rectangular area to retain the required part and remove other parts. Resizing is used to change the size of an image. Voxelization is used to convert three-dimensional spatial data (such as lidar point cloud data) into a voxel grid. Normalization is used to convert data into a uniform scale range to eliminate dimensional differences between different variables. Through normalization, data can be mapped to a specific range so that different features or attributes have similar numerical ranges.

[0029] As an example, see Figure 2 , Figure 2 FIG. 2 shows a flow chart of a signal preprocessing module provided in an embodiment of the present application. Figure 2 As shown, assuming that the sensor includes a lidar, a millimeter-wave radar and a camera, the signal preprocessing module 120 can voxelize and normalize the lidar point cloud data; for pictures, it can normalize the pictures; for millimeter-wave radar data, it can voxelize and normalize the millimeter-wave radar data.

[0030] The target detection module 130 is used to perform target detection based on the preprocessed sensor data (i.e., normalized sensor data) and identify the geometric size and position of obstacles around the vehicle. In addition, when the planning and control module 140 has an output (i.e., the output is not zero), the abstract module 150 can abstract the information output by the planning and control module 140 to obtain abstract high-dimensional regulatory information. The target detection module 130 can perform target detection based on the normalized sensor data under the prior guidance of the high-dimensional regulatory information and identify the geometric size and position of obstacles around the vehicle. The target detection module 130 can transmit the geometric size and position of the detected obstacles to the planning and control module 140.

[0031] As an example, see Figure 3 , Figure 3 FIG. 2 shows a flow chart of the target detection module provided in the embodiment of the present application. Figure 3As shown, the target detection module 130 can receive normalized sensor data and abstract high-dimensional regulation information, use the backbone network (Backbone) to sample the normalized sensor data (i.e., feature extraction) to obtain initial sampling points; use Bevfusion to fuse different sensor data to obtain sensor fusion data; use the neck network (Neck, such as Transformer) based on the attention architecture to combine the abstract high-dimensional regulation information, the initial sampling points and the sensor fusion data to focus on updating the sampling points; use the head network (Head) to achieve weighted target geometry and posture detection according to the updated sampling points.

[0032] The backbone network is the main component of the target detection model and is used to extract features of sensor data. The backbone network may include but is not limited to convolutional neural networks and residual neural networks (Residual Network, referred to as ResNet). For example, the convolutional neural network may be EfficientNet, and the residual neural network may be ResNet101.

[0033] Among them, the neck network is the middle layer connecting the backbone network and the head network, which is mainly used to reduce the dimension or adjust the features from the backbone network to better adapt to the task requirements. The neck network can use convolutional layers, pooling layers, or fully connected layers. For example, the neck network can include a bidirectional feature pyramid network (Bidirectional Feature Pyramid Network, referred to as BiFPN) and Transformer. Transformer is a deep learning model architecture for natural language processing and other sequence-to-sequence tasks. Transformer uses the cross attention mechanism and the self-attention mechanism to increase the sampling points of the region of interest.

[0034] The cross-attention mechanism is a mechanism that calculates the correlation and weighted sum between two different input sequences. Specifically, given two input sequences, the cross-attention mechanism calculates the correlation between each element in one sequence and all elements in the other sequence, and performs a weighted sum of each element in the two sequences based on the correlation, thereby establishing a correlation relationship between different sequences and fusing the information of the two sequences.

[0035] The self-attention mechanism is a self-attention mechanism that calculates the degree of association between each element and other elements in the same input sequence. Specifically, for a given input sequence, the self-attention mechanism calculates the degree of association between each element and all elements in the sequence, and performs a weighted summation of each element in the sequence based on the degree of association, thereby being able to capture the relationship between different elements in the input sequence and better understand the contextual information of the input.

[0036] Among them, Bevfusion is a data fusion method that uses a bird's-eye view as a unified representation space to fuse data from different sensors together, thereby improving the accuracy of 3D target detection.

[0037] Among them, the head network is the last layer of the target detection model, which is used to perform specific tasks based on the features output by the neck network and generate the final output. For example, the head network can be CenterHead. CenterHead can be used for Anchor Free target detection. CenterHead consists of heatmap, width, height, and offset prediction branches. Each branch consists of a 3×3 convolution layer and a 1×1 convolution layer. CenterHead converts the target detection problem into a center point prediction problem. The target is represented by the center point of the target, and then the offset and width of the center point are predicted to obtain the final detection result. Among them, the anchor-free method can directly predict the pose and geometry of the target on the image or feature map without relying on the predefined anchor box, so that the model does not need to define the anchor box in advance, and can handle targets of different sizes and shapes more flexibly.

[0038] See also Figure 4 , Figure 4 FIG. 1 shows a flow chart of the planning and control module provided in the embodiment of the present application. Figure 4 As shown, the planning and control module 140 can receive the target detection result, and perform path planning, decision making and vehicle control processing in sequence according to the target detection result. Among them, the path planning can use the A* algorithm for path planning. The planning and control module 140 can transmit its output information (i.e., regulatory control information) to the abstract module 150. The regulatory control information may include but is not limited to the vehicle's steering wheel steering, pedal signals and discrete point information on the planned path.

[0039] See also Figure 5 , Figure 5 FIG. 1 shows a flow chart of the abstract module provided in the embodiment of the present application. Figure 5As shown, the abstract module 150 can receive regulatory control information, and perform operations such as position embedding, feature abstraction, and feature aggregation on the regulatory control information in turn, thereby abstracting the received regulatory control information into high-dimensional planning control information (referred to as high-dimensional regulatory control information). The abstract module 150 can feed back the high-dimensional regulatory control information to the target detection module 130, providing prior guidance for target detection, thereby improving the performance of target detection. The present application can decouple the target detection module 130 and the abstract module 150, so that the abstract module 150 can receive regulatory control information and abstract it into high-dimensional regulatory control information, and use the high-dimensional regulatory control information to guide the target detection module to focus on the areas of interest that need attention, thereby improving the performance of target detection without increasing additional computing power.

[0040] It is understandable that this application Figures 2 to 5 The flow charts of each module shown are only illustrative examples and should not be understood as limiting the scope of protection of this application. Those skilled in the art can adjust (for example, add, delete or modify) the network structure of each module according to actual needs, and this application does not impose specific restrictions.

[0041] The target detection method in the embodiment of the present application can be applied to a vehicle or a target detection device, and the target detection device can be integrated into a vehicle. The vehicle has an intelligent driving function, and the intelligent driving function includes an automatic driving function and / or an assisted driving function. The vehicle may include but is not limited to a gasoline vehicle or an electric vehicle, and the electric vehicle may include but is not limited to a pure electric vehicle, a hybrid vehicle, or a fuel cell vehicle. Next, the target detection method of the present application will be introduced.

[0042] See also Figure 6 , Figure 6 FIG. 1 is a flow chart of a target detection method provided by an embodiment of the present application. Figure 6 As shown, the target detection method may include steps S110 to S130.

[0043] Step S110: Based on the current driving intention of the vehicle, determine the area of ​​interest within the vehicle's perception range.

[0044] In some embodiments, when the vehicle is in the intelligent driving mode, the current driving intention of the vehicle can be determined through the local path planned by the path planning and control module. For example, based on the vehicle position and the end point of the local path, if the trajectory of the local path points from the current lane of the vehicle to the left lane, it can be determined that the current driving intention of the vehicle is to change lanes to the left lane; if the trajectory of the local path points from the current lane of the vehicle to the right lane, it can be determined that the current driving intention of the vehicle is to change lanes to the right lane; if the trajectory of the local path is basically parallel to the current lane of the vehicle, it can be determined that the current driving intention of the vehicle is to go straight in the current lane of the vehicle. Among them, the intelligent driving mode can include an assisted driving mode and an automatic driving mode.

[0045] In some embodiments, when the vehicle is not in the intelligent driving mode, the deviation angle between the current steering wheel angle and the previous steering wheel angle can be obtained, and the current driving intention of the vehicle can be determined based on the deviation angle. For example, if the deviation angle is in the first angle range, it can be determined that the current driving intention of the vehicle is to change lanes to the left lane; if the deviation angle is in the second angle range, it can be determined that the current driving intention of the vehicle is to change lanes to the right lane; if the deviation angle is in the third angle range, it can be determined that the current driving intention of the vehicle is to go straight in the lane where the vehicle is currently located. Among them, the intersection of the first angle range, the second angle range, and the third angle range is 0 and the union is 180 degrees. The first angle range, the second angle range, and the third angle range can all be pre-set based on actual needs, and this application does not make specific restrictions.

[0046] In some embodiments, a target lane may be determined based on the current driving intention of the vehicle; and an area within the vehicle's perception range where the target lane is located may be determined as an area of ​​interest.

[0047] The target lane refers to the lane that the vehicle needs to focus on. For example, if the vehicle's current driving intention is to change lanes to the left lane, the target lane includes the left lane and the vehicle's current lane; if the vehicle's current driving intention is to change lanes to the right lane, the target lane includes the right lane and the vehicle's current lane; if the vehicle's current driving intention is to go straight in the vehicle's current lane, the target lane includes the vehicle's current lane, the left lane, and the right lane.

[0048] Among them, the vehicle perception range can be determined according to the perception range of the actual sensor. For example, the vehicle perception range can be the perception range of the sensor, or it can be smaller than the perception range of the sensor. This application does not make specific restrictions.

[0049] Step S120: increasing the sampling weight of the region of interest, and sampling target sampling points from the sensor data of the vehicle based on the sampling weight.

[0050] The sensor module of the vehicle collects data acquired by different sensors in real time to obtain sensor data, which may include but is not limited to lidar data, images, millimeter wave radar data, etc. The sampling points can be obtained by sampling the sensor data.

[0051] In some embodiments, sampling points of the region of interest can be statically added. The sampling weight of the region of interest can be pre-set as a first sampling weight, and the sampling weights of other regions within the vehicle's perception range can be set as a second sampling weight, where the second sampling weight is less than the first sampling weight. After determining the region of interest, the sensor data within the region of interest is sampled based on the first sampling weight, and the sensor data of other regions is sampled based on the second sampling weight, thereby focusing on extracting the features of the sensor data of the region of interest, so that more sampling points are focused on the region of interest that needs attention.

[0052] In some embodiments, sampling points of the region of interest can be dynamically added. Sensor data can be preprocessed to obtain preprocessed sensor data; sensor data can be sampled to obtain initial sampling points; based on the region of interest, the initial sampling points can be dynamically updated to increase the sampling weight of the region of interest. Preprocessing can include but is not limited to cropping, resizing, voxelization, and normalization. Preprocessing sensor data can make subsequent data analysis more accurate and improve the accuracy of target detection.

[0053] For example, the high-dimensional regulatory information output by the abstract module can be obtained, and the high-dimensional regulatory information includes information obtained after position embedding processing of the information output by the planning and control module of the vehicle; the initial sampling points are dynamically updated according to the high-dimensional regulatory information, and the sampling weight of the area of ​​interest is increased. Among them, the high-dimensional regulatory information comes from the local path, which can reflect the current driving intention of the vehicle and the driving trajectory for a period of time in the future. The sampling points of the area of ​​interest are increased based on the high-dimensional regulatory information, so that the target detection can be adapted to the regulatory signal, and the features of the sensor data of the area of ​​interest can be adaptively and dynamically extracted, so that more sampling points are focused on the area of ​​interest that needs attention, thereby improving the safety of vehicle driving.

[0054] Since the high-dimensional regulatory information comes from the local path, it can reflect the current driving intention of the vehicle and the driving trajectory for a period of time in the future. In some embodiments, when the vehicle is in motion and in the intelligent driving mode, step S110 may not be executed, and the high-dimensional regulatory information output by the abstract module may be directly obtained. After the initial sampling point is obtained, the initial sampling point is updated according to the high-dimensional regulatory information, thereby utilizing the characteristics of the high-dimensional regulatory information that can reflect the driving intention of the vehicle, and directly increasing the sampling weight of the region of interest according to the high-dimensional regulatory information without separately determining the driving intention, thereby improving the overall target detection efficiency.

[0055] In some embodiments, based on high-dimensional regulatory information, a cross-attention mechanism and a self-attention mechanism can be used to increase the sampling weight of the region of interest, thereby simulating multi-resolution operations. Simulating multi-resolution means focusing more sampling points on the region of interest to obtain more fine-grained features, which is more direct and effective than generating multi-resolution feature maps.

[0056] Specifically, the cross-attention mechanism can be used to update the sampling points by combining the high-dimensional regulatory information and the initial sampling points, thereby utilizing the correlation between the high-dimensional regulatory information and the initial sampling points, and under the prior guidance of the high-dimensional regulatory information, increasing the sampling weight of the region of interest that needs attention, and reducing the sampling weight of other regions. Then, the self-attention mechanism is used to update the updated sampling points, thereby utilizing the correlation between the sampling points in the updated sampling points, further increasing the sampling weight of the region of interest that needs attention, and reducing the sampling weight of other regions.

[0057] In some embodiments, when the sensor data includes data collected by at least two sensors, sampling points (i.e., initial sampling points) and sensor fusion data can be obtained based on the sensor data, and the sensor fusion data is data obtained by fusing different sensor data using the Bevfusion method; according to the high-dimensional regulatory information and the acquired sampling points, a cross-attention mechanism is used to update the sampling points, thereby utilizing the correlation between the high-dimensional regulatory information and the initial sampling points, and under the prior guidance of the high-dimensional regulatory information, the sampling weight of the area of ​​interest that needs attention is increased, and the sampling weight of other areas is reduced; the updated sampling points are updated again using a self-attention mechanism, thereby utilizing the correlation between the sampling points in the updated sampling points, further increasing the sampling weight of the area of ​​interest that needs attention, and reducing the sampling weight of other areas; according to the sampling points that are updated again and the sensor fusion data, a cross-attention mechanism is used to update the sampling points, thereby fusing the sampling points corresponding to different sensor data, further increasing the sampling weight of the area of ​​interest that needs attention, and reducing the sampling weight of other areas.

[0058] It can be understood that the above-mentioned cross-attention mechanism and self-attention mechanism can be repeatedly executed multiple times, so that more sampling points are focused on the region of interest, thereby improving the subsequent local target detection performance in the region of interest.

[0059] As an example, see Figure 7 and Figure 8 , Figure 7 A schematic diagram showing initial sampling points extracted from sensor data provided by an exemplary embodiment of the present application is shown. Figure 8 Shows the Figure 7Schematic diagram of the sampling points obtained after adding sampling points in the area of ​​interest. Figure 7 The black dots shown are the initial sampling points extracted from the sensor data. Figure 7 As shown by the arrow, when the vehicle is moving and the automatic driving mode is turned on, the planning and control module plans to change lanes to the left lane (i.e., the vehicle's current driving intention), and needs to accelerate to merge into the left lane. In this case, the attention to obstacles in the left lane and the lane the vehicle is currently in is greater, and the attention to obstacles in the right lane is less. In other words, the vehicle needs to focus more on the left lane and the lane the vehicle is currently in. At this time, the sampling points on the left lane and the lane the vehicle is currently in can be increased so that the sampling points focus more on the left lane and the lane the vehicle is currently in. Figure 8 As shown in Figure 2, after adding sampling points on the left lane and the lane where the vehicle is currently located, the sampling points ( Figure 8 The black dots in the figure focus more on the left lane and the lane the vehicle is currently in, which can improve the subsequent target detection performance in the local area.

[0060] like Figure 7 and Figure 8 As shown, it can be seen that under the prior guidance of high-speed regulation and control information, the cross-attention mechanism and the self-attention mechanism are used to increase the sampling weights of the areas of interest that need more attention, so that more sampling points can be focused on the left lane and the lane where the vehicle is currently located, thereby improving the accuracy of target detection in the left lane and the lane where the vehicle is currently located. At the same time, there are fewer sampling points on the right lane, which can improve the target detection efficiency of the right lane.

[0061] Step S130: Detect obstacles within the vehicle's perception range according to the target sampling points.

[0062] In the embodiment of the present application, obstacles may include static obstacles and dynamic obstacles. Static obstacles may include but are not limited to road warning signs, crash columns, road boundary fences, etc., and dynamic obstacles may include but are not limited to vehicles, pedestrians, animals, etc.

[0063] According to the target sampling points, obstacles within the vehicle's perception range can be detected (e.g. Figure 8 Obstacles shown in the gray rectangular box in the middle), the specific target detection algorithm can be selected from the relevant target detection algorithms, and this application does not make specific restrictions. Since the sampling weight in the region of interest is increased, the sampling points can be more focused on the region of interest, thereby improving the accuracy of local target detection in the region of interest. At the same time, since the sampling weights of other regions except the region of interest within the vehicle perception range are reduced, the sampling points can be less focused on other regions, thereby improving the efficiency of local target detection in other regions.

[0064] Steps S110 to S130 have the following technical effects: determining the region of interest based on the current driving intention of the vehicle, increasing the sampling weight of the region of interest, so that the sampling points can focus more on the region of interest that needs attention, improving the target detection capability in the region of interest, and improving the safety of vehicle driving, thereby being able to automatically adjust the region of interest that needs attention based on the driving intention of the vehicle, and improving the target detection performance of local areas in different scenarios without increasing additional computing power.

[0065] See also Fig. 9 , Fig. 9 FIG. 2 is a flow chart of a target detection method provided by another embodiment of the present application. Fig. 9 As shown, the target detection method may include steps S210 to S240. The sensor data in this embodiment may include data collected by at least two sensors.

[0066] Step S210: acquiring sensor fusion data and initial sampling points based on the sensor data, wherein the sensor fusion data is data obtained by fusing different sensor data.

[0067] In some embodiments, the sensor data may include laser radar point cloud data, millimeter wave radar data, and images captured by a camera. The preprocessing module may preprocess different types of data separately and send the preprocessed sensor data to the target detection module.

[0068] See also Fig.10 , Fig.10 FIG. 2 shows a flow chart of target detection provided by an exemplary embodiment of the present application. Fig.10 As shown, after the target detection module receives the preprocessed sensor data, it can use the backbone network (such as ResNet or Efficientnet) and the neck network (such as BiFPN) to extract features from the preprocessed sensor data. Then, the 2D data is converted into 3D data using a projection method, and the projection method may include but is not limited to inverse perspective mapping (IPM) projection and LSS (Lift-Splat-Shoot) projection. Then, based on Bevfusion, the features of different sensors are fused to obtain sensor fusion data. After obtaining the sensor fusion data, random sampling can be performed from the sensor fusion data to obtain the initial sampling point.

[0069] Understandably, Fig.10The target detection model shown is only an example and is not intended to limit the scope of protection of the present application. Those skilled in the art can adjust or add or reduce the modules of the target detection model as needed, for example, adjust the backbone network or neck network or projection method, or reduce or increase the feature extraction network in the backbone network or neck network.

[0070] Step S220: Acquire information output by the planning and control module, perform position embedding processing on the information output by the planning and control module, and obtain high-dimensional planning and control information.

[0071] In some embodiments, when the vehicle is moving and not in the automatic driving mode, the information output by the planning and control module includes the steering wheel angle and pedal signal (accelerator pedal opening and closing angle) of the vehicle. At this time, the steering wheel angle and pedal signal of the vehicle output by the planning and control module can be obtained. In this case, the target detection adopts the feedback mode for target detection, and the abstraction module can abstract the steering wheel angle and pedal signal into high-dimensional regulatory information. The input of the target detection module includes high-dimensional regulatory information and pre-processed sensor data, so as to use the high-dimensional regulatory information to guide the target detection module to extract features and increase the sampling weight of the area of ​​interest.

[0072] In some embodiments, when the vehicle is moving and in the automatic driving mode, the information output by the planning and control module includes the steering wheel angle, pedal signal and path planning signal of the vehicle. At this time, the steering wheel angle, pedal signal and path planning signal of the vehicle output by the planning and control module can be obtained. In this case, the target detection adopts the feedback mode for target detection. The abstract module can abstract the steering wheel angle, pedal signal and path planning signal into high-dimensional regulatory information. The input of the target detection module includes high-dimensional regulatory information and pre-processed sensor data, so as to use the high-dimensional regulatory information to guide the target detection module to extract features and increase the sampling weight of the area of ​​interest. Among them, the path planning signal refers to the discrete point information on the local path planned by the planning and control module in the vehicle coordinate system.

[0073] In some embodiments, the information output by the planning and control module is processed by position embedding to obtain high-dimensional regulatory information, which may include: by using a position embedding method, M types of signals are uniformly encoded into a code with a length of 128 and a dimension of 256 to form Mx128x256 high-dimensional feature information, and a multilayer perceptron (Multilayer Perceptron, MLP for short) is used to fuse the feature information to obtain high-dimensional regulatory information. Wherein, when the information output by the planning and control module includes a steering wheel angle and a pedal signal, M=2, and when the information output by the planning and control module includes a steering wheel angle, a pedal signal and a path planning signal, M=3. Multilayer Perceptron (Multilayer Perceptron, MLP for short) is an artificial neural network model, including multiple neurons. Multilayer Perceptron is a feedforward neural network, including an input layer, at least one hidden layer, and an output layer, each layer is composed of multiple neurons, and each neuron generates an output by weighted summing the input values ​​and passing through an activation function. During the training process, the multilayer perceptron updates the weights and biases between neurons through the back-propagation algorithm to minimize the error between the predicted output and the actual output.

[0074] Among them, the position embedding can use sine position embedding, and the output of the position embedding can be calculated by sine and cosine functions of different frequencies:

[0075]

[0076] in,

[0077]

[0078] Among them, t represents input 0-127, i represents 0-255, and d represents dimension depth 255. represents the output, and f(t) represents the function that produces the output.

[0079] It can be understood that when the vehicle is stationary, the sensor module, signal preprocessing module, target detection module, planning and control module continue to execute, but the planning and control module has no output, that is, the information output by the planning and control module is 0, and the target detection is directly detected using a non-feedback mode. The input of the target detection only includes preprocessed sensor data without high-dimensional planning and control information.

[0080] Step S230: According to the high-dimensional regulatory information, the initial sampling points and the sensor fusion data, a cross-attention mechanism and a self-attention mechanism are used to increase the sampling weight of the region of interest, and the initial sampling points are updated based on the sampling weights to obtain the target sampling points.

[0081] Please also read Fig.10 and Fig.11 , Fig.11 FIG. 2 shows a flow chart of step S230 provided in an embodiment of the present application. Fig.10 and Fig.11 As shown, step S230 may include steps S231 to S233.

[0082] Step S231: after obtaining the initial sampling points, the initial sampling points are updated using a cross-attention mechanism according to the high-dimensional regulatory information and the initial sampling points, increasing the sampling weight of the region of interest and reducing the sampling weight of other regions.

[0083] Step S232: Use the self-attention mechanism to update the updated sampling points again, increase the sampling weight of the area of ​​interest, and reduce the sampling weight of other areas.

[0084] Step S233: According to the updated sampling points and the sensor fusion data, the cross attention mechanism is used to update the sampling points, increase the sampling weight of the area of ​​interest, and reduce the sampling weight of other areas.

[0085] After executing steps S231 to S233, target sampling points can be obtained, among which more sampling points focus on the region of interest and fewer sampling points focus on other regions, thereby improving the accuracy of local target detection in the region of interest and improving the efficiency of local target detection in other regions.

[0086] In some embodiments, after executing step S233, the target sampling point can be fed back to step S231, so as to combine the initial sampling point, the target sampling point and the high-dimensional regulatory information, adopt the cross-attention mechanism to update the sampling point, and repeat the operations of step S231 to step S233 N times (N is greater than or equal to 0), thereby increasing the sampling weight of the region of interest and reducing the sampling weights of other regions, so that more sampling points are focused on the region of interest.

[0087] Step S240: Detect obstacles within the vehicle's perception range according to the target sampling points.

[0088] Steps S210 to S240 have the following technical effects: determining the region of interest based on the current driving intention of the vehicle, increasing the sampling weight of the region of interest, so that the sampling points can focus more on the region of interest that needs attention, improving the target detection capability in the region of interest, and improving the safety of vehicle driving, thereby being able to automatically adjust the region of interest that needs attention based on the driving intention of the vehicle, and improving the target detection performance of local areas in different scenarios without increasing additional computing power.

[0089] See also Fig.12 , Fig.12 A schematic diagram of the structure of the target detection device provided by an embodiment of the present application is shown. The target detection device 200 can be integrated in a vehicle. The target detection device 200 includes a perception module 210, a sampling module 220 and a detection module 230. The perception module 210 is used to determine an area of ​​interest within the perception range of the vehicle based on the current driving intention of the vehicle. The sampling module 220 is used to increase the sampling weight of the area of ​​interest and sample target sampling points from the sensor data of the vehicle based on the sampling weight. The detection module 230 is used to detect obstacles within the perception range of the vehicle based on the target sampling points.

[0090] In some embodiments, the sampling module 220 is also used to obtain high-dimensional regulatory information, which includes information obtained after position embedding processing of information output by the vehicle's planning and control module; based on the high-dimensional regulatory information, the sampling weight of the area of ​​interest is increased.

[0091] In some embodiments, the sampling module 220 is further configured to increase the sampling weight of the region of interest by using a cross-attention mechanism and a self-attention mechanism according to the high-dimensional regulatory information.

[0092] In some embodiments, the sampling module 220 is also used to obtain sensor fusion data and initial sampling points based on the sensor data, where the sensor fusion data is data obtained by fusing different sensor data; according to the high-dimensional regulation information and the initial sampling points, a cross-attention mechanism is used to update the sampling points, and the sampling weight of the region of interest is increased; the updated sampling points are updated again using a self-attention mechanism, and the sampling weight of the region of interest is increased; according to the re-updated sampling points and the sensor fusion data, a cross-attention mechanism is used to update the sampling points, and the sampling weight of the region of interest is increased.

[0093] In some embodiments, the sampling module 220 is further used to obtain information output by the planning and control module; perform position embedding processing on the information output by the planning and control module to obtain the high-dimensional regulation and control information.

[0094] In some embodiments, the sampling module 220 is also used to obtain the vehicle's steering wheel angle and pedal signal output by the planning and control module when the vehicle is moving and the vehicle is not in the automatic driving mode; when the vehicle is moving and the vehicle is in the automatic driving mode, obtain the vehicle's steering wheel angle, pedal signal and path planning signal output by the planning and control module.

[0095] In some embodiments, the perception module 210 is further used to determine a target lane based on the current driving intention of the vehicle; and determine the area where the target lane is located within the perception range of the vehicle as an area of ​​interest.

[0096] Those skilled in the art can clearly understand that the above device provided in the embodiment of the present application can implement the method provided in the embodiment of the present application. The specific working process of the above-described device and module can refer to the corresponding process of the method in the embodiment of the present application, which will not be repeated here.

[0097] In the embodiments provided in the present application, the coupling, direct coupling or communication connection between the modules shown or discussed may be indirect coupling or communication coupling through some interfaces, devices or modules, and may be electrical, mechanical or other forms, and the embodiments of the present application do not impose specific limitations on this.

[0098] In addition, each functional module in the embodiment of the present application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0099] See also Fig.13 , Fig.13 The schematic diagram of the structure of the vehicle provided by the embodiment of the present application is shown. The vehicle 300 may include a memory 310 and a processor 320, wherein the memory 310 stores an application program, and the application program is configured to execute the method provided by the embodiment of the present application when called by the processor 320.

[0100] The processor 320 may include one or more processing cores. The processor 320 uses various interfaces and lines to connect various parts of the entire vehicle 300, and is used to run or execute instructions, programs, code sets or instruction sets stored in the memory 310, and call to run or execute data stored in the memory 310, perform various functions of the vehicle 300, and process data.

[0101] The processor 320 can be implemented in at least one of the following hardware forms: digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 320 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 320, but may be implemented separately through a communication chip.

[0102] The memory 310 may include a random access memory (RAM) or a read-only memory (ROM). The memory 310 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 310 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may store data created by the vehicle 300 during use, etc.

[0103] An embodiment of the present application also provides a computer-readable storage medium, on which a program code is stored. The program code is configured to execute the method provided in the embodiment of the present application when called by a processor.

[0104] The computer-readable storage medium may be an electronic memory such as a flash memory, an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a hard disk or a ROM.

[0105] In some embodiments, the computer readable storage medium includes a non-volatile computer readable medium (Non-Transitory Computer-Readable Storage Medium, referred to as Non-TCRSM). The computer readable storage medium has a storage space for the program code that executes any method step in the above method. These program codes can be read from or written into one or more computer program products. The program code can be compressed in an appropriate form.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A target detection method, characterized in that: include: Based on the vehicle's current driving intention, determine the area of ​​interest within the vehicle's perception range; increasing the sampling weight of the region of interest, and sampling target sampling points from the sensor data of the vehicle based on the sampling weight; Obstacles within the vehicle's perception range are detected according to the target sampling points.

2. The method according to claim 1, characterized in that: The increasing the sampling weight of the region of interest comprises: Acquiring high-dimensional regulatory information, wherein the high-dimensional regulatory information includes information obtained by performing position embedding processing on information output by a planning and control module of the vehicle; The sampling weight of the region of interest is increased according to the high-dimensional regulatory information.

3. The method according to claim 2, characterized in that The increasing the sampling weight of the region of interest according to the high-dimensional regulatory information includes: According to the high-dimensional regulatory information, a cross-attention mechanism and a self-attention mechanism are adopted to increase the sampling weight of the region of interest.

4. The method according to claim 3, characterized in that The method of increasing the sampling weight of the region of interest by using a cross attention mechanism and a self-attention mechanism according to the high-dimensional regulatory information includes: Acquire sensor fusion data and an initial sampling point based on the sensor data, wherein the sensor fusion data is data obtained by fusing different sensor data; According to the high-dimensional regulatory information and the initial sampling points, a cross-attention mechanism is used to update the sampling points and increase the sampling weight of the region of interest; P1CN-GQJT240178 and 2024P0913CN1 use the self-attention mechanism to update the updated sampling points again and increase the sampling weight of the region of interest; According to the again updated sampling points and the sensor fusion data, the sampling points are updated by adopting the cross attention mechanism to increase the sampling weight of the region of interest.

5. The method according to claim 2, characterized in that: The obtaining of high-dimensional regulatory information includes: Obtain information output from planning and control modules; The information output by the planning and control module is processed by position embedding to obtain the high-dimensional planning and control information.

6. The method according to claim 5, characterized in that The information output by the acquisition planning and control module; When the vehicle is in motion and the vehicle is not in the autonomous driving mode, obtaining a steering wheel angle and a pedal signal of the vehicle output by the planning and control module; When the vehicle is moving and in the automatic driving mode, the vehicle's steering wheel angle, pedal signal and path planning signal output by the planning and control module are obtained.

7. The method according to claim 1, characterized in that The determining of the area of ​​interest within the vehicle's perception range based on the vehicle's current driving intention includes: Determine the target lane based on the vehicle's current driving intention; The area where the target lane is located within the vehicle's perception range is determined as the area of ​​interest.

8. A target detection device, characterized in that: Said A perception module, for determining an area of ​​interest within the vehicle's perception range based on the vehicle's current driving intention; P1CN-GQJT240178, 2024P0913CN1 sampling module, used to increase the sampling weight of the region of interest, and sample target sampling points from the sensor data of the vehicle based on the sampling weight; The detection module is used to detect obstacles within the vehicle's perception range according to the target sampling points.

9. A vehicle, characterized in that: include: A memory and a processor, wherein an application is stored in the memory, and the application is configured to cause the vehicle to execute the method according to any one of claims 1 to 7 when called by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, and the program codes are configured to, when called by a processor, cause the processor to execute the method according to any one of claims 1 to 7.

Images