Obstacle detection method, device, electronic device and storage medium

Abstract

The present invention discloses an obstacle detection method, device, electronic device and storage medium. The method comprises obtaining a current point cloud to be detected corresponding to a scene in front of a vehicle, wherein the current point cloud to be detected is obtained by projecting a historical frame point cloud corresponding to the scene in front of the vehicle onto a current frame point cloud; constructing a first number of features for each point cloud in the current point cloud to be detected, and obtaining a processed current point cloud to be detected; inputting the processed current point cloud to be detected into a pre-trained obstacle detection model, and obtaining an obstacle detection result, wherein the obstacle detection result includes a static obstacle detection result and a dynamic obstacle detection result. The technical solution of the present invention effectively improves the information density of a single-frame point cloud, thereby improving the obstacle detection effect.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to an obstacle detection method, device, electronic equipment and storage medium. Background Art

[0002] At present, autonomous driving is based on the model of 4D millimeter-wave radar, which uses single-frame point cloud as input to realize obstacle detection. Although the number of point clouds of 4D millimeter-wave radar has increased by an order of magnitude, the number of single-frame point clouds of dual-cascade 4D millimeter-wave radar is still between hundreds and 1,000 points. Due to insufficient radar resolution and small number of valid point clouds for a single target, obstacle detection is not effective. Summary of the invention

[0003] The present invention provides an obstacle detection method, device, electronic device and storage medium to effectively improve the information density of a single-frame point cloud and thus improve the obstacle detection effect.

[0004] According to one aspect of the present invention, there is provided an obstacle detection method, the method comprising:

[0005] Acquire a current point cloud to be detected corresponding to the scene in front of the vehicle, wherein the current point cloud to be detected is obtained by projecting a historical frame point cloud corresponding to the scene in front of the vehicle onto a current frame point cloud;

[0006] Constructing a first number of features for each point cloud in the current point cloud to be detected, to obtain a processed current point cloud to be detected;

[0007] The processed current point cloud to be detected is input into a pre-trained obstacle detection model to obtain an obstacle detection result, wherein the obstacle detection result includes a static obstacle detection result and a dynamic obstacle detection result.

[0008] According to another aspect of the present invention, there is provided an obstacle detection device, the device comprising:

[0009] A point cloud acquisition module is used to acquire a current point cloud to be detected corresponding to the scene in front of the vehicle, wherein the current point cloud to be detected is obtained by projecting a historical frame point cloud corresponding to the scene in front of the vehicle onto a current frame point cloud;

[0010] A processed point cloud acquisition module is used to construct a first number of features for each point cloud in the current point cloud to be detected, so as to obtain a processed point cloud to be detected;

[0011] The detection result acquisition module is used to input the processed current point cloud to be detected into a pre-trained obstacle detection model to obtain an obstacle detection result, wherein the obstacle detection result includes a static obstacle detection result and a dynamic obstacle detection result.

[0012] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the obstacle detection method described in any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the obstacle detection method described in any embodiment of the present invention when executed.

[0017] The technical solution of the embodiment of the present invention is as follows: by acquiring a current point cloud to be detected corresponding to a scene ahead of a vehicle, the current point cloud to be detected is obtained by projecting a historical frame point cloud corresponding to the scene ahead of the vehicle onto a current frame point cloud; constructing a first number of features for each point cloud in the current point cloud to be detected, and obtaining a processed current point cloud to be detected; inputting the processed current point cloud to be detected into a pre-trained obstacle detection model to obtain an obstacle detection result, the obstacle detection result including a static obstacle detection result and a dynamic obstacle detection result, and adopting a technical means of projecting multiple frame point clouds onto one frame for obstacle detection, thereby solving the problem in the prior art that a single frame point cloud is used as input and the obstacle detection effect is poor due to a small number of valid point clouds, and effectively improving the information density of the single frame point cloud, thereby improving the obstacle detection effect.

[0018] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1a A flowchart of an obstacle detection method provided in Embodiment 1 of the present invention;

[0021] Figure 1bA flow chart of a method for implementing obstacle detection using a pre-trained obstacle detection model provided in Embodiment 1 of the present invention;

[0022] Figure 2 A schematic diagram of the structure of an obstacle detection device provided in Embodiment 2 of the present invention;

[0023] Figure 3 Schematic diagram of the structure of an electronic device for implementing the obstacle detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] Embodiment 1

[0027] Figure 1a This is a flowchart of an obstacle detection method provided in the first embodiment of the present invention. This embodiment is applicable to the case of detecting obstacles in the scene ahead of the autonomous driving. The method can be executed by an obstacle detection device. The obstacle detection device can be implemented in the form of hardware and / or software. The obstacle detection device can be configured in the main controller of the vehicle. Figure 1a As shown, the method includes:

[0028] S110, obtaining a current point cloud to be detected corresponding to a scene ahead of the vehicle, wherein the current point cloud to be detected is obtained by projecting a historical frame point cloud corresponding to the scene ahead of the vehicle onto a current frame point cloud.

[0029] Among them, the current point cloud to be detected may refer to the point cloud that can be used for obstacle detection corresponding to the scene in front of the vehicle. The point cloud data of the scene in front of the vehicle can be collected in real time while the vehicle is driving. Therefore, multiple frames of point clouds can be collected for a scene in front of the vehicle, including the current frame point cloud and the historical frame point cloud. The current point cloud to be detected can be obtained by projecting the historical frame point cloud onto the current frame point cloud. The current frame point cloud and the historical frame point cloud of this embodiment can be collected, for example, by a 4D millimeter-wave radar set on the vehicle, or by other means, which is not limited in this embodiment.

[0030] Optionally, obtaining a current point cloud to be detected corresponding to the scene in front of the vehicle may include: obtaining a current frame point cloud and a second number of historical frame point clouds corresponding to the scene in front of the vehicle; projecting the second number of historical frame point clouds onto the current frame point cloud to obtain the current point cloud to be detected.

[0031] The second number of historical frame point clouds may be determined according to a frequency of collecting frame point clouds of a scene ahead of the vehicle.

[0032] The advantage of this setting is that by projecting the historical frame point cloud onto the current frame point cloud, it is possible to effectively avoid the problem of short detectable distance due to the small number of valid point clouds in the existing technology of obstacle detection using a single frame point cloud as input.

[0033] S120 , constructing a first number of features for each point cloud in the current point cloud to be detected, to obtain a processed current point cloud to be detected.

[0034] The first number may refer to the number of features constructed for each point cloud in the current point cloud to be detected. In this embodiment, the first number may be 18, that is, 18 features may be constructed for each point cloud in the current point cloud to be detected, such as (x, y, z, vr, vc, rcs, snr, xc, yc, zc, xp, yp, ud, uv, ua, ue, id, p). The meaning of the 18-dimensional features may be as shown in Table 1:

[0035] Table 1

[0036]

[0037] The advantage of this setting is that it uses as many millimeter-wave radar point cloud attributes as possible. This is because the sparsity of millimeter-wave radar point clouds is taken into consideration. The information richness is used to make up for the shortcoming of the low number of point clouds, which can effectively improve the information density of a single-frame point cloud.

[0038] S130: Input the processed point cloud to be detected into a pre-trained obstacle detection model to obtain an obstacle detection result, where the obstacle detection result includes a static obstacle detection result and a dynamic obstacle detection result.

[0039] In this embodiment, a pre-trained obstacle detection model can be used to detect static obstacles and dynamic obstacles in the scene ahead of the vehicle by taking the processed current point cloud to be detected as input.

[0040] In an optional implementation, inputting the processed current point cloud to be detected into a pre-trained obstacle detection model to obtain an obstacle detection result may include: using a columnar feature network of the obstacle detection model to convert the processed current point cloud to be detected into a pseudo image; using a backbone network of the obstacle detection model to extract multi-scale features from the pseudo image; using a neck network of the obstacle detection model to perform weighted processing on the multi-scale features to obtain weighted multi-scale features; and using a detection head of the obstacle detection model to detect the weighted multi-scale features to determine the obstacle center.

[0041] refer to Figure 1b , project the historical frame point clouds (01 frame, 02 frame, 03 frame, 04 frame point clouds) to the current frame point cloud (00 frame), and construct 18-dimensional features for each point cloud in the projected current point cloud to be detected. Further, input the current point cloud to be detected with 18-dimensional features into the pre-trained obstacle detection model, use PFN (Pillar Feature Net) to process the current point cloud to be detected to obtain the corresponding pseudo image, use backbone to process the pseudo image to obtain multi-scale features, use static neck and motion neck to weight the multi-scale features output by the backbone network respectively to obtain weighted multi-scale features, use static head to process the weighted multi-scale features output by static neck to determine the center of static obstacles, and use motion head to process the weighted multi-scale features output by motion neck to determine the center of dynamic obstacles.

[0042] Optionally, using a columnar feature network of an obstacle detection model to convert the processed current point cloud to be detected into a pseudo image can include: using a columnar feature network to divide the processed current point cloud to be detected into multiple point cloud columnar units on the XY plane; performing quantity restriction processing on the point cloud in each point cloud columnar unit to obtain a first tensor, and the dimension of the first tensor is (D, P, N); compressing the first tensor in D and N dimensions to obtain a second tensor, and the dimension of the second tensor is (C, P); converting the second tensor back to the coordinates of multiple point cloud columnar units in the P dimension to obtain a pseudo image, and the specifications of the pseudo image are (C, H, W), where H represents the height of the pseudo image and W represents the width of the pseudo image.

[0043] refer to Figure 1b, the point cloud to be detected after the processing of the input PFN can be gridded in the xy plane, and one grid represents a pillar (point cloud columnar unit); each pillar is represented by (P, N, D) to obtain a first tensor, wherein P can represent the number of point cloud columnar units, N can represent the number of point clouds in each point cloud columnar unit (if the number of point clouds in each pillar is less than N, 0 can be added, if it exceeds N, N point clouds can be selected from it by random sampling), D can represent the number of feature dimensions of each point cloud, and D is the first number (for example, 18); the first tensor can be processed into a tensor of (P, N, C) by a multi-layer perceptron, and then a second tensor of dimension (P, C) is obtained by performing maximum pooling on N points; the second tensor is further mapped back to the XY plane in the P dimension to obtain a pseudo image, whose size is (H, W, C).

[0044] Optionally, using the backbone network of the obstacle detection model to extract multi-scale features from the pseudo image can include: using the downsampling network of the backbone network to extract features of different scales from the pseudo image to obtain first feature extraction results; using the upsampling network of the backbone network to upsample each first feature extraction result to the same dimension through a deconvolution operation to obtain second feature extraction results; and splicing the second feature extraction results to obtain multi-scale features.

[0045] refer to Figure 1b , the pseudo image of (H, W, C) can be downsampled three times to 8 times, and then upsampled three times to (H / 2, W / 2, 2C), and further feature concatenation is performed to obtain the multi-scale features of (H / 2, W / 2, 6C).

[0046] Optionally, using the neck network of the obstacle detection model to perform weighted processing on the multi-scale features to obtain weighted multi-scale features can include: using the static neck network in the neck network to use the spatial attention mechanism on the multi-scale features, and determining corresponding static weights for different positions of the multi-scale features to obtain the first weighted multi-scale features; using the dynamic neck network in the neck network to use the spatial attention mechanism on the multi-scale features, and determining corresponding dynamic weights for different positions of the multi-scale features to obtain the second weighted multi-scale features.

[0047] refer to Figure 1b The static neck network can use the spatial attention mechanism to determine the static weights for the multi-scale features and obtain the first weighted multi-scale features. The dynamic neck network can use the spatial attention mechanism to determine the dynamic weights for the multi-scale features and obtain the second weighted multi-scale features.

[0048] An important feature of millimeter-wave radar is speed differentiation. Therefore, the detection head uses the multi-scale features output by the shared backbone network and the spatial main mechanism to generate two necks, which are used to predict static obstacles and dynamic obstacles respectively. This can effectively improve the recognition rate of moving obstacles, and through the diversion at the neck, the step-by-step training of the obstacle detection model can be realized.

[0049] Optionally, using the detection head of the obstacle detection model to detect the weighted multi-scale features and determine the obstacle center can include: using a static obstacle detection head in the detection head to detect the first weighted multi-scale features and determine the static obstacle center; using a dynamic obstacle detection head in the detection head to detect the second weighted multi-scale features and determine the dynamic obstacle center.

[0050] The detection head of this embodiment can use a center head as a detection head. Compared with a traditional detection head, it has a faster convergence speed and higher accuracy, which can improve the detection accuracy and improve the detection of static obstacles more.

[0051] The technical solution of the embodiment of the present invention is as follows: by acquiring a current point cloud to be detected corresponding to a scene ahead of a vehicle, the current point cloud to be detected is obtained by projecting a historical frame point cloud corresponding to the scene ahead of the vehicle onto a current frame point cloud; constructing a first number of features for each point cloud in the current point cloud to be detected, and obtaining a processed current point cloud to be detected; inputting the processed current point cloud to be detected into a pre-trained obstacle detection model to obtain an obstacle detection result, the obstacle detection result including a static obstacle detection result and a dynamic obstacle detection result, and adopting a technical means of projecting multiple frame point clouds onto one frame for obstacle detection, thereby solving the problem in the prior art that a single frame point cloud is used as input and the obstacle detection effect is poor due to a small number of valid point clouds, and effectively improving the information density of the single frame point cloud, thereby improving the obstacle detection effect.

[0052] Embodiment 2

[0053] Figure 2 This is a schematic diagram of the structure of an obstacle detection device provided in Embodiment 2 of the present invention. Figure 2 As shown, the device includes: a point cloud acquisition module 210, a processed point cloud acquisition module 220 and a detection result acquisition module 230. Among them:

[0054] Point cloud acquisition module 210, used to acquire a current point cloud to be detected corresponding to the scene in front of the vehicle, wherein the current point cloud to be detected is obtained by projecting a historical frame point cloud corresponding to the scene in front of the vehicle onto a current frame point cloud;

[0055] A processed point cloud acquisition module 220 is used to construct a first number of features for each point cloud in the current point cloud to be detected, so as to obtain a processed point cloud to be detected;

[0056] The detection result acquisition module 230 is used to input the processed current point cloud to be detected into the pre-trained obstacle detection model to obtain an obstacle detection result, which includes a static obstacle detection result and a dynamic obstacle detection result.

[0057] The technical solution of the embodiment of the present invention is as follows: by acquiring a current point cloud to be detected corresponding to a scene ahead of a vehicle, the current point cloud to be detected is obtained by projecting a historical frame point cloud corresponding to the scene ahead of the vehicle onto a current frame point cloud; constructing a first number of features for each point cloud in the current point cloud to be detected, and obtaining a processed current point cloud to be detected; inputting the processed current point cloud to be detected into a pre-trained obstacle detection model to obtain an obstacle detection result, the obstacle detection result including a static obstacle detection result and a dynamic obstacle detection result, and adopting a technical means of projecting multiple frame point clouds onto one frame for obstacle detection, thereby solving the problem in the prior art that a single frame point cloud is used as input and the obstacle detection effect is poor due to a small number of valid point clouds, and effectively improving the information density of the single frame point cloud, thereby improving the obstacle detection effect.

[0058] Optionally, the point cloud acquisition module 210 may be specifically used for:

[0059] Obtaining a current frame point cloud and a second number of historical frame point clouds corresponding to the scene ahead of the vehicle;

[0060] The second number of historical frame point clouds are projected onto the current frame point cloud to obtain the current point cloud to be detected.

[0061] Optionally, the detection result acquisition module 230 may include:

[0062] A pseudo image conversion unit, used for converting the processed current point cloud to be detected into a pseudo image by using a columnar feature network of an obstacle detection model;

[0063] A multi-scale feature extraction unit, used to extract multi-scale features from the pseudo image using the backbone network of the obstacle detection model;

[0064] A weighted multi-scale feature acquisition unit, used for performing weighted processing on the multi-scale features using the neck network of the obstacle detection model to obtain weighted multi-scale features;

[0065] The obstacle center determination unit is used to detect the weighted multi-scale features using the detection head of the obstacle detection model to determine the obstacle center.

[0066] Optionally, the pseudo image conversion unit may be specifically used for:

[0067] Using the columnar feature network, the processed current point cloud to be detected is divided into a plurality of point cloud columnar units on the XY plane;

[0068] Performing quantity restriction processing on the point clouds in each point cloud columnar unit to obtain a first tensor, wherein the dimension of the first tensor is (D, P, N), wherein D represents the number of feature dimensions of each point cloud, D is the first number, P represents the number of the point cloud columnar units, and N represents the number of point clouds in each point cloud columnar unit;

[0069] Compress the first tensor in D dimension and N dimension to obtain a second tensor, where the dimension of the second tensor is (C, P);

[0070] The second tensor is converted back to the coordinates of the multiple point cloud columnar units in the P dimension to obtain the pseudo image, and the specifications of the pseudo image are (C, H, W), where H represents the height of the pseudo image and W represents the width of the pseudo image.

[0071] Optionally, the multi-scale feature extraction unit can be used to:

[0072] Using the downsampling network of the backbone network to extract features of different scales from the pseudo image, to obtain first feature extraction results;

[0073] Using the upsampling network of the backbone network to upsample the first feature extraction results to the same dimension through a deconvolution operation, thereby obtaining second feature extraction results;

[0074] The second feature extraction results are spliced ​​to obtain the multi-scale feature.

[0075] Optionally, the weighted multi-scale feature acquisition unit can be used for:

[0076] Using a static neck network in the neck network to apply a spatial attention mechanism to the multi-scale feature, respectively determining corresponding static weights for different positions of the multi-scale feature, and obtaining a first weighted multi-scale feature;

[0077] The dynamic neck network in the neck network uses a spatial attention mechanism on the multi-scale feature, and corresponding dynamic weights are determined for different positions of the multi-scale feature to obtain a second weighted multi-scale feature.

[0078] Optionally, the obstacle center determination unit may be specifically used for:

[0079] Using a static obstacle detection head in the detection head to detect the first weighted multi-scale features to determine the center of the static obstacle;

[0080] The second weighted multi-scale features are detected using a dynamic obstacle detection head in the detection head to determine a dynamic obstacle center.

[0081] The obstacle detection device provided in the embodiment of the present invention can execute the obstacle detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0082] Embodiment 3

[0083] Figure 3 A schematic diagram of an electronic device 300 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers or various forms of mobile devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0084] like Figure 3 As shown, the electronic device 300 includes at least one processor 301, and a memory connected to the at least one processor 301 in communication, such as a read-only memory (ROM) 302, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 301 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 302 or the computer program loaded from the storage unit 308 to the random access memory (RAM) 13. In the RAM 303, various programs and data required for the operation of the electronic device 300 can also be stored. The processor 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0085] A number of components in the electronic device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the electronic device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0086] The processor 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 301 performs the various methods and processes described above, such as the obstacle detection method.

[0087] In some embodiments, the obstacle detection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the processor 301, one or more steps of the obstacle detection method described above may be performed. Alternatively, in other embodiments, the processor 301 may be configured to perform the obstacle detection method in any other appropriate manner (e.g., by means of firmware).

[0088] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0089] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0090] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0091] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0092] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0093] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0094] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0095] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An obstacle detection method, characterized in that: include: Acquire a current point cloud to be detected corresponding to the scene in front of the vehicle, wherein the current point cloud to be detected is obtained by projecting a historical frame point cloud corresponding to the scene in front of the vehicle onto a current frame point cloud; Constructing a first number of features for each point cloud in the current point cloud to be detected to obtain the processed current point cloud to be detected; the first number of features are 18-dimensional features, and the 18-dimensional features include single point x, single point y, single point z, radial velocity, compensated radial velocity, radar scattering interface, signal-to-noise ratio, point cloud center point x, point cloud center point y, point cloud center point z, distance from pillar center point x, distance from pillar center point y, distance unit, velocity unit, horizontal angle unit, pitch angle unit, multi-frame ID and number of point clouds in pillar; The processed point cloud to be detected is converted into a pseudo image by using a columnar feature network of an obstacle detection model; Extracting multi-scale features from the pseudo image using the backbone network of the obstacle detection model; Using a static neck network of the neck network in the obstacle detection model to apply a spatial attention mechanism to the multi-scale feature, respectively determine corresponding static weights for different positions of the multi-scale feature, and obtain a first weighted multi-scale feature; Using a dynamic neck network in the neck network to apply a spatial attention mechanism to the multi-scale feature, respectively determining corresponding dynamic weights for different positions of the multi-scale feature, and obtaining a second weighted multi-scale feature; Using a static obstacle detection head of a detection head in the obstacle detection model to detect the first weighted multi-scale features to determine a static obstacle center; The second weighted multi-scale features are detected using a dynamic obstacle detection head in the detection head to determine a dynamic obstacle center.

2. The method according to claim 1, characterized in that Get the current point cloud to be detected corresponding to the scene in front of the vehicle, including: Obtaining a current frame point cloud and a second number of historical frame point clouds corresponding to the scene ahead of the vehicle; The second number of historical frame point clouds are projected onto the current frame point cloud to obtain the current point cloud to be detected.

3. The method according to claim 1, characterized in that The processed current point cloud to be detected is converted into a pseudo image using a columnar feature network of an obstacle detection model, including: Using the columnar feature network, the processed current point cloud to be detected is divided into a plurality of point cloud columnar units on the XY plane; Performing quantity restriction processing on the point clouds in each point cloud columnar unit to obtain a first tensor, wherein the dimension of the first tensor is (D, P, N), wherein D represents the number of feature dimensions of each point cloud, D is the first number, P represents the number of the point cloud columnar units, and N represents the number of point clouds in each point cloud columnar unit; Compress the first tensor in D dimension and N dimension to obtain a second tensor, where the dimension of the second tensor is (C, P); The second tensor is converted back to the coordinates of the multiple point cloud columnar units in the P dimension to obtain the pseudo image, and the specifications of the pseudo image are (C, H, W), where H represents the height of the pseudo image and W represents the width of the pseudo image.

4. The method according to claim 1, characterized in that: Extracting multi-scale features from the pseudo image using the backbone network of the obstacle detection model includes: Using the downsampling network of the backbone network to extract features of different scales from the pseudo image, to obtain first feature extraction results; Using the upsampling network of the backbone network to upsample the first feature extraction results to the same dimension through a deconvolution operation, thereby obtaining second feature extraction results; The second feature extraction results are spliced ​​to obtain the multi-scale feature.

5. An obstacle detection device, characterized in that: include: A point cloud acquisition module is used to acquire a current point cloud to be detected corresponding to the scene in front of the vehicle, wherein the current point cloud to be detected is obtained by projecting a historical frame point cloud corresponding to the scene in front of the vehicle onto a current frame point cloud; a processed point cloud acquisition module, used for constructing a first number of features for each point cloud in the current point cloud to be detected, and obtaining the processed point cloud to be detected; the first number of features is an 18-dimensional feature, and the 18-dimensional features include single point x, single point y, single point z, radial velocity, compensated radial velocity, radar scattering interface, signal-to-noise ratio, point cloud center point x, point cloud center point y, point cloud center point z, distance from pillar center point x, distance from pillar center point y, distance unit, velocity unit, horizontal angle unit, pitch angle unit, multi-frame ID and number of point clouds in pillar; A pseudo image conversion unit, used for converting the processed current point cloud to be detected into a pseudo image by using a columnar feature network of an obstacle detection model; A multi-scale feature extraction unit, used to extract multi-scale features from the pseudo image using the backbone network of the obstacle detection model; A weighted multi-scale feature acquisition unit, configured to use a spatial attention mechanism on the multi-scale feature using a static neck network of the neck network in the obstacle detection model, determine corresponding static weights for different positions of the multi-scale feature, and obtain a first weighted multi-scale feature; Using a dynamic neck network in the neck network to apply a spatial attention mechanism to the multi-scale feature, respectively determining corresponding dynamic weights for different positions of the multi-scale feature, and obtaining a second weighted multi-scale feature; The obstacle center determination unit is used to detect the first weighted multi-scale features using the static obstacle detection head of the detection head in the obstacle detection model to determine the static obstacle center; and detect the second weighted multi-scale features using the dynamic obstacle detection head in the detection head to determine the dynamic obstacle center.

6. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the obstacle detection method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the obstacle detection method according to any one of claims 1 to 4 when executed.

Images