Obstacle detection method and device based on semantic segmentation model
By introducing a convolutional block attention mechanism during the upsampling process of the CBAM-Unet model, the problem of low detection accuracy of obstacles in automatic parking is solved, and higher segmentation accuracy and detection accuracy are achieved.
Patent Information
- Application Number
- CN202411413107.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-11
- Publication Date
- 2025-02-18
AI Technical Summary
When the existing CBAM-Unet model is used for obstacle detection during automatic parking, the image segmentation accuracy is low, resulting in corresponding reduction in obstacle detection accuracy.
By introducing a convolutional block attention mechanism in the upsampling process of the CBAM-Unet model, a weight map of the feature map is generated, and the product of the feature map and the weight map is calculated as the upsampled feature map, thereby improving the segmentation accuracy.
The CBAM-Unet model's attention to upsampled feature map features is improved, making the pixels at the boundary of the feature map clearer, thereby improving segmentation accuracy and obstacle detection accuracy.
Smart Images
Figure CN119478883A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of automatic parking, and in particular to an obstacle detection method and device based on a semantic segmentation model. Background Art
[0002] During the automatic parking process of a vehicle, the CBAM-Unet model (a deep learning image semantic segmentation model that incorporates an attention mechanism) can be used to perform semantic segmentation on the images captured by the camera to obtain two-dimensional pixel information for obstacle recognition. However, the current CBAM-Unet model has low image segmentation accuracy, resulting in low obstacle detection accuracy during automatic parking. Summary of the invention
[0003] The present invention solves the technical problem of how to improve the obstacle detection accuracy during automatic parking by providing an obstacle detection method and device based on a semantic segmentation model.
[0004] On the one hand, the present invention provides the following technical solutions:
[0005] An obstacle detection method based on a semantic segmentation model, comprising:
[0006] Obtaining image information captured by a camera and inputting the image information into a deep learning image semantic segmentation model that incorporates an attention mechanism;
[0007] Performing a downsampling process on the feature map of the image information by using the deep learning image semantic segmentation model;
[0008] The feature map obtained after the downsampling process is upsampled by the deep learning image semantic segmentation model, and a weight map of the feature map is generated based on the convolution block attention mechanism after each upsampling, and the product of the feature map and the weight map is calculated and the product is used as the feature map obtained after this upsampling;
[0009] Outputting the upsampled feature map as two-dimensional pixel information through the deep learning image semantic segmentation model;
[0010] The spatial information of the obstacle is generated based on the two-dimensional pixel information.
[0011] Optionally, the process of downsampling the feature map of the image information by using the deep learning image semantic segmentation model includes:
[0012] Cropping the feature map before downsampling to obtain a first feature map whose size is a multiple of the downsampling;
[0013] Processing the first feature map into a second feature map, wherein the size of the second feature map is equal to the original feature map obtained after the next downsampling;
[0014] Performing a 1x1 channel convolution on the second feature map to obtain a third feature map;
[0015] The sum of the third feature map and the original feature map is used as the feature map obtained after the next downsampling.
[0016] Optionally, performing an upsampling process on the feature map obtained after the downsampling process by the deep learning image semantic segmentation model includes:
[0017] Cropping the feature map before upsampling to obtain a fourth feature map whose size is a multiple of the upsampling;
[0018] Processing the fourth feature map into a fifth feature map, wherein the fifth feature map has a size equal to the original feature map obtained by the first convolution calculation after the next upsampling;
[0019] Performing a 1x1 channel convolution on the fifth feature map to obtain a sixth feature map;
[0020] The sum of the sixth feature map and the original feature map is used as the feature map obtained by the first convolution calculation after the next upsampling.
[0021] Optionally, generating spatial information of an obstacle based on the two-dimensional pixel information includes:
[0022] Mapping the two-dimensional pixel information into three-dimensional space coordinates;
[0023] Clustering the three-dimensional space coordinates of the same category into three-dimensional space coordinate sets corresponding to different obstacle individuals;
[0024] Perform spatial bounding box fitting on the three-dimensional space coordinate set corresponding to each obstacle individual to obtain the spatial information of the obstacle individual.
[0025] Optionally, after mapping the two-dimensional pixel information into three-dimensional space coordinates and before clustering the three-dimensional space coordinates of the same category into three-dimensional space coordinate sets corresponding to different obstacle individuals, generating the spatial information of the obstacle based on the two-dimensional pixel information further includes:
[0026] The noise points in the three-dimensional space coordinates are filtered out by a KDtree search algorithm.
[0027] Optionally, clustering the three-dimensional space coordinates of the same category into three-dimensional space coordinate sets corresponding to different obstacle individuals includes:
[0028] The three-dimensional space coordinates of the same category are clustered into three-dimensional space coordinate sets corresponding to different obstacle individuals through the DBSCAN clustering algorithm.
[0029] Optionally, performing spatial bounding box fitting on the three-dimensional space coordinate set corresponding to each obstacle individual includes:
[0030] The PCA algorithm is used to perform spatial bounding box fitting on the three-dimensional space coordinate set corresponding to each obstacle individual.
[0031] On the other hand, the present invention also provides the following technical solutions:
[0032] An obstacle detection device based on a semantic segmentation model, comprising:
[0033] An image acquisition module is used to acquire image information collected by a camera and input the image information into a deep learning image semantic segmentation model that incorporates an attention mechanism;
[0034] A semantic segmentation module, used for downsampling the feature map of the image information through the deep learning image semantic segmentation model;
[0035] And performing an upsampling process on the feature map obtained after the downsampling process through the deep learning image semantic segmentation model, and generating a weight map of the feature map based on the convolution block attention mechanism after each upsampling, calculating the product of the feature map and the weight map and using the product as the feature map obtained after this upsampling;
[0036] And outputting the upsampled feature map as two-dimensional pixel information through the deep learning image semantic segmentation model;
[0037] The information generation module is used to generate spatial information of the obstacle based on the two-dimensional pixel information.
[0038] On the other hand, the present invention also provides the following technical solutions:
[0039] A computer device comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any one of the above obstacle detection methods based on a semantic segmentation model.
[0040] On the other hand, the present invention also provides the following technical solutions:
[0041] A computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of any one of the above obstacle detection methods based on a semantic segmentation model.
[0042] One or more technical solutions provided by the present invention have at least the following technical effects or advantages:
[0043] In the process of upsampling the automatic parking feature map through the CBAM-Unet model, the present invention generates a weight map of the feature map based on the convolution block attention mechanism after each upsampling, calculates the product of the feature map and the weight map, and uses the product as the feature map obtained after this upsampling. This can improve the attention of the CBAM-Unet model to the features of the upsampled feature map, make the pixels at the boundary of the feature map after upsampling clearer, thereby improving the segmentation accuracy of the CBAM-Unet model, and further improving the detection accuracy of obstacles. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.
[0045] Figure 1 Schematic diagram of the principle of the CBAM-Unet model in an embodiment of the present invention;
[0046] Figure 2 is a flow chart of an obstacle detection method based on a semantic segmentation model in an embodiment of the present invention;
[0047] Figure 3 A schematic diagram of the principle of adding a convolutional block attention mechanism to the upsampling process in an embodiment of the present invention;
[0048] Figure 4 A schematic diagram of the principle of adding skip connections to the downsampling process in an embodiment of the present invention;
[0049] Figure 5 A schematic diagram of the principle of adding skip connections to the upsampling process in an embodiment of the present invention;
[0050] Figure 6 Schematic diagram of an obstacle detection device based on a semantic segmentation model according to an embodiment of the present invention. DETAILED DESCRIPTION
[0051] In order to better understand the technical solution of the present invention, the technical solution of the present invention will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0052] First, the principle of the CBAM-Unet model (deep learning image semantic segmentation model with attention mechanism) is basically introduced. The CBAM (Convolutional Block Attention Module) in the CBAM-Unet model is a convolutional block attention mechanism. Figure 1 As shown in the figure, the processing of the input image by the CBAM-Unet model includes the downsampling process of the left half, the upsampling process of the right half, and the final 1x1 convolution; the downsampling process includes 4 downsamplings, and the feature maps obtained after 4 downsamplings are feature Figure 3 , feature map 7, feature map 11 and feature map 15, and two convolution calculations are performed continuously after each downsampling; the upsampling process includes 4 upsamplings, and the feature maps obtained after the 4 upsamplings are feature map 19, feature map 23, feature map 27 and feature map 30, respectively. Two convolution calculations are performed continuously after each upsampling.
[0053] In some embodiments, the CBAM attention mechanism can be added only to the downsampled feature map and directly spliced with the upsampled feature map. This approach can enable the CBAM-Unet model to better focus on the features of the downsampled feature map, but lacks attention to the features of the upsampled feature map, resulting in low segmentation accuracy of the CBAM-Unet model and low obstacle detection accuracy.
[0054] In order to improve the obstacle detection accuracy during automatic parking, the embodiment of the present invention proposes the following Figure 2 The obstacle detection method based on the semantic segmentation model shown includes:
[0055] Step S1, obtaining image information collected by a camera and inputting the image information into a deep learning image semantic segmentation model that incorporates an attention mechanism;
[0056] Step S2, downsampling the feature map of the image information through a deep learning image semantic segmentation model;
[0057] Step S3, performing an upsampling process on the feature map obtained after the downsampling process through a deep learning image semantic segmentation model, and generating a weight map of the feature map based on the convolution block attention mechanism after each upsampling, calculating the product of the feature map and the weight map and using the product as the feature map obtained after this upsampling;
[0058] Step S4, outputting the upsampled feature map as two-dimensional pixel information through a deep learning image semantic segmentation model;
[0059] Step S5: Generate spatial information of the obstacle based on the two-dimensional pixel information.
[0060] In some embodiments, the camera in step S1 may include four fisheye cameras, so that the image information obtained is a surround view image. Step S1 is to input the original image into the CBAM-Unet model. Step S2 can obtain Figure 1 Feature Figure 17.
[0061] Step S3 is described by taking the first upsampling of feature map 17 as an example. Figure 3 As shown, the numbers in the feature map represent pixels. The closer the pixels are, the greater the probability that they belong to the same category. After upsampling, the pixels at the boundary of the feature map are easily blurred, such as the original feature map 19 obtained after the first upsampling. Step S3 generates a weight map of the original feature map 19 based on the convolution block attention mechanism, calculates the product of the original feature map 19 and its weight map, and obtains the optimized feature map 19, that is, the feature map obtained after the first upsampling.
[0062] from Figure 3 It can be seen that step S3 improves the CBAM-Unet model's attention to the upsampled feature map features, and the pixels at the boundary of the optimized feature map 19 are clearer, which means that the segmentation accuracy of the CBAM-Unet model is improved, thereby improving the detection accuracy of obstacles.
[0063] Similarly, step S3 will also generate a weight map of feature map 23 based on the convolution block attention mechanism, calculate the product of feature map 23 and its weight map and use it as the feature map obtained after the second upsampling, generate a weight map of feature map 27 based on the convolution block attention mechanism, calculate the product of feature map 27 and its weight map and use it as the feature map obtained after the third upsampling, and generate a weight map of feature map 30 based on the convolution block attention mechanism, calculate the product of feature map 30 and its weight map and use it as the feature map obtained after the fourth upsampling.
[0064] Step S4 is to perform 1x1 convolution on the feature map 32 and output two-dimensional pixel information. Step S5 can generate spatial information of the obstacle, that is, complete the obstacle detection.
[0065] Considering that some detailed features of the feature map will disappear and some pixels will be 0 during the downsampling process, it is easy to lose position information and reduce the segmentation accuracy of the CBAM-Unet model. In order to further improve the segmentation accuracy of the CBAM-Unet model, in some embodiments, step S2 may include:
[0066] The feature map before downsampling is cropped to obtain a first feature map whose size is a multiple of the downsampling;
[0067] Processing the first feature map into a second feature map, the size of the second feature map is equal to the original feature map obtained after the next downsampling;
[0068] Perform 1x1 channel convolution on the second feature map to obtain the third feature map;
[0069] The sum of the third feature map and the original feature map is used as the feature map obtained after the next downsampling.
[0070] Taking this downsampling as the first downsampling as an example, the feature map before this downsampling is the feature Figure 2 , the next downsampling is the second downsampling, and the original feature map obtained after the next downsampling is the original feature map 7. First, the feature Figure 2 The first feature map is cropped to meet the downsampling multiple (multiple of 2); then the first feature map is downsampled to obtain the second feature map; since the addition of the two feature maps needs to satisfy the width, height and number of channels are equal, the second feature map needs to be convolved with 1x1 channels to obtain the third feature map (feature map 7) whose width, height and number of channels are equal to the original feature map Figure 6 ),like Figure 4 As shown, some pixels in the original feature map 7 are 0; finally, the feature Figure 6 The sum of the original feature map 7 is taken as the feature map obtained after the second downsampling, that is, the optimized feature map 7. Each pixel in the optimized feature map 7 is not 0.
[0071] from Figure 4 It can be seen that by adding skip connections in the downsampling process, the shallow feature map (feature Figure 2 ) is downsampled and added to the current feature map (feature map 7), which enhances the transmission of position information during the downsampling process, helps the downsampling process to transfer the shallow position information to the deep layer, and can make up for the problem of disappearance of detailed features of the deep feature map during the downsampling process, thereby improving the accuracy of the model.
[0072] Similarly, when the downsampling is the second downsampling, the third feature map is feature map 10, and the sum of feature map 10 and the original feature map 11 is used as the feature map obtained after the third downsampling, that is, the optimized feature map 11; when the downsampling is the third downsampling, the third feature map is feature map 14, and the sum of feature map 14 and the original feature map 15 is used as the feature map obtained after the fourth downsampling, that is, the optimized feature map 15.
[0073] Considering that the boundary of the feature map will gradually become blurred during the upsampling process, the semantic difference between the feature maps will gradually weaken, the semantic information will be easily lost, and the segmentation accuracy of the CBAM-Unet model will be easily reduced. In order to further improve the segmentation accuracy of the CBAM-Unet model, in some embodiments, in step S3, the feature map obtained after the downsampling process is upsampled by the deep learning image semantic segmentation model, which may include:
[0074] The feature map before upsampling is cropped to obtain a fourth feature map whose size is a multiple of the upsampling;
[0075] Process the fourth feature map into the fifth feature map, and the size of the fifth feature map is equal to the original feature map obtained by the first convolution calculation after the next upsampling;
[0076] Perform 1x1 channel convolution on the fifth feature map to obtain the sixth feature map;
[0077] The sum of the sixth feature map and the original feature map is used as the feature map obtained by the first convolution calculation after the next upsampling.
[0078] Taking the upsampling as the first upsampling as an example, the feature map before the upsampling is feature map 17, the next upsampling is the second upsampling, and the original feature map obtained by the first convolution calculation after the next upsampling is the original feature map 24. First, the feature map 17 is cropped to obtain the fourth feature map that satisfies the upsampling multiple (multiples of 2); then the fourth feature map is upsampled to obtain the fifth feature map; since the addition of the two feature maps needs to satisfy the width, height and number of channels are equal, the fifth feature map needs to be convolved with 1x1 channels to obtain the sixth feature map (feature map 18) whose width, height and number of channels are equal to the original feature map 24, as shown in Figure 5 As shown; finally, the sum of the feature map 18 and the original feature map 24 is used as the feature map obtained by the first convolution calculation after the second upsampling, that is, the optimized feature map 24.
[0079] from Figure 5 It can be seen that by adding skip connections in the upsampling process, the shallow feature map (feature map 17) is upsampled and added to the current feature map (feature map 24), which enhances the transmission of semantic information in the upsampling process, helps the upsampling process to transfer the semantic information of the deep layer to the shallow layer, and can make up for the problem of weakening the semantic information of the feature map during the upsampling process, making the difference between different pixels more significant, which is conducive to improving the accuracy of the model.
[0080] Similarly, when the upsampling is the second upsampling, the sixth feature map is feature map 22, and the sum of feature map 22 and the original feature map 28 is used as the feature map obtained by the first convolution calculation after the third upsampling, that is, the optimized feature map 28; when the upsampling is the third upsampling, the sixth feature map is feature map 26, and the sum of feature map 26 and the original feature map 31 is used as the feature map obtained by the first convolution calculation after the fourth upsampling, that is, the optimized feature map 31.
[0081] In some embodiments, step S5 may include: mapping two-dimensional pixel information into three-dimensional space coordinates; filtering out noise in the three-dimensional space coordinates through a KDtree search algorithm; clustering three-dimensional space coordinates of the same category into three-dimensional space coordinate sets corresponding to different obstacle individuals; performing spatial bounding box fitting on the three-dimensional space coordinate set corresponding to each obstacle individual to obtain spatial information of the obstacle individual.
[0082] Since automatic parking is completed in a three-dimensional space coordinate system, it is necessary to restore the three-dimensional space coordinates of the obstacle. In some embodiments, the formula Map the two-dimensional pixel information to three-dimensional space coordinates. Where [uv] is the two-dimensional pixel information, [XYZ] is the transformed three-dimensional space coordinates, and H is the linear transformation matrix, which can be calculated by plane target calibration.
[0083] Considering that some noise will inevitably be introduced into the original image, it is necessary to filter out the noise points in the three-dimensional space coordinates. Assuming that there are N three-dimensional space coordinates corresponding to a certain category of obstacles, for each of the N three-dimensional space coordinates, the KDtree search algorithm can be used to search for K adjacent points of the three-dimensional space coordinate, and the average distance between the K adjacent points and the three-dimensional space coordinate can be calculated; repeat the above process N times, and N average distances can be calculated, and then the average of the N average distances is calculated. If the average distance corresponding to a certain three-dimensional space coordinate is greater than 3 times the average value, the three-dimensional space coordinate is filtered out.
[0084] In some embodiments, clustering the three-dimensional space coordinates of the same category into three-dimensional space coordinate sets corresponding to different obstacle individuals may include: clustering the three-dimensional space coordinates of the same category into three-dimensional space coordinate sets corresponding to different obstacle individuals by using a DBSCAN clustering algorithm. Setting the search radius R of the KDtree search algorithm and the point count threshold k of the DBSCAN clustering algorithm thr Taking the obstacle category as a person as an example, P is the set of three-dimensional space coordinates corresponding to all individuals, i is the sequence number of the individual, and the maximum value of i is d, C i is the set of three-dimensional space coordinates corresponding to the i-th person. Select a three-dimensional space coordinate in the set P, and use the KDtree search algorithm to find the adjacent points in the set P whose distance to the three-dimensional space coordinate is within the search radius R. If the number of adjacent points is greater than k thr , then add the adjacent points to the set C 1 The three-dimensional space coordinate set corresponding to the first person is obtained; then the three-dimensional space coordinate and the adjacent point corresponding to the first person are removed from the set P, and a new three-dimensional space coordinate is selected from the set P. The above process is repeated to obtain the three-dimensional space coordinate set C corresponding to the second person. 2Repeat the above process until the set P is empty, and finally we can get the set C that includes the three-dimensional space coordinate set corresponding to each person. 1 , C 2 ,......,C d}.
[0085] Similarly, the set C corresponding to each category of obstacles can be obtained through the DBSCAN clustering algorithm.
[0086] The three-dimensional space coordinate set corresponding to each individual after clustering is a group of discrete points, which need to be given in the form of a bounding box. In some embodiments, the three-dimensional space coordinate set corresponding to each obstacle individual is fitted with a space bounding box, which may include: fitting the space bounding box of the three-dimensional space coordinate set corresponding to each obstacle individual by a PCA algorithm. For the three-dimensional space coordinate set corresponding to a single obstacle individual, which contains n three-dimensional space coordinates P 1 , P 2 ,.......,P n , through the formula Calculate the average position P m ; Then by the formula Calculate the covariance matrix P C ; For the covariance matrix P C Using |P C -λE|=0 Solve the eigenvalue to get λ 1 , 2 , 3 , the corresponding eigenvector ε can be obtained from the eigenvalue 1 , ε 2 , ε 3 , taking the eigenvector as the natural axis direction, the maximum and minimum values along the X, Y, and Z axes can be calculated as the vertices of the cube to complete the PCA fitting.
[0087] like Figure 6 As shown, the embodiment of the present invention also provides an obstacle detection device based on a semantic segmentation model, including:
[0088] An image acquisition module is used to acquire image information collected by the camera and input the image information into a deep learning image semantic segmentation model that integrates the attention mechanism;
[0089] The semantic segmentation module is used to downsample the feature map of image information through a deep learning image semantic segmentation model;
[0090] And the feature map obtained after the downsampling process is upsampled through the deep learning image semantic segmentation model, and after each upsampling, the weight map of the feature map is generated based on the convolution block attention mechanism, the product of the feature map and the weight map is calculated, and the product is used as the feature map obtained after this upsampling;
[0091] And output the upsampled feature map as two-dimensional pixel information through a deep learning image semantic segmentation model;
[0092] The information generation module is used to generate spatial information of obstacles based on two-dimensional pixel information.
[0093] In some implementations, the semantic segmentation module may be specifically used to:
[0094] The feature map before downsampling is cropped to obtain a first feature map whose size is a multiple of the downsampling;
[0095] Processing the first feature map into a second feature map, the size of the second feature map is equal to the original feature map obtained after the next downsampling;
[0096] Perform 1x1 channel convolution on the second feature map to obtain the third feature map;
[0097] The sum of the third feature map and the original feature map is used as the feature map obtained after the next downsampling.
[0098] In some implementations, the semantic segmentation module may be specifically used to:
[0099] The feature map before upsampling is cropped to obtain a fourth feature map whose size is a multiple of the upsampling;
[0100] Process the fourth feature map into the fifth feature map, and the size of the fifth feature map is equal to the original feature map obtained by the first convolution calculation after the next upsampling;
[0101] Perform 1x1 channel convolution on the fifth feature map to obtain the sixth feature map;
[0102] The sum of the sixth feature map and the original feature map is used as the feature map obtained by the first convolution calculation after the next upsampling.
[0103] In some implementations, the information generation module may be specifically used to:
[0104] Mapping two-dimensional pixel information into three-dimensional space coordinates;
[0105] Cluster the three-dimensional space coordinates of the same category into a set of three-dimensional space coordinates corresponding to different obstacle individuals;
[0106] The spatial bounding box is fitted for the three-dimensional spatial coordinate set corresponding to each obstacle individual to obtain the spatial information of the obstacle individual.
[0107] In some implementations, the information generation module may also be specifically used to:
[0108] The noise points in the three-dimensional space coordinates are filtered out by the KDtree search algorithm.
[0109] In some implementations, the information generation module may also be used to:
[0110] The DBSCAN clustering algorithm is used to cluster the three-dimensional spatial coordinates of the same category into a set of three-dimensional spatial coordinates corresponding to different obstacle individuals.
[0111] In some implementations, the information generation module may also be used to:
[0112] The PCA algorithm is used to fit the spatial bounding box of the three-dimensional spatial coordinate set corresponding to each obstacle individual.
[0113] Based on the same inventive concept as the obstacle detection method based on the semantic segmentation model mentioned above, an embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, and the processor executes the computer program to implement the steps of any one of the obstacle detection methods based on the semantic segmentation model mentioned above.
[0114] Since the computer device introduced in the embodiment of the present invention is a computer device used to implement the obstacle detection method based on the semantic segmentation model in the embodiment of the present invention, based on the obstacle detection method based on the semantic segmentation model introduced in the embodiment of the present invention, the technical personnel in the field can understand the specific implementation of the computer device in the embodiment of the present invention and its various variations, so how the computer device implements the method in the embodiment of the present invention is not described in detail here. As long as the computer device used by the technical personnel in the field to implement the obstacle detection method based on the semantic segmentation model in the embodiment of the present invention, it belongs to the scope of protection of the present invention.
[0115] Based on the same inventive concept as the obstacle detection method based on the semantic segmentation model mentioned above, the present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the obstacle detection methods based on the semantic segmentation model mentioned above are implemented.
[0116] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0117] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0118] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0119] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0120] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. An obstacle detection method based on a semantic segmentation model, characterized in that: include: Obtaining image information captured by a camera and inputting the image information into a deep learning image semantic segmentation model that incorporates an attention mechanism; Performing a downsampling process on the feature map of the image information by using the deep learning image semantic segmentation model; The feature map obtained after the downsampling process is upsampled by the deep learning image semantic segmentation model, and a weight map of the feature map is generated based on the convolution block attention mechanism after each upsampling, and the product of the feature map and the weight map is calculated and the product is used as the feature map obtained after this upsampling; Outputting the upsampled feature map as two-dimensional pixel information through the deep learning image semantic segmentation model; The spatial information of the obstacle is generated based on the two-dimensional pixel information.
2. The obstacle detection method based on the semantic segmentation model according to claim 1, characterized in that: The process of downsampling the feature map of the image information by using the deep learning image semantic segmentation model includes: Cropping the feature map before downsampling to obtain a first feature map whose size is a multiple of the downsampling; Processing the first feature map into a second feature map, wherein the size of the second feature map is equal to the original feature map obtained after the next downsampling; Performing a 1x1 channel convolution on the second feature map to obtain a third feature map; The sum of the third feature map and the original feature map is used as the feature map obtained after the next downsampling.
3. The obstacle detection method based on the semantic segmentation model according to claim 1, characterized in that: The upsampling process of the feature map obtained after the downsampling process by the deep learning image semantic segmentation model includes: Cropping the feature map before upsampling to obtain a fourth feature map whose size is a multiple of the upsampling; Processing the fourth feature map into a fifth feature map, wherein the fifth feature map has a size equal to the original feature map obtained by the first convolution calculation after the next upsampling; Performing a 1x1 channel convolution on the fifth feature map to obtain a sixth feature map; The sum of the sixth feature map and the original feature map is used as the feature map obtained by the first convolution calculation after the next upsampling.
4. The obstacle detection method based on the semantic segmentation model according to claim 1, characterized in that: The generating the spatial information of the obstacle based on the two-dimensional pixel information includes: Mapping the two-dimensional pixel information into three-dimensional space coordinates; Clustering the three-dimensional space coordinates of the same category into three-dimensional space coordinate sets corresponding to different obstacle individuals; Perform spatial bounding box fitting on the three-dimensional space coordinate set corresponding to each obstacle individual to obtain the spatial information of the obstacle individual.
5. The obstacle detection method based on the semantic segmentation model according to claim 4, characterized in that: After mapping the two-dimensional pixel information into three-dimensional space coordinates and before clustering the three-dimensional space coordinates of the same category into three-dimensional space coordinate sets corresponding to different obstacle individuals, generating the spatial information of the obstacle based on the two-dimensional pixel information further includes: The noise points in the three-dimensional space coordinates are filtered out by a KDtree search algorithm.
6. The obstacle detection method based on the semantic segmentation model according to claim 4, characterized in that: The clustering of the three-dimensional space coordinates of the same category into three-dimensional space coordinate sets corresponding to different obstacle individuals includes: The three-dimensional space coordinates of the same category are clustered into three-dimensional space coordinate sets corresponding to different obstacle individuals through the DBSCAN clustering algorithm.
7. The obstacle detection method based on the semantic segmentation model according to claim 4, characterized in that: The performing spatial bounding box fitting on the three-dimensional space coordinate set corresponding to each obstacle individual includes: The PCA algorithm is used to perform spatial bounding box fitting on the three-dimensional space coordinate set corresponding to each obstacle individual.
8. An obstacle detection device based on a semantic segmentation model, characterized in that: include: An image acquisition module is used to acquire image information collected by a camera and input the image information into a deep learning image semantic segmentation model that incorporates an attention mechanism; A semantic segmentation module, used for downsampling the feature map of the image information through the deep learning image semantic segmentation model; And performing an upsampling process on the feature map obtained after the downsampling process through the deep learning image semantic segmentation model, and generating a weight map of the feature map based on the convolution block attention mechanism after each upsampling, calculating the product of the feature map and the weight map and using the product as the feature map obtained after this upsampling; And outputting the upsampled feature map as two-dimensional pixel information through the deep learning image semantic segmentation model; The information generation module is used to generate spatial information of the obstacle based on the two-dimensional pixel information.
9. A computer device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.