Occluded target detection method and related device

Abstract

Embodiments of the present application disclose a method for detecting an occluded target and related equipment. The method comprises: obtaining a training sample set and inputting the training sample set into an occluded target detection network to obtain candidate data in the training sample set; performing iterative operation on the sparse point cloud data by using an encoder and a decoder in the occluded target detection network to obtain occluded target overall shape point cloud data; obtaining aggregated features by using a channel attention fusion operation in the occluded target detection network according to the occluded target overall shape point cloud data, a known region point set and sparse point cloud features; performing feature refinement iterative calculation by using a multi-scale transformer model according to the aggregated features and the known region point set to obtain occluded target global features; performing confidence calculation operation and position regression operation in the occluded target detection network based on the occluded target global features to generate an occluded target detection model; and detecting a to-be-detected sample set by using the occluded target detection model to complete occluded target detection.

Description

TECHNICAL FIELD

[0001] The present specification relates to the field of target recognition, and more particularly, to a method for detecting occluded targets and related equipment. BACKGROUND

[0002] The unmanned driving technology can automatically plan a safe and reasonable driving route, avoid traffic congestion and effectively reduce the probability of traffic accidents. However, unmanned driving itself is a highly challenging task. The complexity of the environment makes it difficult to accurately locate the target, thereby affecting the driving safety. Therefore, it is crucial to study a target detection method for accurately detecting and perceiving specific target information in the surrounding environment to ensure the safety of unmanned driving.

[0003] Traditional two-dimensional target detection methods use RGB images as input data. However, RGB images are easily affected by environmental factors such as light and climate. Moreover, two-dimensional images lack three-dimensional depth information, making it difficult to accurately locate the three-dimensional spatial position of the target. Compared with two-dimensional images, three-dimensional point cloud data collected by devices such as laser radars contain high-precision depth information in complex environments, which can more accurately locate the target. Moreover, the detection performance is not affected by environmental factors. Therefore, three-dimensional target detection methods based on point cloud data are widely used in the field of unmanned driving.

[0004] However, due to the occlusion between targets in actual scenes, it is difficult for point cloud collection devices to obtain complete shape point cloud data of occluded targets, resulting in a lack of sufficient shape feature information of such occluded targets, which makes it difficult to be detected, thereby affecting the detection performance of the unmanned driving system in complex environments. Therefore, it is of great significance to study the detection method of occluded targets in complex scenes to improve the ability of the unmanned driving system to perceive target information in complex environments. SUMMARY

[0005] A series of simplified concepts are introduced in the summary section, which will be further described in detail in the specific embodiment section. The summary section of the present application does not mean to attempt to limit the key features and necessary technical features of the claimed technical solution, nor does it mean to determine the protection scope of the claimed technical solution.

[0006] In order to improve the recognition accuracy of occluded targets, in a first aspect, the present application provides a method for detecting occluded targets, which comprises:

[0007] Obtaining a training sample set and inputting it into an occluded target detection network to obtain candidate data in the training sample set through a point cloud multi-scale feature extraction subnetwork in the occluded target detection network, wherein the candidate data includes a candidate box, sparse point cloud data and sparse point cloud features;

[0008] The sparse point cloud data is iteratively operated based on the sparse point cloud data by using the encoder and the decoder in the occlusion target detection network to obtain occlusion region shape features, and shape point cloud reconstruction is performed in the occlusion target detection network according to the occlusion region shape features to obtain occlusion target overall shape point cloud data.

[0009] According to the occlusion target overall shape point cloud data, a known region point set and the sparse point cloud features, channel attention fusion is performed in the occlusion target detection network to obtain aggregated features, wherein the known region point set is randomly sampled based on the sparse point cloud data.

[0010] Based on the aggregated features and the known region point set, feature refinement is iteratively calculated by using a multi-scale transformer model in the occlusion target detection network to obtain occlusion target global features.

[0011] Based on the occlusion target global features, confidence calculation and position regression are performed in the occlusion target detection network to obtain a confidence score and a regression offset of the occlusion target in the training sample set, and the parameters of the occlusion target detection network are updated by using a loss function based on the confidence score and the regression offset to learn and generate an occlusion target detection model.

[0012] The occlusion target detection model is used to detect a to-be-detected sample set to obtain an occlusion target detection frame and occlusion target category information, and occlusion target detection is completed.

[0013] Optionally, the iterative operation includes a first iterative operation and a second iterative operation.

[0014] The sparse point cloud data is iteratively operated based on the sparse point cloud data by using the encoder and the decoder in the occlusion target detection network to obtain occlusion region shape features, and shape point cloud reconstruction is performed in the occlusion target detection network according to the occlusion region shape features to obtain occlusion target overall shape point cloud data, including:

[0015] According to the sparse point cloud data, graph embedding is performed in the occlusion target detection network to obtain graph embedding local structure features, and the first iterative operation is performed by using the encoder based on the graph embedding local structure features to obtain encoder output structure features, and projection mapping is performed in the occlusion target detection network on the encoder output structure features to obtain an occlusion region center point sequence and occlusion region center point structure features.

[0016] The first embedding operation is performed in the occlusion target detection network on the occlusion region center point sequence and the occlusion region center point structure features to obtain occlusion region initial local shape features.

[0017] According to the initial local shape feature of the occlusion region and the encoder output structure feature, the second iteration operation and the dimension transformation operation are performed by using the decoder to obtain the occlusion region shape feature, and the shape point cloud reconstruction operation is performed by using the occlusion region shape feature, the occlusion region center point sequence and the sparse point cloud data to obtain the overall shape point cloud data of the occlusion target.

[0018] Optionally, the graph embedding operation includes a first iteration farthest point sampling operation, a graph convolution operation and a second embedding operation, and the projection mapping operation includes a global pooling operation and a first multi-layer perceptron.

[0019] The graph embedding local structure feature is obtained by performing the graph embedding operation in the occlusion target detection network according to the sparse point cloud data, and the encoder output structure feature is obtained by performing the first iteration operation on the basis of the graph embedding local structure feature by using the encoder.

[0020] The known region center point sequence and the known region topology feature are obtained by performing the first iteration farthest point sampling operation and the graph convolution operation on the sparse point cloud data.

[0021] The known region center point sequence and the known region topology feature are obtained by performing the first iteration farthest point sampling operation and the graph convolution operation on the sparse point cloud data.

[0022] The graph embedding local structure feature is obtained by performing the second embedding operation on the known region center point sequence and the known region topology feature.

[0023] The encoder output structure feature is obtained by performing the first iteration operation on the basis of the graph embedding local structure feature by using the encoder, and the first iteration operation is performed on the basis of the feature self-encoding operation and the first feedforward network.

[0024] The global information is extracted by using the global pooling operation according to the encoder output structure feature.

[0025] The occlusion region center point sequence and the occlusion region center point structure feature are obtained by using the first multi-layer perceptron on the basis of the global information.

[0025] Optionally, the shape point cloud reconstruction operation includes a first splicing operation, a second multi-layer perceptron and a second splicing operation.

[0026] The above-mentioned second iterative operation and dimension transformation operation are performed on the above-mentioned initial local shape feature of the occlusion region and the above-mentioned encoder output structure feature by using the above-mentioned decoder to obtain the above-mentioned occlusion region shape feature, and the above-mentioned shape point cloud reconstruction operation is performed on the above-mentioned occlusion region shape feature, the above-mentioned occlusion region center point sequence and the above-mentioned sparse point cloud data to obtain the above-mentioned overall shape point cloud data of the occlusion target, comprising:

[0027] The above-mentioned initial local shape feature of the occlusion region is taken as the first layer input shape feature of the above-mentioned decoder, and the above-mentioned second iterative operation is performed in combination with the above-mentioned encoder output structure feature to obtain the decoder output shape feature, wherein the above-mentioned second iterative operation is based on feature self-decoding operation, feature cross-decoding operation and second feedforward network decoding operation;

[0028] The above-mentioned dimension transformation operation is performed on the above-mentioned decoder output shape feature by using the third multilayer perceptron to obtain the above-mentioned occlusion region shape feature;

[0029] The above-mentioned first splicing operation is performed based on the above-mentioned occlusion region center point sequence and the above-mentioned occlusion region shape feature to obtain the first splicing feature;

[0030] The above-mentioned spatial mapping is performed on the above-mentioned first splicing feature by using the above-mentioned second multilayer perceptron to obtain the occlusion region point cloud data;

[0031] The above-mentioned second splicing operation is performed on the above-mentioned occlusion region point cloud data and the above-mentioned sparse point cloud data to obtain the above-mentioned overall shape point cloud data of the occlusion target.

[0032] Optionally, the above-mentioned channel attention fusion operation is used in the above-mentioned occlusion target detection network to obtain the aggregated feature based on the above-mentioned overall shape point cloud data of the occlusion target, the known region point set and the above-mentioned sparse point cloud feature, comprising:

[0033] The above-mentioned overall shape feature of the occlusion target is obtained by using the fourth multilayer perceptron to aggregate the above-mentioned overall shape point cloud data of the occlusion target within the neighborhood radius range of the points in the above-mentioned known region point set;

[0034] The above-mentioned known feature is sampled in the above-mentioned sparse point cloud feature according to the index number of the points in the above-mentioned known region point set, and the above-mentioned known region position feature is obtained by updating the above-mentioned known feature by using the convolutional network;

[0035] The above-mentioned initial mixed feature is obtained by using the third splicing operation based on the above-mentioned overall shape feature of the occlusion target and the above-mentioned known region position feature;

[0036] The above-mentioned channel attention feature is obtained by performing the point-by-point pooling operation based on the above-mentioned initial mixed feature;

[0037] The first linear projection operation and the normalization operation are performed on the channel attention feature to obtain a channel attention map. The channel attention map and the known region position feature are multiplied to obtain the aggregated feature after the second linear projection operation.

[0038] Optionally, the feature refinement iterative calculation includes a second iterative farthest point sampling operation, an aggregation operation, a self-attention operation, a first cross-attention operation, a third linear projection operation, a second cross-attention operation, and a third feedforward network.

[0039] The feature refinement iterative calculation based on the aggregated feature and the known region point set using the multi-scale transformer model in the occlusion target detection network to obtain the occlusion target global feature includes:

[0040] The aggregated feature and the known region point set are respectively taken as the first layer input feature and the first layer input point set of the multi-scale transformer model. The second iterative farthest point sampling operation is performed based on the first layer input point set of the multi-scale transformer model to obtain a first layer center point set, wherein the first layer center point set is a first layer output point set.

[0041] The aggregation operation is used based on the first layer center point set to respectively aggregate the first layer input feature of the multi-scale transformer model to obtain a first layer first scale feature and a first layer second scale feature. The self-attention operation is performed on the first layer first scale feature to obtain a first layer first scale self-attention feature. The first cross-attention operation is performed based on the first layer first scale self-attention feature and the first layer second scale feature to obtain a first layer first resolution feature.

[0042] For the first layer input feature, the third linear projection operation is performed to obtain a first layer second resolution feature. The second cross-attention operation is performed based on the first layer first resolution feature and the first layer second resolution feature, and the third feedforward network is updated to obtain a first layer output feature, completing the first refinement calculation of the feature refinement iterative calculation.

[0043] Based on the first layer output feature and the first layer output point set, the multi-scale transformer model continues to perform the target number of refinement calculations of the feature refinement iterative calculation through the remaining layers of the multi-scale transformer model to obtain a last layer output point set and a last layer output feature of the multi-scale transformer model to generate the occlusion target global feature, wherein the remaining layers of the multi-scale transformer model do not include the first layer of the multi-scale transformer model.

[0044] Optionally, the confidence calculation operation and the position regression operation in the occlusion target detection network based on the global feature of the occlusion target are performed to obtain a confidence score and a regression offset of the occlusion target in the training sample set, and a loss function is used to constrain the parameters of the occlusion target detection network based on the confidence score and the regression offset to learn and generate an occlusion target detection model, including:

[0045] The confidence calculation operation is performed based on the global feature of the occlusion target using a first fully connected neural network to generate the confidence score of the occlusion target in the training sample set to determine the category value of the occlusion target;

[0046] The position regression operation is performed based on the global feature of the occlusion target using a second fully connected neural network to generate the regression offset of the candidate box corresponding to the occlusion target, and the difference between the length, width, height, angle parameter and center point coordinate of the candidate box and the length, width, height, angle parameter and center point coordinate of the real target box in the training sample set is taken as the regression target;

[0047] The cross-entropy classification loss is calculated based on the category value of the occlusion target and the real target category label in the training sample set, and the position regression loss is calculated based on the regression offset of the candidate box corresponding to the occlusion target and the regression target, and the parameters of the occlusion target detection network are updated based on the sum of the cross-entropy classification loss and the position regression loss to generate the occlusion target detection model.

[0048] In a second aspect, the present application further provides an occlusion target detection device, comprising:

[0049] The extraction unit is configured to obtain a training sample set and input the training sample set into an occlusion target detection network, and obtain candidate data in the training sample set through a point cloud multi-scale feature extraction sub-network in the occlusion target detection network, wherein the candidate data includes a candidate box, sparse point cloud data and sparse point cloud features;

[0050] The encoding and decoding unit is configured to perform iterative operation on the sparse point cloud data using an encoder and a decoder in the occlusion target detection network to obtain occlusion region shape features, and perform shape point cloud reconstruction operation in the occlusion target detection network based on the occlusion region shape features to obtain overall shape point cloud data of the occlusion target;

[0051] The fusion unit is configured to obtain aggregated features by channel attention fusion operation in the occlusion target detection network based on the overall shape point cloud data of the occlusion target, a known region point set and the sparse point cloud features, wherein the known region point set is randomly sampled based on the sparse point cloud data;

[0052] The acquisition unit is configured to acquire global features of the occluded target by performing feature refinement iteration calculation based on the aggregated features and the known region point set in the occluded target detection network.

[0053] The generation unit is configured to perform confidence calculation operation and position regression operation in the occluded target detection network based on the global features of the occluded target, to acquire a confidence score and a regression offset of the occluded target in the training sample set, and to update parameters of the occluded target detection network by using a loss function to constrain the confidence score and the regression offset, thereby learning and generating an occluded target detection model.

[0054] The detection unit is configured to detect a to-be-detected sample set by using the occluded target detection model to obtain an occluded target detection frame and occluded target category information, thereby completing occluded target detection.

[0055] In a third aspect, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor is configured to implement the steps of the occluded target detection method according to any one of the first aspect when executing the computer program stored in the memory.

[0056] In a fourth aspect, the present application further provides a computer readable storage medium having a computer program stored thereon, and the computer program is executable by a processor to implement the occluded target detection method according to any one of the first aspect.

[0057] In summary, the application provides an occluded object detection method. The training sample set is input into an occluded object detection network, and a point cloud multi-scale feature extraction subnetwork is used to obtain candidate data in the scene point cloud. An encoder and a decoder are used for iterative operation to complete and reconstruct the overall shape of the sparse point cloud data in the candidate box to enhance the shape features of the occluded object. Channel attention fusion operation is used to aggregate the overall shape features of the occluded object obtained by the completion and reconstruction operation and the known region position features of the candidate box before completion that contain the position information of the occluded object, to obtain aggregated features that sufficiently fuse the spatial geometric position information of the occluded object. An improved multi-scale transformer model is used for feature refinement iterative calculation to sufficiently refine the aggregated features, enhance the key point information, generate global features of the occluded object, and finally perform confidence calculation and position regression operation based on the global features of the occluded object to obtain a confidence score and a regression offset. The loss value is calculated by using a loss function on the true label data in the training sample set to constrain and adjust the parameters of the occluded object detection network, and the occluded object detection model is learned to detect the sample set to be detected, generate an occluded object detection box and occluded object category information, and complete the occluded object detection. The completion and reconstruction operation in the occluded object detection method effectively extracts the structure information of the sparse point cloud data based on the encoder-decoder architecture, reconstructs the overall shape point cloud data of the occluded object, and generates the overall shape features of the occluded object to enhance the shape information of the occluded object. The channel attention fusion operation in the detection method can sufficiently fuse the shape features and position features with different semantic information from different modes to generate aggregated features with enhanced spatial geometric information of the occluded object. The multi-scale transformer model in the detection method effectively captures the multi-level semantic correlation of the aggregated features containing rich spatial geometric information by attention interaction calculation on different scale features, and refines and enhances the key point features of the occluded object. The occluded object detection method can reconstruct the shape features of the occluded object, refine the key point information of the occluded object, and effectively improve the detection performance of the occluded object by combining the completion and reconstruction operation, the channel attention fusion operation, and the multi-scale transformer model.

[0058] The occluded object detection method of the application, other advantages, objects and features of the application will be embodied in part by the following description, and will be understood by those skilled in the art through research and practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0059] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of preferred embodiments and are not meant to limit the present description. Moreover, the same reference numerals are used throughout the various drawings to represent similar or same components. In the drawings:

[0060] Figure 1 A flowchart of a method for detecting an occluded target according to an embodiment of the present application is shown in FIG. 1.

[0061] Figure 2 A flowchart of a method for generating three-dimensional target candidate data according to an embodiment of the present application is shown in FIG. 2.

[0062] Figure 3 A structure diagram of a method for completing and reconstructing an occluded target detection model according to an embodiment of the present application is shown in FIG. 3.

[0063] Figure 4 A structure diagram of a method for detecting an occluded target according to an embodiment of the present application is shown in FIG. 4.

[0064] Figure 5 A comparison diagram of detection accuracy of a method for detecting an occluded target according to an embodiment of the present application and a baseline method is shown in FIG. 5.

[0065] Figure 6 A diagram of detection effect of a method for detecting an occluded target according to an embodiment of the present application is shown in FIG. 6.

[0066] Figure 7 An occluded target detection device according to an embodiment of the present application is shown in FIG. 7.

[0067] Figure 8 A structure diagram of an electronic device for detecting an occluded target according to an embodiment of the present application is shown in FIG. 8. DETAILED DESCRIPTION

[0068] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, if any, are used for distinguishing between similar objects talking about the embodiments and do not necessarily have to appear in the description or claims of the present application in this special order or in any order at all, it is to be understood that the embodiments, if any, described herein can be carried out in other embodiments than the one described or conceptually illustrated, possibly by means of its complementary concept; the same applies to the nature and arrangement of all aspects described herein since any formation of the described embodiments, if any, can serve the intended purpose in any manner and therefore the scope of the application should be governed by the requests of the claims and not the details of the description or the drawings. In the following, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings accompanying the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application.

[0069] Please refer to Figure 1 A flowchart of a method for detecting an occluded target according to an embodiment of the present application is shown in FIG. 1.

[0070] S110. Obtain the training sample set and input it into the occluded target detection network. Obtain the candidate data in the training sample set through the point cloud multi-scale feature extraction subnetwork in the occluded target detection network. The candidate data includes candidate boxes, sparse point cloud data and sparse point cloud features.

[0071] For example, point cloud data pre-collected from the training sample set is transformed into a fixed size of M×D. p Scene point cloud P s The input is fed into the occluded target detection network, where the point cloud features E of size M×D0 are first extracted by the point cloud multi-scale feature extraction subnetwork. s Based on scene point cloud P s E of foreground and scene point cloud features s The Region Proposal Generation Network (RPN) generates an initial set of candidate boxes corresponding to the foreground points, and the Non-Maximum Suppression (NMS) method is used to remove redundant initial candidate boxes to obtain the remaining Z-squared points. c 1 initial candidate box, based on Z c The parameters of the initial candidate bounding boxes in the scene point cloud P s Find the corresponding Z in c A point cloud target, and in Z c H point cloud targets with sparse point counts and severe occlusion (labeled 3) are selected from an initial candidate bounding box. Because these H targets with sparse point counts and occlusion labels have limited features and lack spatial geometric information, the position parameters of the corresponding H initial candidate bounding boxes are inaccurate. Therefore, the point cloud target data in the H initial candidate bounding boxes represents only a portion of the actual occluded target data. The initial candidate bounding boxes corresponding to these H targets with sparse point counts and occlusion labels constitute a bounding box of size H×D. b Candidate box B, the point cloud data corresponding to candidate box B has a size of H×M1×D p Sparse point cloud data P r Based on candidate bounding box B and sparse point cloud data P r In scene point cloud features E s Extracting sparse point cloud features E from the candidate bounding box r E r The size is H×M1×D0, where M1 is P r E r The number of midpoints, M represents the scene point cloud P. s and scene point cloud features E s The number of midpoints, D p For P r P s The dimension of the midpoint, D0, is E. s E r Feature dimensions, H, Db Z represents the dimension of candidate box B. c M, D p D0, H, D b M1 are both positive integers. For example: a scene point cloud P of size 16384×3 in the training sample set. s The input is fed into the occluded target detection network. First, the multi-scale feature extraction subnetwork of the point cloud in the occluded target detection network extracts the scene point cloud features E with a size of 16384×128. s Based on scene point cloud features E s An initial candidate box set is generated by the Region Proposal Generation Network (RPN), and 64 initial candidate boxes are selected from this set using the Non-Maximum Suppression (NMS) method. Then, based on the parameters of these 64 initial candidate boxes, points are plotted in the scene point cloud P. s Sixty-four point cloud targets were found, and 13 severely occluded point cloud targets with a point count between 100 and 2048 and an occlusion label of 3 were selected. For point cloud targets with fewer than 2048 points, zero-padding was performed to fix the point count to 2048. This resulted in a sparse point cloud dataset P of size 13 × 2048 × 3. r And a candidate bounding box B of size 13×7, based on the scene point cloud features E s The sparse point cloud feature E with a size of 13×2048×128 was cropped from the middle. r .

[0072] S120. Based on the above sparse point cloud data, the encoder and decoder in the above occlusion target detection network are used to perform iterative calculations to obtain the shape features of the occlusion area. Based on the shape features of the occlusion area, the shape point cloud reconstruction operation in the above occlusion target detection network is performed to obtain the overall shape point cloud data of the occlusion target.

[0073] For example, based on the sparse point cloud data P in the candidate box obtained in step S110 r The encoder and decoder in the occluded object detection network are used to perform iterative operations to generate the shape feature E of the occluded region with a size of H×M2×D2. u Based on the shape features E of the occluded region u The shape point cloud reconstruction operation in the occluded target detection network generates a size of H×M3×D. p Point cloud data of the overall shape of the occluded target P c Where M2 is E u The number of midpoints, D2 is E u Feature dimensions, M3 is P c The number of midpoints, D p For P c The dimensions of the midpoint are M2, D2, M3, and D.p are positive integers.

[0074] S130, obtaining aggregated features by using a channel attention fusion operation in the occlusion target detection network according to the occlusion target overall shape point cloud data, the known region point set and the sparse point cloud features.

[0075] For example, for the sparse point cloud data P r of size HxM4x D p obtained by step S110, a known region point set P roi of size HxM4x D roi is randomly sampled, and a channel attention fusion operation in the occlusion target detection network is performed based on the known region point set P r , the sparse point cloud features E c obtained by step S110 and the occlusion target overall shape point cloud data P t obtained by step S120, to obtain aggregated features E roi of size HxM4x D0. t , where M4 is the number of points in P p , D roi is the dimension of the points in P t , D0 is the feature dimension of E p , M4, D t and D0 are positive integers.

[0076] S140, performing feature refinement iterative calculation based on the aggregated features and the known region point set by using a multi-scale transformer model in the occlusion target detection network to obtain occlusion target global features.

[0077] For example, based on the aggregated features E t and the known region point set P roi obtained by step S130, a multi-scale transformer model is used to perform feature refinement iterative calculation to obtain occlusion target global features G of size HxM6x D6, where M6 is the number of points in the occlusion target global features G, D6 is the feature dimension of the occlusion target global features G, and M6 and D6 are positive integers.

[0078] S150, performing confidence calculation operation and position regression operation in the occlusion target detection network based on the occlusion target global features to obtain confidence scores and regression offsets of the occlusion targets in the training sample set, and updating parameters of the occlusion target detection network by using a loss function to constrain the confidence scores and the regression offsets, and learning to generate an occlusion target detection model.

[0079] Exemplarily, based on the occlusion target global feature G obtained by step S140, confidence calculation operation in the occlusion target detection network and position regression operation in the occlusion target detection network are respectively performed to obtain the confidence score Score with a size of HxD7 and the regression offset Res with a size of HxD b of the occlusion target in the training sample set, and the loss is calculated by using the loss function based on the confidence score Score and the regression offset Res to constrain the update of the parameters of the occlusion target detection network composed of the point cloud multi-scale feature extraction sub-network and the RPN in step S110, the encoder, the decoder and the shape point cloud reconstruction operation in step S120, the channel attention fusion operation in step S130, the multi-scale transformer model in step S140, and the confidence calculation operation and the position regression operation in step S150, and the occlusion target detection model is learned, wherein D7 is the dimension of Score, D b is the dimension of Res, and D7, D b are positive integers.

[0080] S160, the above occlusion target detection model is used to detect the to-be-detected sample set to obtain the occlusion target detection frame and the occlusion target category information, and the occlusion target detection is completed.

[0081] Exemplarily, the to-be-detected sample set is obtained, and the occlusion target detection model generated by step S150 is used to detect the scene point cloud in the to-be-detected sample set, that is, the candidate data in the to-be-detected sample set is first obtained by step S110, and then step S120 is executed based on the candidate data obtained by the preliminary detection to reconstruct the shape information of the occlusion target in the to-be-detected sample set, and then step S130 is executed to fuse the shape information and the position information of the occlusion target in the to-be-detected sample set based on the channel attention mechanism to obtain the known region point set and the aggregated feature with rich spatial geometric information, and step S140 is executed based on the known region point set and the aggregated feature of the occlusion target in the to-be-detected sample set to refine the spatial geometric information of the occlusion target in the to-be-detected sample set, and the global feature of the Z occlusion targets is generated, and finally the confidence calculation and the position regression operation in step S150 are executed based on the global feature of the occlusion target to generate the occlusion target category information corresponding to the Z occlusion targets in the to-be-detected sample set and the occlusion target detection frame with a size of ZxD b , and the occlusion target detection is completed, wherein Z, D b is the dimension of the occlusion target detection frame, Z, D bare positive integers. For example: based on the occlusion target detection model obtained by step S150, the scene point cloud in the to-be-detected sample set is detected, that is, by sequentially performing the confidence calculation operation and the position regression operation in steps S110, S120, S130, S140 and S150, the confidence score and the regression offset of the occlusion target in the to-be-detected sample set are obtained, it is judged that the size of the occlusion target category information in the to-be-detected sample set is 11x1 "vehicle", "pedestrian" or "bicycle" based on the confidence score, and the size of the occlusion target detection box in the to-be-detected sample set is 11x7 based on the regression offset and the parameters of the candidate box obtained by executing step S110. Take a certain occlusion target in the to-be-detected sample set as an example, after detection by the occlusion target detection model, an occlusion target detection box with a length of 4.16 meters, a width of 1.56 meters, a height of 1.42 meters, a center point at (4.58 meters, 1.41 meters, 33.13 meters), and an angle of 72.80 degrees is generated. The occlusion target category information is "vehicle", and the occlusion target detection is completed.

[0082] In summary, the application provides an occluded object detection method. By inputting a training sample set into an occluded object detection network, candidate data in scene point cloud is first obtained by using a point cloud multi-scale feature extraction subnetwork, then the overall shape of sparse point cloud data in the candidate box is reconstructed by iterative operation of an encoder and a decoder to enhance the shape features of the occluded object, and the overall shape features of the occluded object obtained by the reconstruction operation and the known region position features of the occluded object position information in the candidate box before reconstruction are aggregated based on a channel attention fusion operation to obtain aggregated features fully fused with the spatial geometric position information of the occluded object, and the aggregated features are iteratively calculated based on an improved multi-scale transformer model to fully refine the aggregated features, enhance the key point information, generate global features of the occluded object, and finally perform confidence calculation and position regression operation based on the global features of the occluded object to obtain a confidence score and a regression offset, and the loss value is calculated by using a loss function based on the real label data in the training sample set to constrain and adjust the parameters of the occluded object detection network, so as to learn an occluded object detection model to detect a to-be-detected sample set and generate an occluded object detection box and occluded object category information, thereby completing occluded object detection. The reconstruction operation in the occluded object detection method effectively extracts the structure information of the sparse point cloud data based on the encoder-decoder architecture, reconstructs the overall shape point cloud data of the occluded object, generates the overall shape features of the occluded object, and enhances the shape information of the occluded object. The channel attention fusion operation in the detection method can fully fuse the shape features and position features with different semantic information from different modes to generate aggregated features with enhanced spatial geometric information of the occluded object. The multi-scale transformer model in the detection method effectively captures the multi-level semantic correlation of the aggregated features containing rich spatial geometric information by attention interaction calculation on different scale features, and refines and enhances the key point features of the occluded object. The occluded object detection method can reconstruct the shape features of the occluded object, refine the key point information of the occluded object, and effectively improve the detection performance of the occluded object by combining the reconstruction operation, the channel attention fusion operation, and the multi-scale transformer model.

[0083] In some embodiments, the iterative operation includes a first iterative operation and a second iterative operation.

[0084] Step S120 can specifically include:

[0085] S1201. Based on the above sparse point cloud data, perform graph embedding operation in the above occlusion target detection network to obtain graph embedding local structural features. Based on the above graph embedding local structural features, use the above encoder to perform the above first iteration operation to obtain encoder output structural features. Perform projection mapping operation in the above occlusion target detection network on the above encoder output structural features to obtain occlusion region center point sequence and occlusion region center point structural features.

[0086] For example, based on the sparse point cloud data P in the candidate box obtained in step S110 r The graph embedding operation in the occluded object detection network is used to obtain the local structural features E of graph embedding with size H×M2×D1. q and E q The first iteration calculates the encoder output structural features E, which are the first layer input structural features of the encoder in the occluded target detection network, and the result is H×M2×D1. enc Based on the encoder output structural features E enc Using the projection mapping operation in the occluded object detection network, a result of size H×M2×D is obtained. p The sequence of center points of the occluded area P u Structural features E of the center point of the occlusion region with size H×M2×D2 cen Where M2 is E q E enc P u E cen The number of midpoints, D1 is E q E enc Feature dimension, D p For P u The dimension of the midpoint, D2, is E. cen Feature dimensions, M2, D1, D p D2 are both positive integers.

[0087] In some implementations, the graph embedding operation includes a first iterative farthest point sampling operation, a graph convolution operation, and a second embedding operation; the projection mapping operation includes a global pooling operation and a first multilayer perceptron; step S1201 may specifically include:

[0088] S12011. Perform the first iteration of farthest point sampling operation and the graph convolution operation on the above sparse point cloud data to obtain the known region center point sequence and the known region topological features.

[0089] S12012. Perform the second embedding operation on the known region center point sequence and the known region topological features to obtain the graph embedding local structural features.

[0090] S12013, based on the above-mentioned graph embedding local structure feature, using the above-mentioned encoder to perform the above-mentioned first iteration operation to obtain the above-mentioned encoder output structure feature, wherein the above-mentioned first iteration operation is based on the feature self-encoding operation and the first feedforward network for encoding;

[0091] S12014, extracting global information using the above-mentioned global pooling operation according to the above-mentioned encoder output structure feature;

[0092] S12015, based on the above-mentioned global information, using the above-mentioned first multilayer perceptron to obtain the above-mentioned occlusion region center point sequence and the above-mentioned occlusion region center point structure feature.

[0093] For example, in the occlusion target detection network, for the sparse point cloud data P r in the candidate box obtained in step S110, a first iteration farthest point sampling operation is performed to obtain a known region center point sequence K p with a size of HxM2xD c , and a graph convolution operation is performed based on K c and P r to obtain a known region topology feature E f with a size of HxM2xD1, and a second embedding operation is performed to embed the known region center point sequence K c into the known region topology feature E f to obtain a graph embedding local structure feature E q , wherein M2 is the number of points in K c , E f , D p is the dimension of the point in K c , D1 is the feature dimension of E f , M2, D p , and D1 are all positive integers.

[0094] The graph embedding local structure feature E q is taken as the first layer input structure feature S (1) of the encoder, i.e., S (1) with a size of HxM2xD1, S (1) is iteratively updated and refined by the L-layer encoder through a first iteration operation, the first iteration operation includes L times of calculation as shown in formula (1), and the input structure feature S (r) of the rth layer of the encoder with a size of HxM2xD1 is calculated through the feature self-encoding operation based on the self-attention mechanism and the first feedforward network for the rth time as shown in formula (1), to obtain the output structure feature S (r+1) of the rth layer of the encoder with a size of HxM2xD1, wherein M2 is the number of points in S (1) , S (r) , S (r+1)The number of midpoints, D1 is S (1) S (r) S (r+1) The feature dimensions of the midpoint, L, r, M2, and D1, are all positive integers.

[0095] S (r+1) =ψ1(SAT(S (r) ),r=1,...,L (1)

[0096] Where ψ1 represents the first feedforward network, and SAT represents the feature autoencoder operation for feature autoencoder computation. δ represents the normalization operation implemented by the softmax function. This represents the matrix transpose operation. For S (r) The projection matrices are all of size H×M2×M2, and the value of d is... Among them, M2 is The dimensions M2, d, and D1 are all positive integers.

[0097] The output structure feature S of the encoder's r-th layer (r+1) The input structural features of the (r+1)th layer of the encoder are continuously iteratively updated until the Lth layer of the encoder.

[0098] The first-layer input structure feature S of the encoder (1) The first iteration is performed based on the L-layer encoder to obtain the encoder output structural features E. enc ;

[0099] Based on encoder output structural features E enc Global pooling is used to extract global information, and the first multilayer perceptron is used to reconstruct the global information into a sequence P of center points of the occluded region. u and structural features of the center point of the occluded area E cen ;

[0100] For example: In an occluded target detection network, the sparse point cloud data P in the candidate bounding box with a size of 13×2048×3 obtained in step S110 is processed. r The first iteration of farthest point sampling operation yields a sequence K of known region center points of size 13×1024×3. c And based on K c and P r Performing graph convolution operations yields the topological features E of the known region, which are 13×1024×768 pixels in size. f And based on K c and E f Performing the second embedding operation yields a graph embedding local structural feature E of size 13×1024×768. q Eq The input is fed into the three-layer encoder as the first-layer input structural feature S of the encoder. (1) The output structural feature S of the first layer of the encoder is obtained by performing the first calculation as shown in equation (1) after passing through the first layer of the encoder. (2) S (1) First, an autoencoded feature of size 13×1024×768 is obtained through the feature autoencoding operation of the first layer of the encoder. Then, this autoencoded feature is updated by the first feedforward network to obtain the output structural feature S of size 13×1024×768 of the first layer of the encoder. (2) S (2) As the input structural feature of the second layer of the encoder, a second calculation as shown in equation (1) is performed to obtain the output structural feature S of the second layer of the encoder with a size of 13×1024×768. (3) S (3) As the input structural feature of the third layer of the encoder, the third calculation as shown in equation (1) is performed to obtain the output structural feature S of the third layer of the encoder with a size of 13×1024×768. (4) S (4) E, as the encoder output structural feature enc Global information is extracted through global pooling, and then the first multilayer perceptron reshapes this global information into a sequence P of center points of the occlusion region with a size of 13×1024×3. u The structural feature E of the center point of the occlusion region with a size of 13×1024×1024 cen .

[0101] S1202. Perform the first embedding operation in the above-mentioned occlusion target detection network on the above-mentioned occlusion region center point sequence and the above-mentioned occlusion region center point structural features to obtain the initial local shape features of the occlusion region.

[0102] For example, the sequence P of the center points of the occlusion area obtained in step S1201 u and structural features of the center point of the occluded area E cen The first embedding operation in the occluded target detection network is used to obtain the initial local shape features E of the occluded region with size H×M2×D1. loc That is, firstly, the corresponding neural network is used to generate the sequence P of the center points of the occluded area. u The feature dimension mapping transformation is used to transform the structural features E of the center point of the occluded region. cen The feature dimension is used to obtain the center point feature of the occluded region. Then, the center point feature of the occluded region is concatenated with the structural feature E of the center point of the occluded region along the channel dimension. cen The initial local shape features E of the occluded region are obtained through a convolutional layer. loc Where M2 is the initial local shape feature E of the occluded region.loc the number of midpoints, D1 is E loc the feature dimension, M2, D1 are all positive integers. For example: the first embedding operation in the occluded target detection network is performed on the occluded region center point sequence P of size 13x1024x3 obtained in step S1201 u to an occluded region center point structure feature E of size 13x1024x1024 cen , obtaining an occluded region initial local shape feature E of size 13x1024x768 loc .

[0103] S1203, according to the above occluded region initial local shape feature and the above encoder output structure feature, using the above decoder to perform the above second iteration operation and dimension transformation operation to obtain the above occluded region shape feature, using the above occluded region shape feature, the above occluded region center point sequence and the above sparse point cloud data to perform the above shape point cloud reconstruction operation, obtaining the above occluded target overall shape point cloud data.

[0104] For example, the occluded region initial local shape feature E loc obtained by step S1202 and the encoder output structure feature E enc obtained by step S1201 are input into the decoder in the occluded target detection network to perform the second iteration operation and the dimension transformation operation, obtaining the occluded region shape feature E u , and performing the shape point cloud reconstruction operation in the occluded target detection network on the occluded region shape feature E u , the occluded region center point sequence P u , the sparse point cloud data P r , obtaining the occluded target overall shape point cloud data P c .

[0105] In some embodiments, the above shape point cloud reconstruction operation includes a first splicing operation, a second multi-layer perceptron and a second splicing operation; step S1203 can specifically include steps S12031-S12035:

[0106] S12031, taking the above occluded region initial local shape feature as the first layer input shape feature of the above decoder, and combining the above encoder output structure feature to perform the above second iteration operation to obtain the decoder output shape feature, wherein the above second iteration operation is based on feature self-decoding operation, feature cross-decoding operation and second feedforward network decoding operation;

[0107] S12032, using a third multi-layer perceptron to perform the above dimension transformation operation on the above decoder output shape feature to obtain the above occluded region shape feature;

[0108] S12033, performing the first splicing operation based on the sequence of center points of the occlusion region and the shape feature of the occlusion region to obtain first splicing features;

[0109] S12034, performing spatial mapping on the first splicing features using the second multi-layer perceptron to obtain occlusion region point cloud data;

[0110] S12035, performing the second splicing operation on the occlusion region point cloud data and the sparse point cloud data to obtain the overall shape point cloud data of the occlusion target.

[0111] For example, the initial local shape feature E loc of the occlusion region obtained in step S1202 is input into the decoder in the occlusion target detection network as the first layer input shape feature U (1) of the decoder, combined with the encoder output structure feature E enc obtained in step S1201, and iteratively updated and refined by the L-layer decoder through a second iterative operation, which includes L times of calculation as shown in formula (2). The size of the input shape feature U (r) of the rth layer of the decoder is HxM2xD1, and after the rth calculation as shown in formula (2), the output shape feature U (r of the rth layer of the decoder is obtained, whose size is HxM2xD1. +1) , wherein M2 is the number of midpoints of U (r) , D1 is the feature dimension of U (r+1) , L, r, M2, and D1 are all positive integers. (r) (r+1)

[0112] U (r+1) = ψ2(CRT(SAT2(U (r) ), E enc )), r = 1,..., L (2)

[0113] , wherein SAT2 represents a feature self-decoding operation for performing feature self-decoding calculation, and U (r) is a projection matrix of U , CRT represents a feature cross-decoding operation, and U (r) is a projection matrix of SAT2(U ), and U enc is a projection matrix of E , ψ2 is a second feedforward network, and M2 is ​​The dimension is M2, where M2 is a positive integer.

[0114] The first layer input shape feature U of the decoder (1) The second iteration operation is performed based on the L-layer decoder to obtain the decoder output shape feature E of size H×M2×D1. dec Where M2 is E dec The number of midpoints, D1 is E dec The feature dimensions, M2 and D1, are both positive integers.

[0115] Decoder output shape feature E dec The occlusion region shape feature E is obtained by using a dimensionality transformation operation implemented by a third multilayer perceptron. u .

[0116] The first splicing operation is used to splice the sequence of occlusion region center points P obtained in step S1201 along the channel dimension. u Shape characteristics E of the occluded area u The first concatenated feature of size H×M2×D3 is obtained, and the first concatenated feature is spatially mapped using a second multilayer perceptron to obtain a feature of size H×M1×D. p Point cloud data of the occluded area P occ Then, a second stitching operation is used to stitch together the point cloud data P of the occluded area along the channel dimension. occ Compared with the sparse point cloud data P obtained in step S110 r The point cloud data P representing the overall shape of the occluded target is obtained. c Where M2 is the number of points in the first stitched feature, D3 is the feature dimension of the first stitched feature, and M1 is the point cloud data P of the occluded region. occ The number of midpoints, D p For P occ The dimensions of the midpoint are M1, M2, D3, and D. p All are positive integers.

[0117] For example: the initial local shape feature E of the occlusion region with a size of 13×1024×768 obtained in step S1202. loc The input shape feature U is fed into the three-layer decoder as the first layer input shape feature. (1) The first calculation as shown in equation (2) is performed in the first layer of the decoder, that is, first based on U (1) A feature self-decoding operation is performed to obtain a self-decoded feature of size 13×1024×768. Then, based on the self-decoded feature and the encoder output structure feature E of size 13×1024×768 obtained in step S1201, encThe feature cross-decoding operation is performed to obtain cross-decoding features with a size of 13x1024x768, and finally the cross-decoding features are updated by the corresponding second feedforward network to obtain output shape features U of the first layer of the decoder with a size of 13x1024x768 (2) ; then, U (2) is calculated as the input shape feature of the second layer of the decoder for the second time according to formula (2) to obtain output shape features U of the second layer of the decoder with a size of 13x1024x768 (3) , U (3) is calculated as the input shape feature of the third layer of the decoder for the third time by the third layer of the decoder to obtain output shape features of the third layer of the decoder, i.e., the decoder output shape features E with a size of 13x1024x768 dec , and then the decoder output shape features E dec are subjected to a dimension transformation operation implemented by a third multilayer perceptron to obtain an occlusion area shape feature E u with a size of 13x1024x1024, and a first splicing operation is used to splice the occlusion area shape feature E u and the occlusion area center point sequence P u obtained in step S1201 to obtain a first spliced feature with a size of 13x1024x1027, which is input into a second multilayer perceptron for spatial mapping to obtain occlusion area point cloud data P occ with a size of 13x2048x3, and a second splicing operation is performed to splice the occlusion area point cloud data P occ and the sparse point cloud data P r with a size of 13x2048x3 obtained in step S110 to obtain occlusion target overall shape point cloud data P c with a size of 13x4096x3.

[0118] In some embodiments, step S130 specifically comprises:

[0119] S1301, using a fourth multilayer perceptron to aggregate the occlusion target overall shape point cloud data in a neighborhood radius range of the points in the known area point set to obtain occlusion target overall shape features;

[0120] S1302, sampling known features from the sparse point cloud features according to the index numbers of the points in the known area point set to obtain known area position features by updating the known features using a convolution network;

[0121] S1303, obtaining initial mixed features by a third splicing operation according to the occlusion target overall shape features and the known area position features;

[0122] S1304. Based on the above initial mixed features, perform point-by-point pooling to obtain channel attention features;

[0123] S1305. Perform a first linear projection operation and a normalization operation on the above-mentioned channel attention features to obtain a channel attention map. Multiply the above-mentioned channel attention map with the above-mentioned known region location features and then perform a second linear projection operation to obtain the above-mentioned aggregated features.

[0124] For example, for the point cloud data P of the overall shape of the occluded target obtained in step S120 c Given the known region point set P roi Using each point as the center, search for the overall shape point cloud data P of the occluded target within a given neighborhood radius. c The neighborhood point set is obtained, and a fourth multilayer perceptron is used to apply it to the known region point set P. roi At each point in the matrix, a set of neighboring points is aggregated to obtain the overall shape feature E of the occluded target with size H×M4×D0. all And from the known region point set P roi The index number of each point in the sparse point cloud feature E obtained in step S110 r The known features are obtained by sampling, and the known features are updated using a convolutional network to obtain the known region location features E of size H×M4×D0. roi Where M4 is E all E roi The number of midpoints, D0 is E all E roi The feature dimensions, M4 and D0, are both positive integers.

[0125] For the overall shape feature E of the occluded target all and known regional location characteristics E roi The third concatenation operation is performed to concatenate E along the feature dimension. all and E roi The initial mixed feature E with size H×M4×D4 is obtained. cat Based on the initial mixed features E cat Perform pointwise pooling along the feature dimension to obtain a channel attention feature E of size H×M6×D4. max and E max Perform the first linear projection operation implemented by the first linear neural network and the normalization operation implemented by the softmax function to obtain a channel attention map E of size H×M6×D0. map and E map With E roi The product obtained by multiplication is subjected to a second linear projection operation implemented by a second linear neural network to obtain the aggregated feature E. tWhere M4 is the initial mixed feature E cat The number of midpoints, D4 is the initial blending feature E cat Channel attention features E max The feature dimension, M6, is the channel attention feature E. max Channel attention map E map The number of midpoints, D0 is the channel attention map E map The feature dimensions, M4, D4, M6, and D0, are all positive integers.

[0126] For example: for the sparse point cloud data P in the candidate box obtained in step S110 r Randomly sampling 512 points yields a known region point set P of size 13 × 512 × 3. roi Given the known region point set P roi Centered on each point, search within a 0.2-meter neighborhood radius for the overall shape point cloud data P of the occluded target obtained in step S120. c The neighborhood point set is obtained, and a fourth multilayer perceptron is used to apply it to the known region point set P. roi By aggregating neighborhood point sets at each point, the overall shape feature E of the occluded target, which is 13×512×128, is obtained. all And from the known region point set P roi The index number of each point in the sparse point cloud feature E obtained in step S110 r The known features are obtained by sampling, and the known features are updated using a convolutional network to obtain the known region location features E of size 13×512×128. roi ;

[0127] When the overall shape features of the occluded target are E all and known regional location characteristics E roi Perform a third concatenation operation on the feature dimensions to obtain an initial mixed feature E of size 13×512×256. cat In E cat Perform pointwise pooling on the feature dimension to obtain a channel attention feature E of size 13×1×256. max and E max Performing the first linear projection operation implemented by the first linear neural network and the normalization operation implemented by the softmax function yields a channel attention map E of size 13×1×128. map E map With E roi After multiplication, a second linear projection operation implemented by a second linear neural network is performed to obtain an aggregated feature E of size 13×512×128. t .

[0128] In some embodiments, the feature refinement iterative calculation includes a second iterative farthest point sampling operation, an aggregation operation, a self-attention operation, a first cross-attention operation, a third linear projection operation, a second cross-attention operation, and a third feedforward network; and the step S140 can specifically include S1401-S1404:

[0129] S1401, taking the aggregated feature and the known region point set as the first layer input feature and the first layer input point set of the multi-scale transformer model, respectively, performing the second iterative farthest point sampling operation based on the first layer input point set of the multi-scale transformer model to obtain a first layer center point set, wherein the first layer center point set is a first layer output point set;

[0130] S1402, based on the first layer center point set, using the aggregation operation to aggregate the first layer input feature of the multi-scale transformer model to obtain a first layer first scale feature and a first layer second scale feature, performing the self-attention operation on the first layer first scale feature to obtain a first layer first scale self-attention feature, and performing the first cross-attention operation based on the first layer first scale self-attention feature and the first layer second scale feature to obtain a first layer first resolution feature;

[0131] S1403, for the first layer input feature, performing the third linear projection operation to obtain a first layer second resolution feature, performing the second cross-attention operation based on the first layer first resolution feature and the first layer second resolution feature, and updating through the third feedforward network to obtain a first layer output feature, completing the first refinement calculation of the feature refinement iterative calculation;

[0132] S1404, based on the first layer output feature and the first layer output point set, continuing to perform the target number of refinement calculations of the feature refinement iterative calculation through the remaining layers of the multi-scale transformer model, obtaining a last layer output point set and a last layer output feature of the multi-scale transformer model to generate the global feature of the occluded target, wherein the remaining layers of the multi-scale transformer model do not include the first layer of the multi-scale transformer model.

[0133] For example, the aggregated feature E t and the known region point set P roi are taken as the first layer input feature T (1) and the first layer input point set X (1) of the multi-scale transformer model in the occluded target detection network, respectively., input into the J-layer multi-scale transformer model for feature refinement iterative calculation, wherein the feature refinement iterative calculation includes J times of refinement calculation, wherein J is a positive integer, and taking the n-th refinement calculation in the feature refinement iterative calculation as an example:

[0134] For the input feature T (n) with a size of and the input point set X (n) with a size of , first, a second iterative farthest point sampling operation is used to sample points in X (n) to form a center point set X (n+1) of the n-th layer of the multi-scale transformer model with a size of , that is, X (n+1) is the output point set of the n-th layer of the multi-scale transformer model, wherein is the number of points in the input feature T (n) and the input point set X (n) , is the number of points in X (n+1) , is the feature dimension of the input feature T (n) , D p is the dimension of the points in X (n) , X (n+1) , n, D p are positive integers;

[0135] For the center point set X (n+1) of the n-th layer of the multi-scale transformer model, find a1 points in the input point set X (n) within a first radius range of the center point set X (n+1) to form a first neighborhood radius point set, and use a first aggregation operation to aggregate the input feature T (n) corresponding to the index of the points in the first neighborhood radius point set to obtain a first scale feature T1 (n+1) of the n-th layer of the multi-scale transformer model with a size of , and simultaneously, find a2 points in the input point set X (n) within a second radius range of the center point set X (n+1) to form a second neighborhood radius point set, and use a second aggregation operation to aggregate the input feature T (n) corresponding to the second neighborhood radius point set to obtain a second scale feature T2 (n+1) of the n-th layer of the multi-scale transformer model with a size of , wherein T1 (n+1) T2 (n+1) the number of midpoints, T1 (n+1) T2 (n+1) dimensions, a1, a2, are positive integers;

[0136] performing a self-attention operation on the first scale feature T1 (n+1) based on the nth layer of the multi-scale transformer model to obtain a first scale self-attention feature T with a size of s (n+1) :

[0137]

[0138] wherein, V1 (n) is a projection matrix of T1 (n+1) with a size of d1 (n) a value of is V1 (n) dimension, d1 (n) , are positive integers.

[0139] performing a first cross-attention operation on the first scale self-attention feature T s (n+1) and the second scale feature T2 (n+1) based on the nth layer of the multi-scale transformer model to obtain a first resolution feature T with a size of q (n+1) wherein, are positive integers:

[0140]

[0141] wherein, is a projection matrix of T s (n+1) is a projection matrix of T2 (n+1) , with a size of is a dimension of , is a positive integer;

[0142] ​Input features T of the nth layer based on the multi-scale transformer model (n) Perform a third linear projection operation implemented by a linear neural network to obtain a value of... Second resolution feature T k (n+1) For the second resolution feature T of the nth layer of the multi-scale transformer model k (n+1) and the first resolution feature T q (n+1) The second cross-attention operation is performed to obtain cross features, which are then updated using the third feedforward network to obtain the output features T of the nth layer of the multi-scale transformer model. (n+1) T (n+1) The size is in, For T k (n+1) The number of midpoints For T (n+1) The number of midpoints For T (n+1) T k (n+1) Feature dimensions, All are positive integers. The calculation process of the second cross-attention operation and the third feedforward network is shown in (5):

[0143] T (n-1) =ψ3(CrosT(T) q (n+1) T k (n+1) ), n=1,…,J (5)

[0144] in, CrosT represents the second cross-attention operation, with a size of . of yes The projection matrix, V3 (n) It is T k (n+1 The projection matrix of ) V3 (n) The size is ψ3 is the third feedforward network, where, for Dimensions for V3 (n) Dimensions All are positive integers.

[0145] The input point set X of the nth layer of the multi-scale transformer model (n) and input features T (n) After the nth refinement calculation in the above feature refinement iteration, the output point set X is obtained. (n+1) and output features T (n+1) X (n+1) and T (n+1) It is fed into the (n+1)th layer of the multi-scale transformer model to continue the iterative cycle.

[0146] The first layer input point set X of the multi-scale transformer model (1) and the first layer input features T (1) After feature refinement and iterative calculation using a J-layer multi-scale transformer model, the size of the multi-scale transformer model is obtained as follows: The last layer output point set X (J+1) and size are The last layer output feature T (J +1) , will X (J+1) Integrating into T (J+1) The global features G of the occluded target are obtained from the data, where, X represents (J+1) T (J+1) The number of midpoints T represents (J+1) Feature dimensions, D p X represents (J+1) The dimension of the midpoint D p All are positive integers.

[0147] For example: the aggregated feature E with a size of 13×512×128 obtained in step S130. t And a known region point set P of size 13×512×3 roi The inputs are fed into a three-layer multi-scale transformer model in the occluded object detection network, and are respectively used as the first-layer input features T of the multi-scale transformer model. (1) and the first layer input point set X (1) Feature refinement iterative calculations are performed. In the first layer of the multi-scale transformer model, the farthest point sampling operation of the second iteration is first used on the input point set X. (1) The mid-sampling yields a center point set X of size 13×128×3. (2)and 16 points in the second radius range of 0.4 meters are found to obtain a first neighborhood radius point set and a second neighborhood radius point set, and input features corresponding to the first neighborhood radius point set and the second neighborhood radius point set are aggregated to obtain a first scale feature T1 with a size of 13x128x256 (2) and a second scale feature T2 with a size of 13x128x256 (2) based on the first scale feature T1 (2) to obtain a first scale self-attention feature T s (2) and the first scale self-attention feature T s (2) and the second scale feature T2 (2) to obtain a first resolution feature T q (2) and the first resolution feature T (1) , a third linear projection operation realized by a linear neural network is performed to obtain a second resolution feature T k (2) based on the first resolution feature T q (2) and the second resolution feature T k (2) a second cross-attention operation is performed, wherein a projection matrix with a size of 13x128x128 is multiplied by the first resolution feature T q (2) to obtain a query vector with a size of 13x128x256, a projection matrix with a size of 13x512x512 V3 (1) is multiplied by the second resolution feature T k (2) to obtain a key vector and a value vector both with a size of 13x512x256, and the above-mentioned query vector, key vector and value vector are multiplied to obtain a cross feature with a size of 13x128x256, and the cross feature is updated through a third feedforward network to obtain a first layer output feature T (2) of the multi-scale transformer model, a center point set X (2) of the first layer of the multi-scale transformer model is taken as the first layer output point set X (2) , and the first layer output point set X (2) and the first layer output feature T(2) The data is fed into the second layer of the multi-scale transformer model for the second refinement calculation in the feature refinement iteration, resulting in an output point set X of size 13×32×3 for the second layer of the multi-scale transformer model. (3) The output feature T is of size 13×32×256. (3) , will X (3) and T (3) The input is fed into the third layer of the multi-scale transformer model for the third refinement calculation in the feature refinement iteration, resulting in the output point set X of the third layer of the multi-scale transformer model with a size of 13×1×3. (4) The output feature T is of size 13×1×512. (4) , will X (4) Integrating into T (4) The global feature G of the occluded target with a size of 13×1×512 is obtained.

[0148] In some embodiments, step S150 includes steps S1501 to S1503:

[0149] S1501. Based on the global features of the occluded target, the first fully connected neural network is used to perform the confidence calculation operation to generate the confidence score of the occluded target in the training sample set to determine the category value of the occluded target.

[0150] S1502. Based on the global features of the occluded target, the second fully connected neural network is used to perform the above-mentioned position regression operation to generate the regression offset of the candidate box corresponding to the occluded target. The difference between the length, width, height, angle parameters and center point coordinates of the candidate box and the length, width, height, angle parameters and center point coordinates of the real target box in the training sample set is used as the regression target.

[0151] S1503. Calculate the cross-entropy classification loss based on the category value of the occluded target and the real target category label in the training sample set. Calculate the position regression loss based on the regression offset of the candidate box corresponding to the occluded target and the regression target. Update the parameters of the occluded target detection network based on the sum of the cross-entropy classification loss and the position regression loss to generate the occluded target detection model.

[0152] For example, the confidence score Score with a size of HxD7 is obtained by using the first fully connected neural network based on the global feature G of the occluded target obtained in step S140, the class value of the occluded target in the training sample set is generated according to the confidence score Score, the cross-entropy loss is calculated according to the class value and the real target class label corresponding to the occluded target in the training sample set, the cross-entropy classification loss of the training sample set is obtained, the regression offset Res of the candidate box with a size of HxD b is generated by using the second fully connected neural network based on the global feature G of the occluded target, and the parameter residual Target with a size of HxD b between the candidate box and the real target box corresponding to the occluded target in the training sample set is taken as the regression target, the position regression loss is calculated by using the smooth L1 loss function according to the regression offset Res of the occluded target and the regression target Target, the occluded target detection network is reversely constrained based on the sum of the cross-entropy classification loss and the position regression loss, the parameters of the occluded target detection network are adjusted, the occluded target detection model is generated, and the training process of the occluded target detection network is completed, wherein H, D7, D b are the dimensions of Res, Score and Target, and H, D7 and D b are all positive integers.

[0153] In some embodiments, as shown in Figure 4 , the structure of the occluded target detection model in the occluded target detection method proposed in the present application is divided into five parts: a candidate data generation module, an occluded target completion reconstruction operation based on an encoder-decoder structure, a channel attention fusion operation, a feature refinement module and an occluded target detection result generation module, wherein the structure diagram of the candidate data generation module is as shown in Figure 2 , and the structure diagram of the occluded target completion reconstruction operation based on the encoder-decoder structure is as shown in Figure 3 . In order to obtain the detection result by using the occluded target detection method proposed in the present application, the occluded target detection network constructed in the method is trained by using the 3D standard data set KITTI to obtain the occluded target detection model, and the occluded target detection model is tested and evaluated by using the 3D standard data set KITTI to verify the effectiveness and practicability of the occluded target detection method proposed in the present application. The KITTI data set provides 7481 training samples and 7518 test samples, in the implementation process of the occluded target detection method proposed in the present application, 3712 samples are divided from the 7481 training samples to constitute a training sample set, and 3769 samples constitute a to-be-detected sample set, according to the occlusion degree and the proportion of the occluded target in the scene samples, the to-be-detected sample set is divided into different occlusion levels, wherein the sample set with the highest occlusion level constitutes a serious occlusion level sample set, there are a large number of occluded targets in the scene, and the remaining samples constitute a general occlusion level sample set.

[0154] Figure 5 The paper presents a comparison of the detection accuracy of the method of this invention with that of four benchmark methods on samples with two levels of occlusion in the sample set to be tested. Figure 5 As can be seen, the method of this invention achieves excellent detection performance and significantly higher detection accuracy than the other four benchmark methods on severely occluded samples in the sample set to be detected, proving the effectiveness of this method in detecting occluded targets. At the same time, for ordinary occluded sample scenes in the sample set to be detected, this method also achieves significant detection performance and significantly higher detection accuracy than other benchmark methods, indicating that this method can effectively improve the overall scene target detection performance, proving the practicality of this method.

[0155] Figure 6 This diagram illustrates the detection performance of the occluded target detection method proposed in this application on the sample set to be detected in the KITTI dataset. Figure 6 The scene point cloud in the first image is executed as follows: Figure 4 The occlusion target detection model constructed by the method of the present invention shown effectively detected the bounding boxes and category information "vehicle" for 11 occlusion targets in the scene. For Figure 6 The scene point clouds in the other three sets of images, execute as follows Figure 4 The occlusion target detection models constructed by the method of the present invention can accurately detect the target detection boxes and category information corresponding to each occluded target in the scene. This shows that the method of the present invention can effectively detect occluded targets in different scenarios with severe occlusion, proving that the occlusion target detection method proposed in this application, which is based on reconstructing the shape features of occluded targets and effectively enhancing and refining the spatial geometric information of occluded targets, can effectively improve the detection performance of occluded targets. Of course, the KITTI dataset used here is only an example. In practice, other public datasets or point cloud data prepared by the user can also be used for training and evaluation.

[0156] In summary, the occlusion target detection method provided by this invention has high theoretical value. For complex scene samples of different types with a large number of occluded targets, this method can effectively detect and locate occluded targets in complex scenes, achieving excellent detection performance and significantly higher detection accuracy than other methods. Moreover, this method has been implemented in software and has great engineering application value.

[0157] Please see Figure 7 The present invention also proposes an occlusion target detection device, comprising:

[0158] The extraction unit 21 is configured to obtain a training sample set, input the training sample set into the occlusion object detection network, and obtain candidate data in the training sample set by using the point cloud multi-scale feature extraction subnetwork in the occlusion object detection network, wherein the candidate data includes a candidate box, sparse point cloud data, and sparse point cloud features.

[0159] The encoding and decoding unit 22 is configured to perform iterative operation on the sparse point cloud data by using the encoder and the decoder in the occlusion object detection network to obtain occlusion region shape features, and perform shape point cloud reconstruction operation in the occlusion object detection network according to the occlusion region shape features to obtain occlusion object overall shape point cloud data.

[0160] The fusion unit 23 is configured to obtain aggregated features by using channel attention fusion operation in the occlusion object detection network according to the occlusion object overall shape point cloud data, a known region point set, and the sparse point cloud features, wherein the known region point set is obtained by random sampling based on the sparse point cloud data.

[0161] The obtaining unit 24 is configured to perform feature refinement iterative calculation by using a multi-scale transformer model in the occlusion object detection network based on the aggregated features and the known region point set to obtain occlusion object global features.

[0162] The generating unit 25 is configured to perform confidence calculation operation and position regression operation in the occlusion object detection network based on the occlusion object global features to obtain a confidence score and a regression offset of the occlusion object in the training sample set, and update parameters of the occlusion object detection network by using a loss function based on the confidence score and the regression offset to learn and generate an occlusion object detection model.

[0163] The detection unit 26 is configured to detect a to-be-detected sample set by using the occlusion object detection model to obtain an occlusion object detection box and occlusion object category information, and complete occlusion object detection.

[0164] As shown in Figure 8 The embodiment of the present application also provides an electronic device 300, which includes a memory 310, a processor 320, and a computer program 311 stored in the memory 320 and executable on the processor, and the processor 320 implements the steps of any method for occlusion object detection when executing the computer program 311.

[0165] Since the electronic device introduced in the embodiment is the device used for implementing the device for detecting the occluded target in the embodiment of the present application, based on the method introduced in the embodiment of the present application, the person skilled in the art can understand the specific implementation of the electronic device in the embodiment and various changes thereof, so how the electronic device implements the method in the embodiment of the present application is not described in detail here, as long as the device used for implementing the method in the embodiment of the present application by the person skilled in the art belongs to the scope of the present application.

[0166] In the implementation process, the computer program 311 can realize the functions of the device for detecting the occluded target in the embodiment of the present application when the computer program 311 is executed by the processor. Figure 1 Any of the embodiments in the corresponding embodiment.

[0167] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in an embodiment can be referred to the related description of other embodiments.

[0168] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0169] The present application is described with reference to flowcharts and / or block diagrams according to the method, device (system), and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing devices to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices produce a device for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks. Figure 1 The device for implementing the functions specified in one block or multiple blocks.

[0170] These computer program instructions can also be stored in a computer readable storage medium that can guide the computer or other programmable data processing devices to work in a specific way, so that the instructions stored in the computer readable storage medium produce a manufactured product including instruction devices, which implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one flow or multiple flows and / or blocks. Figure 1 The device for implementing the functions specified in one block or multiple blocks.

[0171] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable devices to generate a computer implemented process, so that the instructions executed on the computer or other programmable devices provide a process for implementing the functions specified in the flowchart Figure 1 one flowchart or multiple flowcharts and / or one block or multiple blocks. Figure 1 the functions specified in the flowchart

[0172] The embodiments of the present application also provide a computer program product, which comprises computer software instructions, when the computer software instructions are run on a processing device, so that the processing device executes the method as ​ the flowchart of the occlusion target detection method in the corresponding embodiments.

[0173] The computer program product comprises one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flowchart or function according to the embodiments of the present application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that the computer can store or be integrated into a data storage device such as a server, data center, etc. containing one or more available media sets. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as DVD), or semiconductor media (such as solid state disk (SSD)) and the like.

[0174] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above described system, device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0175] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. For example, the division of the units is only a logical function division. There can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0176] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0177] In addition, each functional unit in the embodiments of the present application can be integrated in a processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of software functional units.

[0178] If the integrated unit is implemented in the form of software functional units and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such an understanding, the technical solutions of the present application essentially, or the part that contributes to the prior art, or all or a part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, and various media that can store program codes.

[0179] The above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting occluded targets, characterized in that, include: A training sample set is obtained and input into the occluded target detection network. Candidate data in the training sample set is obtained through the point cloud multi-scale feature extraction subnetwork in the occluded target detection network. The candidate data includes candidate boxes, sparse point cloud data, and sparse point cloud features. Based on the sparse point cloud data, the encoder and decoder in the occlusion target detection network are used to perform iterative calculations to obtain the shape features of the occlusion region. Based on the shape features of the occlusion region, the shape point cloud reconstruction operation in the occlusion target detection network is performed to obtain the overall shape point cloud data of the occlusion target. Aggregated features are obtained by using the channel attention fusion operation in the occlusion target detection network based on the point cloud data of the overall shape of the occluded target, the known region point set, and the sparse point cloud features. The known region point set is obtained by random sampling based on the sparse point cloud data. Based on the aggregated features and the known region point set, the global features of the occluded target are obtained by performing feature refinement iterative calculation using the multi-scale transformer model in the occluded target detection network. Based on the global features of the occluded target, the confidence calculation and position regression operations in the occluded target detection network are performed to obtain the confidence score and regression offset of the occluded target in the training sample set. Based on the confidence score and the regression offset, the parameters of the occluded target detection network are updated using the loss function constraint to learn and generate the occluded target detection model. The occlusion target detection model is used to detect the sample set to be detected, and the occlusion target detection box and occlusion target category information are obtained, thus completing the occlusion target detection.

2. The method as described in claim 1, characterized in that, The iterative operation includes a first iterative operation and a second iterative operation; The step of iteratively calculating the occlusion region shape features using the encoder and decoder in the occlusion target detection network based on the sparse point cloud data, and then performing shape point cloud reconstruction operations in the occlusion target detection network based on the occlusion region shape features to obtain the overall shape point cloud data of the occluded target, includes: Based on the sparse point cloud data, the graph embedding operation in the occlusion target detection network is performed to obtain local structural features of the graph embedding. Based on the local structural features of the graph embedding, the encoder is used to perform the first iterative operation to obtain the encoder output structural features. The encoder output structural features are then subjected to the projection mapping operation in the occlusion target detection network to obtain the occlusion region center point sequence and the occlusion region center point structural features. The first embedding operation in the occlusion target detection network is performed on the sequence of center points of the occlusion region and the structural features of the center points of the occlusion region to obtain the initial local shape features of the occlusion region; Based on the initial local shape features of the occluded region and the output structural features of the encoder, the decoder is used to perform the second iterative operation and dimension transformation operation to obtain the shape features of the occluded region. The shape features of the occluded region, the sequence of center points of the occluded region, and the sparse point cloud data are used to perform the shape point cloud reconstruction operation to obtain the overall shape point cloud data of the occluded target.

3. The method as described in claim 2, characterized in that, The graph embedding operation includes a first iterative farthest point sampling operation, a graph convolution operation, and a second embedding operation; the projection mapping operation includes a global pooling operation and a first multilayer perceptron. The step of performing graph embedding operations in the occlusion target detection network based on the sparse point cloud data to obtain local structural features of the graph embedding, performing the first iterative operation using the encoder to obtain encoder output structural features based on the local structural features of the graph embedding, and performing projection mapping operations in the occlusion target detection network on the encoder output structural features to obtain the occlusion region center point sequence and occlusion region center point structural features includes: Perform the first iteration of farthest point sampling operation and the graph convolution operation on the sparse point cloud data to obtain the known region center point sequence and the known region topological features; The second embedding operation is performed on the known region center point sequence and the known region topological features to obtain the graph embedding local structural features; Based on the graph embedding local structural features, the encoder is used to perform the first iterative operation to obtain the encoder output structural features, wherein the first iterative operation is based on feature autoencoding operation and encoding by a first feedforward network; Global information is extracted using the global pooling operation based on the encoder output structure features; Based on the global information, the first multilayer perceptron is used to obtain the sequence of center points of the occluded region and the structural features of the center points of the occluded region.

4. The method as described in claim 2, characterized in that, The shape point cloud reconstruction operation includes a first stitching operation, a second multilayer perceptron, and a second stitching operation. The step involves using the decoder to perform the second iterative operation and dimensional transformation operation based on the initial local shape features of the occluded region and the output structural features of the encoder to obtain the shape features of the occluded region. Then, using the shape features of the occluded region, the sequence of center points of the occluded region, and the sparse point cloud data, a shape point cloud reconstruction operation is performed to obtain the overall shape point cloud data of the occluded target. This includes: The initial local shape features of the occluded region are used as the first-layer input shape features of the decoder, and the second iterative operation is performed in combination with the encoder output structure features to obtain the decoder output shape features. The second iterative operation is a decoding operation based on feature self-decoding operation, feature cross-decoding operation, and second feedforward network. The dimensional transformation operation is performed on the shape features output by the decoder using a third multilayer perceptron to obtain the shape features of the occluded region. The first stitching feature is obtained by performing the first stitching operation based on the sequence of center points of the occluded region and the shape features of the occluded region; The first stitched feature is spatially mapped using the second multilayer perceptron to obtain point cloud data of the occluded area. The point cloud data of the occluded area and the sparse point cloud data are subjected to the second stitching operation to obtain the point cloud data of the overall shape of the occluded target.

5. The method as described in claim 1, characterized in that, The step of obtaining aggregated features using the channel attention fusion operation in the occlusion target detection network based on the overall shape point cloud data of the occluded target, the known region point set, and the sparse point cloud features includes: Within the neighborhood radius of the known region's point set points, a fourth multilayer perceptron is used to aggregate the point cloud data of the overall shape of the occluded target to obtain the overall shape features of the occluded target. Based on the index number of the points in the known region, known features are sampled from the sparse point cloud features, and the known features are updated using a convolutional network to obtain the location features of the known region. Based on the overall shape features of the occluded target and the location features of the known region, a third stitching operation is used to obtain the initial blended features; Channel attention features are obtained by performing point-by-point pooling based on the initial mixed features; The channel attention features are subjected to a first linear projection operation and a normalization operation to obtain a channel attention map. The channel attention map is then multiplied with the known region location features, and a second linear projection operation is performed to obtain the aggregated features.

6. The method as described in claim 1, characterized in that, The feature refinement iterative calculation includes a second iteration farthest point sampling operation, an aggregation operation, a self-attention operation, a first cross-attention operation, a third linear projection operation, a second cross-attention operation, and a third feedforward network; The step of obtaining global features of the occluded target by using the multi-scale transformer model in the occluded target detection network to perform feature refinement and iterative calculation based on the aggregated features and the known region point set includes: The aggregated features and the known region point set are used as the first-layer input features and the first-layer input point set of the multi-scale transformer model, respectively. The second iteration farthest point sampling operation is performed based on the first-layer input point set of the multi-scale transformer model to obtain the first-layer center point set, wherein the first-layer center point set is the first-layer output point set. Based on the set of first-layer center points, the aggregation operation is used to aggregate the first-layer input features of the multi-scale transformer model to obtain the first-layer first-scale features and the first-layer second-scale features. The self-attention operation is performed on the first-layer first-scale features to obtain the first-layer first-scale self-attention features. Based on the first-layer first-scale self-attention features and the first-layer second-scale features, the first cross-attention operation is performed to obtain the first-layer first-resolution features. For the first layer input features, the third linear projection operation is performed to obtain the first layer second resolution features. Based on the first layer first resolution features and the first layer second resolution features, the second cross attention operation is performed and the first layer output features are updated through the third feedforward network to complete the first refinement calculation of the feature refinement iteration calculation. Based on the first-layer output features and the first-layer output point set, the target number of feature refinement iterations is further calculated through the remaining layers of the multi-scale transformer model to obtain the last-layer output point set and last-layer output features of the multi-scale transformer model to generate the global features of the occluded target. The remaining layers of the multi-scale transformer model do not include the first layer of the multi-scale transformer model.

7. The method as described in claim 1, characterized in that, The process of calculating confidence and performing position regression operations in the occluded target detection network based on the global features of the occluded target, obtaining the confidence score and regression offset of the occluded targets in the training sample set, updating the parameters of the occluded target detection network based on the confidence score and the regression offset using a loss function constraint, and learning to generate an occluded target detection model includes: Based on the global features of the occluded target, a first fully connected neural network is used to perform the confidence calculation operation to generate confidence scores of the occluded targets in the training sample set to determine the category value of the occluded target; Based on the global features of the occluded target, a second fully connected neural network is used to perform the position regression operation to generate the regression offset of the candidate box corresponding to the occluded target. The difference between the length, width, height, angle parameters and center point coordinates of the candidate box and the length, width, height, angle parameters and center point coordinates of the real target box in the training sample set is used as the regression target. The cross-entropy classification loss is calculated based on the category value of the occluded target and the true target category label in the training sample set. The position regression loss is calculated based on the regression offset of the candidate box corresponding to the occluded target and the regression target. The parameters of the occluded target detection network are updated based on the sum of the cross-entropy classification loss and the position regression loss to generate the occluded target detection model.

8. A target occlusion detection device, characterized in that, include: An extraction unit is used to acquire a training sample set and input it into an occluded target detection network. The point cloud multi-scale feature extraction subnetwork in the occluded target detection network is used to acquire candidate data in the training sample set, wherein the candidate data includes candidate boxes, sparse point cloud data and sparse point cloud features. The encoding and decoding unit is used to perform iterative calculations based on the sparse point cloud data using the encoder and decoder in the occlusion target detection network to obtain the shape features of the occlusion region, and to perform shape point cloud reconstruction operations in the occlusion target detection network based on the shape features of the occlusion region to obtain the overall shape point cloud data of the occlusion target. The fusion unit is used to obtain aggregated features by using the channel attention fusion operation in the occlusion target detection network based on the point cloud data of the overall shape of the occluded target, the known region point set and the sparse point cloud features, wherein the known region point set is obtained by random sampling based on the sparse point cloud data; The acquisition unit is used to obtain global features of the occluded target by performing feature refinement and iterative calculation based on the aggregated features and the known region point set using the multi-scale transformer model in the occluded target detection network; The generation unit is used to perform confidence calculation and position regression operations in the occlusion target detection network based on the global features of the occlusion target, obtain the confidence score and regression offset of the occlusion target in the training sample set, update the parameters of the occlusion target detection network based on the confidence score and the regression offset using the loss function constraint, and learn to generate an occlusion target detection model. The detection unit is used to detect the sample set to be detected using the occlusion target detection model, obtain the occlusion target detection box and occlusion target category information, and complete the occlusion target detection.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program stored in the memory, implements the steps of the occlusion target detection method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the occlusion target detection method as described in any one of claims 1-7.

Images