Model training, target detection method, device, electronic device and storage medium

Abstract

The present application relates to the field of data processing technology, and provides model training, target detection methods, devices, electronic devices and storage media. The model training method includes: obtaining a sample point cloud data set and an auxiliary model, the auxiliary model is used to determine the key point cloud features of the sample target in the sample point cloud data set and to supplement the key point cloud features with local information; the sample point cloud data set and the auxiliary model are used to train the initial target detection model to determine the preset target detection model. The model training method provided in the present application does not require improvements to the target detection model itself, nor does it increase the complexity and computational complexity of the subsequent target detection stage. Therefore, it can greatly improve the precision and accuracy of the training model, thereby ensuring the accuracy of target detection without adding additional time and computational costs, and greatly improving the applicability and efficiency of target detection.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to model training, target detection methods, devices, electronic devices and storage media. Background Art

[0002] In recent years, 3D object detection has been widely used in fields such as autonomous driving and smart homes. In particular, in smart homes, object detection, which focuses on the category, bounding box, and orientation of objects such as office and household items in indoor scenes, enables smart devices like robot vacuums to accurately plan their paths, avoiding collisions and obstacle avoidance failures. Therefore, improving the accuracy of 3D object detection has become a key issue that needs to be addressed. Summary of the Invention

[0003] This application aims to solve at least one of the technical problems existing in the related art. To this end, this application proposes a model training method that improves the accuracy of 3D object detection without adding additional time and computational costs, significantly improving the applicability and efficiency of 3D object detection.

[0004] This application also proposes a target detection method.

[0005] This application also proposes a model training device.

[0006] This application also proposes a target detection device.

[0007] The present application also provides an electronic device.

[0008] The present application also proposes a non-transitory computer-readable storage medium.

[0009] The present application also proposes a computer program product.

[0010] The model training method according to the first embodiment of the present application includes:

[0011] Acquire a sample point cloud dataset and an auxiliary model, wherein the auxiliary model is used to determine key point cloud features of sample targets in the sample point cloud dataset and to supplement local information of the key point cloud features;

[0012] The sample point cloud dataset and the auxiliary model are used to train an initial target detection model to determine a preset target detection model.

[0013] According to the model training method provided in the present application, the initial target detection model is trained by combining the auxiliary model with the sample point cloud dataset. There is no need to improve the target detection model itself, and therefore the complexity and computational complexity of the subsequent target detection stage will not be increased. Moreover, since the auxiliary model is used to determine the key point cloud features of the sample target in the sample point cloud dataset and to supplement the key point cloud features with local information, the precision and accuracy of the training model can be greatly improved when the auxiliary model and the sample point cloud dataset are used to train the model, thereby ensuring the accuracy of target detection without adding additional time and computational costs, and greatly improving the applicability and efficiency of target detection.

[0014] According to one embodiment of the present application, the using the sample point cloud dataset and the auxiliary model to train the initial target detection model to determine the preset target detection model includes:

[0015] Using a preset batch of sample point cloud data in the sample point cloud dataset and the auxiliary model, the initial target detection model is trained to obtain the model loss of the intermediate target detection model after a preset number of trainings;

[0016] Based on a successful matching result between the model loss and the preset model loss, the intermediate target detection model corresponding to the model loss is determined to be the preset target detection model.

[0017] According to one embodiment of the present application, after obtaining the model loss of the intermediate object detection model after a preset number of trainings, the method further includes:

[0018] Based on the failure of matching the model loss with the preset model loss, the intermediate target detection model corresponding to the model loss is trained using the next preset batch of sample point cloud data in the sample point cloud data set and the auxiliary model to determine the preset target detection model.

[0019] According to one embodiment of the present application, the training of the initial target detection model using a preset batch of sample point cloud data in the sample point cloud dataset and the auxiliary model to obtain the model loss of the intermediate target detection model after a preset number of trainings includes:

[0020] For each training, the corresponding batch of sample point cloud data in the sample point cloud data set is input into the auxiliary model to determine the global point cloud features of the samples obtained by the auxiliary model this time;

[0021] Inputting the corresponding batch of sample point cloud data into the previously trained intermediate target detection model to determine the sample original point cloud features extracted by the currently trained intermediate target detection model;

[0022] Based on the corresponding batch of sample point cloud data, the sample global point cloud features, the sample original point cloud features, the auxiliary model and the intermediate target detection model, the model loss of the intermediate target detection model after this training is determined.

[0023] According to one embodiment of the present application, the auxiliary model includes an information extraction module and an information supplementation module. Inputting the corresponding batch of sample point cloud data in the sample point cloud dataset into the auxiliary model and determining the global point cloud features of the samples acquired by the auxiliary model this time include:

[0024] Using the information extraction module, extracting key information from the corresponding batch of sample point cloud data in the sample point cloud data set, and determining key point cloud features of the sample targets in the corresponding batch of sample point cloud data;

[0025] The information supplement module and the corresponding batch of sample point cloud data are used to perform local information supplement on the key point cloud features to determine the global point cloud features of the samples acquired this time by the auxiliary model.

[0026] According to one embodiment of the present application, using the information extraction module to extract key information from a corresponding batch of sample point cloud data in the sample point cloud data set to determine key point cloud features of sample targets in the corresponding batch of sample point cloud data includes:

[0027] Obtaining sample bounding box parameter information of a corresponding batch of sample point cloud data in the sample point cloud dataset;

[0028] Determining point cloud data within the sample bounding box based on the sample bounding box parameter information;

[0029] Using the information extraction module to perform multi-layer perception processing on the sample bounding box parameter information and the point cloud data within the sample bounding box, respectively, to determine the sample target bounding box feature information and the feature data within the sample bounding box;

[0030] The information extraction module is used to fuse the sample target bounding box feature information and the feature data within the sample bounding box to determine the key point cloud features of the sample target in the corresponding batch of sample point cloud data.

[0031] According to one embodiment of the present application, using the information supplementation module and the corresponding batch of sample point cloud data to perform local information supplementation on the key point cloud features and determine the global point cloud features of the samples acquired by the auxiliary model this time includes:

[0032] Sampling the corresponding batch of sample point cloud data to determine sampled point cloud data;

[0033] Perform multi-level feature extraction on the sampling point cloud data using the information supplement module to determine local features of the sampling point cloud;

[0034] The information supplement module is used to fuse the key point cloud features and the local features of the sampled point cloud to determine the global point cloud features of the samples acquired by the auxiliary model this time.

[0035] According to one embodiment of the present application, determining the model loss of the intermediate target detection model after the current training based on the corresponding batch of sample point cloud data, the sample global point cloud features, the sample original point cloud features, the auxiliary model, and the intermediate target detection model includes:

[0036] Determining a first loss based on the global point cloud features of the sample and the intermediate target detection model after this training;

[0037] Determining a second loss based on the corresponding batch of sample point cloud data and the intermediate target detection model after the current training;

[0038] Determining a third loss based on the corresponding batch of sample point cloud data and the auxiliary model;

[0039] Determining a fourth loss based on the sample original point cloud features determined by the intermediate target detection model after the current training and the sample global point cloud features;

[0040] Based on the first loss, the second loss, the third loss, and the fourth loss, a model loss of the intermediate object detection model after this training is determined.

[0041] The target detection method according to the second embodiment of the present application includes:

[0042] Obtaining the point cloud data to be measured and the preset target detection model determined by training according to the model training method described in the first aspect;

[0043] The point cloud data to be measured is input into the preset target detection model, and the target detection result of the target to be measured in the point cloud data to be measured is determined according to the output result of the preset target detection model.

[0044] According to the target detection method provided in the present application, the target detection result of the target to be measured can be determined by inputting the point cloud data to be measured into a preset target detection model for target detection. This achieves high-precision detection of the target to be measured in the point cloud data to be measured without adding additional time and computational costs, thereby greatly improving the applicability and efficiency of three-dimensional target detection.

[0045] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 This is a flow chart of the model training method provided in the embodiment of the present application;

[0048] Figure 2 This is a schematic diagram of the LAI processing structure provided in an embodiment of the present application;

[0049] Figure 3 This is a schematic diagram of the LKM processing structure provided in an embodiment of the present application;

[0050] Figure 4 Schematic diagram of the target detection method provided in the embodiment of the present application;

[0051] Figure 5 Schematic diagram of the overall process of the target training and target detection method provided in the embodiment of the present application;

[0052] Figure 6 Schematic diagram of the overall structure of the target training and target detection method provided in the embodiment of the present application;

[0053] Figure 7 This is a schematic diagram comparing target detection results under indoor scene point clouds provided by an embodiment of the present application;

[0054] Figure 8 This is a schematic diagram of the structure of the model training device provided in an embodiment of the present application;

[0055] Figure 9 is a schematic structural diagram of a target detection device provided in an embodiment of the present application;

[0056] Figure 10 It is a schematic diagram of the physical structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0058] Among the existing methods for target detection, three-dimensional target detection networks with feature extraction and bounding box prediction functions are usually used for three-dimensional target detection. When performing three-dimensional target detection, the feature extraction part of the three-dimensional target detection network can be improved, such as fusing two-dimensional and three-dimensional data from a data perspective, and designing new convolution operators from a convolution perspective to enhance the capture of point cloud features; the bounding box prediction part of the three-dimensional target detection network can also be improved, such as adding specific constraints for bounding box prediction.

[0059] However, while directly improving the 3D object detection network can improve 3D object detection accuracy, it also increases the complexity and computational effort of the entire detection process. Furthermore, the 3D object detection network, which improves detection accuracy at the expense of time, also has significant hardware limitations, resulting in limited applicability for 3D object detection. Based on this, the inventors have proposed a model training and object detection method, device, electronic device, and storage medium.

[0060] The following combination Figures 1-10 The model training, target detection method, device, electronic device and storage medium of the present application are described, wherein the execution subject of the model training method can be a terminal device, and the terminal device can be other electronic devices such as a personal computer (PC), a portable device, a laptop, a smart phone, a tablet computer and a portable wearable device. It is understandable that the execution subject of the model training method can also be a server. The present application does not limit the specific form of the terminal device or server. The following method embodiment is illustrated by taking the execution subject as a terminal device as an example.

[0061] Figure 1 A flow chart of the model training method provided in this application, such as Figure 1 As shown, the model training method includes the following steps:

[0062] Step 110 : Acquire a sample point cloud dataset and an auxiliary model. The auxiliary model is used to determine key point cloud features of sample targets in the sample point cloud dataset and to supplement local information of the key point cloud features.

[0063] It is understood that in one embodiment, the sample point cloud dataset can be obtained by using a depth camera sensor to collect a 3D sample image dataset containing sample targets for indoor living scenes or indoor office scenes. The 3D sample image dataset can be composed of multiple scene image data, each of which contains 2D image data and depth image data. Point cloud conversion is then performed on each depth image data in the 3D sample image dataset to determine the sample point cloud dataset corresponding to the 3D sample image dataset. In this case, the sample point cloud dataset can also be point cloud data collected for multiple indoor living scenes and / or indoor office scenes. The sample targets can be 3D targets in indoor living scenes, such as sofas, desks, washing machines, and refrigerators; the sample targets can also be printers, desks, computer hosts, and office chairs in indoor office scenes. This is not specifically limited here. Those skilled in the art will understand that the sample point cloud dataset can be for other scenes, such as commercial scenes, transportation scenes, etc., and this is not limited here.

[0064] Step 120: Use the sample point cloud dataset and the auxiliary model to train the initial target detection model to determine the preset target detection model.

[0065] The initial target detection model may be an existing three-dimensional target detection model including a feature extraction function and a bounding box detection function.

[0066] It can be understood that the auxiliary model can be a label-guided three-dimensional target detection auxiliary training model (Label-Guided Auxiliary Training Improves 3D Object Detector, LG3D), and is only connected to the initial target detection model during the model training phase to achieve the auxiliary training purpose. When the initial target detection model is trained using sample point cloud data and the auxiliary model until the training is completed, the corresponding target detection model at the end of the training is determined as the preset target detection model, and the auxiliary model is removed at the same time, so that the preset target detection model can be used directly for three-dimensional target detection in the future.

[0067] The model training method provided in the embodiment of the present application trains the initial target detection model through the auxiliary model in combination with the sample point cloud dataset. There is no need to improve the target detection model itself, and therefore it will not increase the complexity and computational complexity of the subsequent target detection stage; and, since the auxiliary model is used to determine the key point cloud features of the sample target in the sample point cloud dataset and to supplement the key point cloud features with local information, the use of the auxiliary model and the sample point cloud dataset to train the model can greatly improve the precision and accuracy of the training model, thereby achieving the goal of ensuring the accuracy of target detection without adding additional time and computational costs, and also greatly improving the applicability and efficiency of target detection.

[0068] It is understandable that in order to improve the accuracy of model training, a model training times threshold can be pre-set so that a trained model is obtained when the model training times threshold is reached. Based on this, the implementation process of step 120 may include:

[0069] First, the initial target detection model is trained using the preset batch of sample point cloud data and the auxiliary model in the sample point cloud dataset to obtain the model loss of the intermediate target detection model after the preset number of trainings. Then, based on the successful matching result between the model loss and the preset model loss, the intermediate target detection model corresponding to the model loss is determined to be the preset target detection model.

[0070] It is understandable that since the sample point cloud dataset is point cloud data for several scene image data in indoor life scenes or indoor office scenes, the sample point cloud dataset can be trained in batches, and a preset number of batches of sample point cloud data can be used to participate in model training, and the order of the batches of sample point cloud data corresponds to the number of training times. For example, when the number of scene image data is 10,000, 4 or 8 scene image data sample point cloud data can be executed in each batch, and the number of scene image data executed in each batch can be a power of 2, and training can be performed 100 times or 200 times first. If the fourth batch of sample point cloud data is used to participate in model training, it can be considered that the intermediate target detection model after the third training is currently being trained for the fourth time.

[0071] After the initial target detection model has been trained a preset number of times, it is determined whether the model loss of the intermediate target detection model after the preset number of training matches the preset model loss. If the model loss of the intermediate target detection model after the preset number of training matches the preset model loss, model training is stopped, and the intermediate target detection model corresponding to the model loss at the time of model training stop is determined as the trained preset target detection model. The preset model loss can be an empirical loss value, a loss threshold range, or an ideal loss level that can be achieved after a preset number of training times set manually based on experience. This is not specifically limited here.

[0072] The model training method provided in the embodiment of the present application improves the model training speed and reduces the model training complexity by using a preset batch of sample point cloud data in the sample point cloud dataset and an auxiliary model to train the initial target detection model a preset number of times, thereby effectively improving the model training accuracy.

[0073] It is understandable that when the model effect after the preset number of trainings does not meet the preset requirements, the preset number of trainings can be repeated. Based on this, after obtaining the model loss of the intermediate target detection model after the preset number of trainings, the method of the embodiment of the present application may further include:

[0074] Based on the failure of matching the model loss with the preset model loss, the next preset batch of sample point cloud data and the auxiliary model in the sample point cloud dataset are used to train the intermediate target detection model corresponding to the model loss to determine the preset target detection model.

[0075] It is understandable that, for the case where the initial target detection model has been trained a preset number of times, if it is determined that the model loss of the intermediate target detection model after the preset number of training does not match the preset model loss, it means that the intermediate target detection model obtained at this time has not met the preset requirements. At this time, the next preset batch of sample point cloud data can be selected from the sample point cloud data set to continue model training. For example, for a sample point cloud data set determined for 10,000 scene image data, if 200 batches of sample point cloud data are selected from the sample data set for training and fail to meet the preset requirements, another 200 batches of sample point cloud data can be selected from the remaining sample point cloud data of the sample point cloud data set to participate in subsequent training. Among them, the batch size of the preset batch of point cloud data selected again can be the same as or different from the batch size of the previous preset batch of point cloud data, which is not specifically limited here. In addition, when selecting the next preset batch of point cloud data from the remaining sample point cloud data of the sample point cloud data set, it can be selected sequentially or at intervals. This is also not specifically limited here.

[0076] The model training method provided in the embodiment of the present application ensures the reliability and accuracy of model training and improves the flexibility and stability of model training by reselecting a preset batch of sample point cloud data to continue training when the intermediate target detection model after the initial target detection model has been trained a preset number of times does not match the preset model loss.

[0077] It is understandable that in order to avoid the additional time and computational costs of training the initial target detection model by directly improving it, an auxiliary model can be used to iteratively train the initial target detection model instead of directly improving the model itself. Based on this, the initial target detection model is trained using a preset batch of sample point cloud data from the sample point cloud dataset and the auxiliary model, and the model loss of the intermediate target detection model after a preset number of training times is obtained. The process may include:

[0078] First, for each training, the corresponding batch of sample point cloud data in the sample point cloud dataset is input into the auxiliary model to determine the global point cloud features of the samples obtained by the auxiliary model this time; secondly, the corresponding batch of sample point cloud data is input into the intermediate target detection model after the previous training to determine the original point cloud features of the samples extracted by the intermediate target detection model after this training; then, based on the corresponding batch of sample point cloud data, the global point cloud features of the samples, the original point cloud features of the samples, the auxiliary model and the intermediate target detection model, the model loss of the intermediate target detection model after this training is determined.

[0079] It can be understood that since the auxiliary model is used to determine the key point cloud features of the sample targets in the sample point cloud dataset and to supplement the key point cloud features with local information, after the corresponding batch of sample data is input into the auxiliary model, the global point cloud features of the samples obtained by the auxiliary model this time can be determined. In addition, since the initial target detection model has a feature extraction function, for each training, the input corresponding batch of sample point cloud data can be directly extracted. That is, after the corresponding batch of sample point cloud data is input into the midpoint target detection model determined after the previous training, the original point cloud features of the samples extracted by the intermediate target detection model after this training can be determined. At this time, the intermediate target detection model after the previous training is also trained again. For example, after the 8th batch of sample point cloud data is input into the intermediate target detection model after the 7th training, the original point cloud features of the samples corresponding to the 8th batch of sample point cloud data can be determined, and the intermediate target detection model after the 8th training can also be obtained. If the first batch of sample point cloud data is used, the initial target detection model can be trained for the first time. That is, after the first batch of sample point cloud data is input into the initial target detection model, the sample original point cloud features extracted by the intermediate target detection model after the first training can be determined. Then, based on the corresponding batch of sample point cloud data, the sample global point cloud features, the sample original point cloud features, the auxiliary model and the intermediate target detection model, the model loss of the intermediate target detection model after this training is determined, such as the loss generated when the corresponding batch of sample point cloud data is directly input into the intermediate target detection model for feature extraction, the loss generated when the sample point cloud features determined in the auxiliary model are input into the intermediate target detection model for target detection, the distance loss between the sample original point cloud features and the sample original point cloud features, and the loss generated in the process of the auxiliary model determining the sample global point cloud features.

[0080] The model training method provided in this application determines the global point cloud features of the samples determined by the auxiliary model, extracts the original point cloud features of the samples by the intermediate target detection model after this training, and combines the corresponding batch sample point cloud data and the intermediate target detection model to determine the model loss of the intermediate target detection model after this training, thereby improving the accuracy and reliability of model training.

[0081] It is understandable that since the auxiliary model is used to determine the key point cloud features of the sample target in the sample point cloud dataset and to supplement the key point cloud features with local information, the auxiliary module can be used to illustrate the specific auxiliary functions of the auxiliary model in the model training stage by having information extraction and information supplement functions. Based on this, the corresponding batch of sample point cloud data in the sample point cloud dataset is input into the auxiliary model to determine the global point cloud features of the samples obtained by the auxiliary model. The process may include:

[0082] First, the information extraction module is used to extract key information from the corresponding batch of sample point cloud data in the sample point cloud dataset, and the key point cloud features of the sample targets in the corresponding batch of sample point cloud data are determined; then the information supplementation module and the corresponding batch of sample point cloud data are further used to supplement the key point cloud features with local information, and determine the global point cloud features of the samples obtained by the auxiliary model this time.

[0083] Among them, the sample targets can be the three-dimensional targets to be identified in each batch of sample point cloud data, and the sample targets can be three-dimensional targets in indoor living scenes, such as sofas, desks, washing machines and refrigerators; they can also be printers, desks, computer hosts and office chairs in indoor office scenes.

[0084] It can be understood that the information extraction module of the auxiliary model can be called a label annotation inducer (LAI), and the information supplement module can be called a label knowledge mapper (LKM), and the output end of the LAI is connected to the input end of the LKM. At this time, for the case where each batch of sample point cloud data is a sample point cloud data of a power of 2 scene image data, if each scene image data identifies a three-dimensional target to be identified, then each batch of sample point cloud data enters the LAI to extract the key information of each three-dimensional target, thereby obtaining the key point cloud features of the sample target in the corresponding batch of sample point cloud data, and the key point cloud features will be input into the LKM, so that the LKM performs local information supplementation on the key point cloud features based on the corresponding batch of sample point cloud data, thereby obtaining the sample global point cloud features output by the auxiliary target this time, and the sample global point cloud features are used to identify the target category, target orientation and target bounding box position of the sample target in the corresponding batch of sample point cloud data.

[0085] The model training method provided in the embodiment of the present application ensures the indispensability of the auxiliary model in the auxiliary training model stage by extracting the key point cloud features of the sample targets in the corresponding batch of sample point cloud data through the information extraction model in the auxiliary model and the information supplementation module performing local information supplementation on the key point cloud features, thereby effectively reducing the computational amount and complexity of the initial target detection model in the training stage.

[0086] It is understandable that, considering that each 3D target to be identified is identified in the 3D sample image dataset collected for indoor living scenes or indoor office scenes, and when the 3D sample image dataset is converted into a sample point cloud dataset, each corresponding 3D target to be identified is also identified in a bounding box and displayed with bounding box parameter information, key information can be extracted using the identified bounding box parameter information. Based on this, an information extraction module is used to extract key information from the corresponding batch of sample point cloud data in the sample point cloud dataset, and determine the key point cloud features of the sample targets in the corresponding batch of sample point cloud data, including:

[0087] First, the sample bounding box parameter information of the corresponding batch of sample point cloud data in the sample point cloud dataset is obtained; then, based on the sample bounding box parameter information, the point cloud data within the sample bounding box is determined; then, the information extraction module is used to perform multi-layer perception processing on the sample bounding box parameter information and the point cloud data within the sample bounding box respectively to determine the sample target bounding box feature information and the feature data within the sample bounding box; finally, the information extraction module is used to fuse the sample target bounding box feature information and the feature data within the sample bounding box to determine the key point cloud features of the sample target in the corresponding batch of sample point cloud data.

[0088] It can be understood that since the sample targets contained in each batch of sample point cloud data in the sample point cloud dataset are all marked in the bounding box and displayed with the bounding box parameter information, the bounding box parameter information of each sample of the corresponding batch of sample point cloud data in the sample point cloud dataset can be directly obtained, where the i-th sample bounding box parameter information can be a i , a i =(x i ,y i ,z i ,l i ,w i ,h i ,c i ), (x i ,y i ,z i ) represents the three-dimensional coordinates of the center point of the i-th sample bounding box, w i 、h i and z i are the length, width and height of the bounding box of the i-th sample, c i Indicates the target category of the sample target corresponding to the i-th bounding box. In addition, point cloud data is collected for each bounding box of the corresponding batch of sample point cloud data in the sample point cloud dataset, and the point cloud data within the sample bounding box of the corresponding batch of sample point cloud data can be determined. The process of inputting the sample bounding box parameter information and the point cloud data within the sample bounding box into LAI for processing can be referred to. Figure 2LAI specifically contains a label encoder, a label point cloud encoder, and an attention mechanism query (Query, Q)-key value (Key, K)-value item (Value, V). The label encoder and label point cloud encoder are multi-layer perceptrons. The bounding box parameter information of each sample in the corresponding batch of sample point cloud data is input into the label encoder and the label point cloud encoder respectively for encoding, and two sets of encoding vectors of the same dimension are obtained respectively. The two sets of encoding vectors of the same dimension are then input into the attention mechanism QKV for fusion to determine the key point cloud features of the sample targets in the corresponding batch of sample point cloud data.

[0089] It should be noted that if the corresponding batch of sample point cloud data in the sample point cloud dataset is represented as n represents the total number of sample point cloud data of the corresponding batch in the sample point cloud dataset, and the point cloud data within the sample bounding box of the corresponding batch of sample point cloud data is represented as m represents the total number of point cloud data within all sample bounding boxes of the corresponding batch of sample point cloud data in the sample point cloud dataset. Then, the original labels of the N sample targets corresponding to the batch of sample point cloud data in the sample point cloud dataset can be expressed as A, The length of the array representing the parameter information of the bounding box of the i-th sample is C L .

[0090] In order to enhance the auxiliary training effect of the initial target detection model, the label encoder can be set to have a data enhancement function, that is, the three-dimensional coordinates of the center point of each sample bounding box are randomly jittered to determine the three-dimensional coordinates of the new center point of each sample bounding box, where the three-dimensional coordinates of the new center point of the i-th sample bounding box are (x' i ,y' i ,z' i ), x' i =x i +φ x l i , y' i =y i +φ y w i , z' i =z i +φ z h i ,φ x 、φ y 、φ z All of them are collected from the uniform distribution [-0.1, 0.1], so as to obtain the pseudo labels A' of N sample targets corresponding to the batch of sample point cloud data in the sample point cloud dataset, A'={a'1,...,a' i ,...,a' N}, a' i =(x'i ,y' i ,z' i ,l i ,w i ,h i ,c i ), and then further merge the original label A and the pseudo label A' to determine the new label set A corresponding to the batch of sample point cloud data in the sample point cloud dataset all , A all =A∪A', When 1≤i≤N, When N≤i≤2N, Then, the new label set of the corresponding batch of sample point cloud data in the sample point cloud dataset is input into the label encoder for encoding, and the encoding vector β output by the label encoder is determined. A , β A ={e1,...,e i ,...,e N}, e i Represents the encoded data obtained after the label of the i-th sample bounding box passes through the label encoder, The length of the array of encoded data obtained after the label of the i-th sample bounding box passes through the label encoder is C L1 .

[0091] Input the bounding box parameter information of each sample of the corresponding batch of sample point cloud data into the label point cloud encoder for encoding, and determine the encoding vector output by the label point cloud encoder. The encoding process of the label point cloud encoder here can refer to the encoding process of the label encoder mentioned above. It will not be repeated here. If the encoding vector output by the label point cloud encoder is recorded as ψ(p l ), then the encoding vector β A and the encoding vector ψ(p l ) is input into the attention mechanism QKV for fusion: the encoding vector β A As the query value Q, the encoding vector ψ(p l ) as the key value K and the item value V, the fusion process can be:

[0092] Softmax is a normalized exponential operation, · represents a dot product operation, the superscript T represents a transpose operation, p represents the first scaling factor, and × represents a tensor multiplication operation. In this way, the key point cloud features of the sample targets in the corresponding batch of sample point cloud data are obtained by fusion, and the key point cloud features are represented in the form of vectors. Since the key point cloud features are determined by fusing the outputs of the label encoder and the label point cloud encoder, the key point cloud features can not only represent the point cloud features of the sample targets, but also the label features. The label features can refer to the inputs of the label encoder and the label point cloud encoder. The key point cloud features determined at this time can also achieve the purpose of fusing the high-dimensional data of the corresponding batch of sample point cloud data and the high-dimensional data of the point cloud data within the sample bounding box into a low-dimensional vector.

[0093] The model training method provided in the embodiment of the present application determines the key point cloud features of the sample targets in the corresponding batch of sample point cloud data by performing multi-layer perception processing on the sample bounding box parameter information of the corresponding batch of sample point cloud data and the point cloud data within the sample bounding box in the sample point cloud data set through an auxiliary model, and then fusing the feature data obtained by the multi-layer perception. In this way, the reliability and accuracy of determining the key point cloud features are improved by combining the multi-layer perception processing technology and the attention fusion mechanism, thereby providing a reliable guarantee for subsequently improving the accuracy of model training.

[0094] It is understandable that relying solely on key point cloud features is insufficient for accurate identification of sample targets. Therefore, the information supplementation module can be used to identify local feature information from the corresponding batch of sample point cloud data to supplement the key point cloud features. Based on this, the information supplementation module and the corresponding batch of sample point cloud data are used to supplement the key point cloud features with local information and determine the global point cloud features of the samples acquired by the auxiliary model. The process may include:

[0095] First, the corresponding batch of sample point cloud data is sampled to determine the sampled point cloud data; then, the information supplement module is used to perform multi-level feature extraction on the sampled point cloud data to determine the local features of the sampled point cloud; further, the information supplement module is used to fuse the key point cloud features and the local features of the sampled point cloud to determine the global point cloud features of the samples obtained by the auxiliary model this time.

[0096] It is understandable that, considering the large number of sample point cloud data in each batch, each batch of sample point cloud data can be randomly sampled first, the sampled point cloud data obtained after random sampling is determined, and then the sampled point cloud data and the key point cloud features output by LAI are input into LKM. The process of inputting the sampled point cloud data and the key point cloud features output by LAI into LKM for processing can be referred to. Figure 3LKM specifically contains the PointNet++ network and the attention mechanism QKV. Since the PointNet++ network has the function of hierarchical feature extraction, when the sampled point cloud data enters the LKM, it first performs multi-level feature extraction on the sampled point cloud data through the PointNet++ network, that is, hierarchical feature extraction, to obtain the local features of the sampled point cloud. Then, the local features of the sampled point cloud and the key point cloud features are fused through the attention mechanism QKV in the LKM. The fusion process is as follows:

[0097] Softmax is a normalized exponential operation, Q' represents the local features of the sampled point cloud, K' and V' both represent key point cloud features, · represents a dot product operation, the superscript T indicates a transpose operation, p' represents the second scaling factor, and × represents a tensor multiplication operation. This fusion is used to obtain the global point cloud features of the samples acquired by the auxiliary model.

[0098] The model training method proposed in the embodiment of the present application first performs multi-level feature extraction on the sample point cloud data obtained by sampling the corresponding batch of sample point cloud data in the sample point cloud dataset through the auxiliary model, and then fuses the extracted sample point cloud local features and key point cloud features to determine the sample global point cloud features obtained by the auxiliary model this time. In this way, the reliability and accuracy of determining the sample global point cloud features are improved by combining the layered feature extraction technology and the attention fusion mechanism, providing a reliable guarantee for subsequently improving the accuracy of model training.

[0099] It is understandable that in order to improve the accuracy of model training, the model loss of the intermediate target detection model obtained in each training can be determined, so as to quickly judge whether the intermediate target detection model obtained in the current training is well trained based on the model loss. Based on this, the model loss of the intermediate target detection model after this training is determined based on the corresponding batch of sample point cloud data, sample global point cloud features, sample original point cloud features, auxiliary model and intermediate target detection model, including:

[0100] A first loss is determined based on the global point cloud features of the sample and the intermediate target detection model after this training; a second loss is determined based on the corresponding batch of sample point cloud data and the intermediate target detection model after this training; a third loss is determined based on the corresponding batch of sample point cloud data and the auxiliary model; a fourth loss is determined based on the original point cloud features of the sample and the global point cloud features of the sample determined by the intermediate target detection model after this training; and a model loss of the intermediate target detection model after this training is determined based on the first loss, the second loss, the third loss, and the fourth loss.

[0101] It can be understood that based on the global point cloud features of the sample and the intermediate target detection model after this training, the first loss can be determined to be L det1 , L det1=d(τ(p,(p r ,A oh )),A), based on the corresponding batch sample point cloud data and the intermediate target detection model after this training, the second loss can be determined to be L det2 , L det2 =d(f(p),A), based on the corresponding batch sample point cloud data and the auxiliary model, the third loss can be determined as Based on the sample original point cloud features and sample global point cloud features determined by the intermediate target detection model after this training, the fourth loss can be determined to be L aux , L aux =λL dis (f(p),τ(p,(p l ,A oh ))), based on the first loss, the second loss, the third loss and the fourth loss, it can be determined that the model loss of the intermediate target detection model after this training is L total , L total =L det1 +L det2 +L tf +L aux .

[0102] Among them, A oh Represents the one-hot encoding form of the original label A of the N sample targets corresponding to the batch of sample point cloud data in the sample point cloud dataset, τ(p,(p r ,A oh )) represents the point cloud data p corresponding to the batch of sample point cloud data in the sample point cloud dataset, the point cloud data p within the sample bounding box of the corresponding batch of sample point cloud data r and the one-hot encoding form A of the original label A oh Input into LKM to calculate the fusion feature, d(τ(p,(p r ,A oh )), A) represents the intermediate target detection model after this training for the fusion feature (τp, (p r ,A oh )) and the sum of the classification loss and position loss calculated from the original label A, f(p) represents the calculation of feature extraction performed by the intermediate target detection model after this training for the corresponding batch of sample point cloud data p in the sample point cloud dataset, λ represents a hyperparameter and is used to control the fourth loss L aux In the model loss L total The proportion of L dis represents the mean square error calculation; express or The value of the corresponding label, that is, hour hour ρ obj (e i ) represents the encoded data e obtained by the label encoder of the i-th sample bounding box in the corresponding batch of sample point cloud data p in the sample point cloud dataset i The input corresponds to the fully connected layer for calculation, and the calculated result is activated based on the sigmoid activation function.

[0103] The model training method provided in this application determines the model loss of the intermediate target detection model after this training through the detection process in which the sample global point cloud features output by the auxiliary model enter the intermediate target detection model after this training, the detection process in which the corresponding batch of sample point cloud data directly enters the intermediate target detection model, the distance loss between the features obtained by the corresponding batch of sample point cloud data entering the intermediate target detection model and the sample global point cloud features, and the loss of the auxiliary model in multiple perception processes, thereby improving the accuracy and comprehensiveness of determining the model loss, improving the accuracy of model training, and thus ensuring the accuracy and reliability of subsequent three-dimensional target detection.

[0104] Reference Figure 4 , is a flow chart of the target detection method provided by this application, wherein the target detection method may also be executed by a terminal device or a server. This application does not limit the specific form of the terminal device or server. The following method embodiments are described using the terminal device as an example.

[0105] like Figure 4 As shown, the target detection method includes the following steps:

[0106] Step 410: Acquire the point cloud data to be measured, and train the preset target detection model determined according to the model training method provided in any embodiment of the present application.

[0107] It is understandable that the point cloud data to be measured can also be for indoor living scenes or indoor office scenes, using a depth camera sensor to collect three-dimensional image data containing the target to be measured. The three-dimensional image data to be measured can contain two-dimensional image data and depth image data, and then the depth image data in the three-dimensional image data to be measured is converted into a point cloud to determine the point cloud data to be measured corresponding to the three-dimensional image data to be measured. At this time, the point cloud data to be measured can also be point cloud data collected for indoor living scenes or indoor office scenes. Among them, the target to be measured can also be a three-dimensional target similar to a sofa, desk, washing machine, and refrigerator in an indoor living scene, or a three-dimensional target similar to a printer, desk, computer host, and office chair in an indoor office scene. No specific limitation is made here.

[0108] Step 420: Input the point cloud data to be measured into a preset target detection model, and determine the target detection result of the target to be measured in the point cloud data according to the output result of the preset target detection model.

[0109] It can be understood that the point cloud data to be measured is input into the preset target detection model for feature extraction and bounding box detection, and based on the output results of the preset target detection model, the target category, target orientation and target three-dimensional bounding box position of the target to be measured in the point cloud data to be measured can be determined.

[0110] The target detection method provided in the embodiment of the present application can determine the target detection result of the target to be measured by inputting the point cloud data to be measured into a preset target detection model for target detection. This achieves high-precision detection of the target to be measured in the point cloud data to be measured without adding additional time and computational costs, thereby greatly improving the applicability and efficiency of three-dimensional target detection.

[0111] It is understood that the overall flow diagram of the target training and target detection method provided in the embodiment of the present application is as follows: Figure 5 As shown, the training phase includes a process of training the initial target detection model M times using the sample point cloud dataset and the auxiliary model, and a process of performing target detection using the target point cloud data to be tested and the preset target detection model during the detection phase. The target point cloud data to be tested can be part of the point cloud data in the sample point cloud dataset. Where M is a preset number of times, each training process can be cross-referenced with the previous embodiment, and the target detection process can also be cross-referenced with the previous embodiment, and of course, the implementation method is not limited to the previous embodiment.

[0112] It is understood that the overall structural diagram of the target training and target detection method provided in the embodiment of the present application is as follows: Figure 6 As shown, in Figure 6 In the figure, the thin arrows indicate the model training phase, and the thick arrows indicate the target detection phase, including the process of using the sample point cloud dataset, LAI, and LKM to train the model and determine the model loss after each training, as well as the process of using the preset target detection model to perform feature extraction and bounding box detection on the target point cloud data. Each training process and the process of determining the model loss can be compared with the aforementioned embodiment, and the process of using the preset target detection model to perform feature extraction and bounding box detection on the target point cloud data can also be compared with the aforementioned embodiment. Of course, the implementation method is not limited to the aforementioned embodiment.

[0113] It is understandable that, combined with Figure 7 The target detection result comparison diagram under the indoor scene point cloud provided by the embodiment of the present application is analyzed and explained in the present application method. Figure 7In [1], VoteNet represents the target prediction result obtained by using the existing voting network, GroupFree3D represents the target prediction result obtained by using the existing non-combined 3D network, GT represents the original dataset label of the indoor scene point cloud, VoteNet+Ours represents the target prediction result obtained by using the method of this application to assist the voting network, and GroupFree3D+Ours represents the target prediction result obtained by using the method of this application to assist the non-combined 3D network. Figure 7 It can be seen that the size and number of bounding boxes in the target prediction results obtained with the assistance of the method of the present application are closer to the actual label situation of the original dataset of the indoor scene point cloud than the size and number of bounding boxes obtained by using the existing voting network and the uncombined 3D network for target prediction, which shows that the use of the method of the present application can greatly improve the accuracy and reliability of target detection.

[0114] The model training device provided in this application is described below. The model training device described below and the model training method described above can be referenced to each other.

[0115] Reference Figure 8 , is a structural diagram of the model training device provided in this application, such as Figure 8 As shown, the model training device 800 includes:

[0116] An information acquisition unit 810 is used to acquire a sample point cloud dataset and an auxiliary model, wherein the auxiliary model is used to determine key point cloud features of a sample target in the sample point cloud dataset and to supplement local information of the key point cloud features;

[0117] The model training unit 820 is used to train the initial target detection model using the sample point cloud dataset and the auxiliary model to determine the preset target detection model.

[0118] It can be understood that the model training unit 820 can be specifically used to use a preset batch of sample point cloud data and an auxiliary model in the sample point cloud data set to train the initial target detection model, and obtain the model loss of the intermediate target detection model after a preset number of training times; based on the successful matching result of the model loss and the preset model loss, the intermediate target detection model corresponding to the model loss is determined to be the preset target detection model.

[0119] It can be understood that the model training unit 820 can also be specifically used to train the intermediate target detection model corresponding to the model loss based on the failure result of matching the model loss with the preset model loss, using the next preset batch of sample point cloud data and the auxiliary model in the sample point cloud data set to determine the preset target detection model.

[0120] It can be understood that the model training unit 820 can also be specifically used to input the corresponding batch of sample point cloud data in the sample point cloud data set into the auxiliary model for each training, and determine the global point cloud features of the samples obtained by the auxiliary model this time; input the corresponding batch of sample point cloud data into the intermediate target detection model after the previous training, and determine the original point cloud features of the samples extracted by the intermediate target detection model after this training; based on the corresponding batch of sample point cloud data, the sample global point cloud features, the sample original point cloud features, the auxiliary model and the intermediate target detection model, determine the model loss of the intermediate target detection model after this training.

[0121] It can be understood that the model training unit 820 can also be specifically used to use the information extraction module to extract key information from the corresponding batch of sample point cloud data in the sample point cloud data set, and determine the key point cloud features of the sample targets in the corresponding batch of sample point cloud data; use the information supplement module and the corresponding batch of sample point cloud data to perform local information supplement on the key point cloud features, and determine the global point cloud features of the samples obtained by the auxiliary model this time.

[0122] It can be understood that the model training unit 820 can also be specifically used to obtain the sample bounding box parameter information of the corresponding batch of sample point cloud data in the sample point cloud data set; based on the sample bounding box parameter information, determine the point cloud data within the sample bounding box; use the information extraction module to perform multi-layer perception processing on the sample bounding box parameter information and the point cloud data within the sample bounding box, respectively, to determine the sample target bounding box feature information and the feature data within the sample bounding box; use the information extraction module to fuse the sample target bounding box feature information and the feature data within the sample bounding box to determine the key point cloud features of the sample target in the corresponding batch of sample point cloud data.

[0123] It can be understood that the model training unit 820 can also be used to sample the corresponding batch of sample point cloud data to determine the sampled point cloud data; use the information supplement module to perform multi-level feature extraction on the sampled point cloud data to determine the local features of the sampled point cloud; use the information supplement module to fuse the key point cloud features and the local features of the sampled point cloud to determine the global point cloud features of the samples obtained by the auxiliary model this time.

[0124] It can be understood that the model training unit 820 can also be specifically used to determine the first loss based on the global point cloud features of the sample and the intermediate target detection model after this training; determine the second loss based on the corresponding batch of sample point cloud data and the intermediate target detection model after this training; determine the third loss based on the corresponding batch of sample point cloud data and the auxiliary model; determine the fourth loss based on the original point cloud features of the sample and the global point cloud features of the sample determined by the intermediate target detection model after this training; determine the model loss of the intermediate target detection model after this training based on the first loss, the second loss, the third loss and the fourth loss.

[0125] The target detection device provided in the present application is described below. The target detection device described below and the target detection method described above can be referenced to each other.

[0126] Reference Figure 9 , is a schematic diagram of the structure of the target detection device provided by this application, such as Figure 8 As shown, the target detection device 900 includes:

[0127] The model acquisition unit 910 is used to obtain a point cloud dataset to be tested and a preset target detection model determined by training according to the model training method provided in any embodiment of the present application;

[0128] The target detection unit 920 is used to input the target point cloud data set to be measured into a preset target detection model, and determine the target detection result of the target target in the target point cloud data set to be measured according to the output result of the preset target detection model.

[0129] Figure 10 An example of a physical structure diagram of an electronic device is shown below. Figure 10 As shown, the electronic device 1000 may include: a processor 1010, a communication interface 1020, a memory 1030, and a communication bus 1040, wherein the processor 1010, the communication interface 1020, and the memory 1030 communicate with each other via the communication bus 1040. The processor 1010 may call the logic instructions in the memory 1030 to execute the following method:

[0130] Obtaining a sample point cloud dataset and an auxiliary model, the auxiliary model is used to determine key point cloud features of sample targets in the sample point cloud dataset and to supplement local information of the key point cloud features;

[0131] Use the sample point cloud dataset and auxiliary model to train the initial target detection model and determine the preset target detection model. Alternatively:

[0132] Obtaining the point cloud data to be measured and a preset target detection model determined by training according to the model training method provided in any embodiment of the present application;

[0133] The point cloud data to be measured is input into a preset target detection model, and the target detection result of the target to be measured in the point cloud data to be measured is determined according to the output result of the preset target detection model.

[0134] In addition, the logic instructions in the above-mentioned memory 1030 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the relevant technology or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0135] On the other hand, embodiments of the present application disclose a computer program product, comprising a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions. When the program instructions are executed by a computer, the computer can perform the methods provided in the above-mentioned method embodiments, for example, including:

[0136] Obtaining a sample point cloud dataset and an auxiliary model, the auxiliary model is used to determine key point cloud features of sample targets in the sample point cloud dataset and to supplement local information of the key point cloud features;

[0137] Use the sample point cloud dataset and auxiliary model to train the initial target detection model and determine the preset target detection model. Alternatively:

[0138] Obtaining the point cloud data to be measured and a preset target detection model determined by training according to the model training method provided in any embodiment of the present application;

[0139] The point cloud data to be measured is input into a preset target detection model, and the target detection result of the target to be measured in the point cloud data to be measured is determined according to the output result of the preset target detection model.

[0140] In another aspect, embodiments of the present application further provide a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the transmission method provided in each of the above embodiments is implemented, for example, including:

[0141] Obtaining a sample point cloud dataset and an auxiliary model, the auxiliary model is used to determine key point cloud features of sample targets in the sample point cloud dataset and to supplement local information of the key point cloud features;

[0142] Use the sample point cloud dataset and auxiliary model to train the initial target detection model and determine the preset target detection model. Alternatively:

[0143] Obtaining the point cloud data to be measured and a preset target detection model determined by training according to the model training method provided in any embodiment of the present application;

[0144] The point cloud data to be measured is input into a preset target detection model, and the target detection result of the target to be measured in the point cloud data to be measured is determined according to the output result of the preset target detection model.

[0145] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0146] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0147] Finally, it should be noted that the above embodiments are intended only to illustrate the present application and are not intended to limit the present application. Although the present application has been described in detail with reference to the embodiments, those skilled in the art will appreciate that various combinations, modifications, or equivalent substitutions of the technical solutions of the present application do not depart from the spirit and scope of the technical solutions of the present application and are intended to be encompassed by the claims of the present application.

Claims

1. A model training method, characterized in that: include: Acquire a sample point cloud dataset and an auxiliary model, wherein the auxiliary model is used to determine key point cloud features of sample targets in the sample point cloud dataset and to supplement local information of the key point cloud features; The initial target detection model is trained using the sample point cloud dataset and the auxiliary model to determine a preset target detection model; the sample point cloud dataset is determined by performing point cloud conversion on each depth image data in the three-dimensional sample image dataset; The auxiliary model is a label-guided 3D object detection auxiliary training model, which is connected to the initial object detection model during the model training phase to achieve the auxiliary training purpose; After the preset target detection model is trained, the auxiliary model is removed so that the preset target detection model can be directly used for three-dimensional target detection in the future; The using the sample point cloud dataset and the auxiliary model to train the initial target detection model to determine the preset target detection model includes: Using a preset batch of sample point cloud data in the sample point cloud dataset and the auxiliary model, the initial target detection model is trained to obtain the model loss of the intermediate target detection model after a preset number of trainings; Based on a successful matching result between the model loss and the preset model loss, determining that the intermediate target detection model corresponding to the model loss is the preset target detection model; The auxiliary model includes an information extraction module and an information supplementation module, which inputs the corresponding batch of sample point cloud data in the sample point cloud dataset into the auxiliary model and determines the global point cloud features of the samples obtained by the auxiliary model, including: Using the information extraction module, extracting key information from the corresponding batch of sample point cloud data in the sample point cloud data set, and determining key point cloud features of the sample targets in the corresponding batch of sample point cloud data; The information supplement module and the corresponding batch of sample point cloud data are used to perform local information supplement on the key point cloud features to determine the global point cloud features of the samples acquired this time by the auxiliary model.

2. The model training method according to claim 1, characterized in that After obtaining the model loss of the intermediate object detection model after the preset number of trainings, the method further includes: Based on the failure of matching the model loss with the preset model loss, the intermediate target detection model corresponding to the model loss is trained using the next preset batch of sample point cloud data in the sample point cloud data set and the auxiliary model to determine the preset target detection model.

3. The model training method according to claim 1, characterized in that The training of the initial target detection model using a preset batch of sample point cloud data in the sample point cloud dataset and the auxiliary model to obtain a model loss of the intermediate target detection model after a preset number of trainings includes: For each training, the corresponding batch of sample point cloud data in the sample point cloud data set is input into the auxiliary model to determine the global point cloud features of the samples obtained by the auxiliary model this time; Inputting the corresponding batch of sample point cloud data into the previously trained intermediate target detection model to determine the sample original point cloud features extracted by the currently trained intermediate target detection model; Based on the corresponding batch of sample point cloud data, the sample global point cloud features, the sample original point cloud features, the auxiliary model and the intermediate target detection model, the model loss of the intermediate target detection model after this training is determined.

4. The model training method according to claim 1, characterized in that The step of extracting key information from a corresponding batch of sample point cloud data in the sample point cloud data set using the information extraction module to determine key point cloud features of sample targets in the corresponding batch of sample point cloud data includes: Obtaining sample bounding box parameter information of a corresponding batch of sample point cloud data in the sample point cloud dataset; Determining point cloud data within the sample bounding box based on the sample bounding box parameter information; Using the information extraction module to perform multi-layer perception processing on the sample bounding box parameter information and the point cloud data within the sample bounding box, respectively, to determine the sample target bounding box feature information and the feature data within the sample bounding box; The information extraction module is used to fuse the sample target bounding box feature information and the feature data within the sample bounding box to determine the key point cloud features of the sample target in the corresponding batch of sample point cloud data.

5. The model training method according to claim 1, characterized in that The method of using the information supplement module and the corresponding batch of sample point cloud data to perform local information supplement on the key point cloud features and determine the global point cloud features of the samples acquired by the auxiliary model this time includes: Sampling the corresponding batch of sample point cloud data to determine sampled point cloud data; Perform multi-level feature extraction on the sampling point cloud data using the information supplement module to determine local features of the sampling point cloud; The information supplement module is used to fuse the key point cloud features and the local features of the sampled point cloud to determine the global point cloud features of the samples acquired by the auxiliary model this time.

6. The model training method according to claim 3, characterized in that The determining of the model loss of the intermediate target detection model after the current training based on the corresponding batch of sample point cloud data, the sample global point cloud features, the sample original point cloud features, the auxiliary model, and the intermediate target detection model includes: Determining a first loss based on the global point cloud features of the sample and the intermediate target detection model after this training; Determining a second loss based on the corresponding batch of sample point cloud data and the intermediate target detection model after the current training; Determining a third loss based on the corresponding batch of sample point cloud data and the auxiliary model; Determining a fourth loss based on the sample original point cloud features determined by the intermediate target detection model after the current training and the sample global point cloud features; Based on the first loss, the second loss, the third loss, and the fourth loss, a model loss of the intermediate object detection model after this training is determined.

7. A target detection method, characterized in that: include: Obtaining point cloud data to be measured and a preset target detection model determined by training according to the model training method according to any one of claims 1 to 6; The point cloud data to be measured is input into the preset target detection model, and the target detection result of the target to be measured in the point cloud data to be measured is determined according to the output result of the preset target detection model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the model training method according to any one of claims 1 to 6, or the target detection method according to claim 7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the model training method according to any one of claims 1 to 6, or the target detection method according to claim 7.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, it implements the model training method according to any one of claims 1 to 6, or the target detection method according to claim 7.

Images