Detection model training method, target detection method, device and storage medium

Abstract

The embodiments of the present application relate to a method for training a detection model, a target detection method, an apparatus, and a storage medium. The method comprises: generating second laser data based on first laser data; wherein the number of lines of the first laser data is higher than that of the second laser data; training a knowledge distillation student network based on the second laser data, a knowledge distillation teacher network, and a preset loss function to obtain a target student network; wherein the knowledge distillation teacher network is related to the first laser data, and the knowledge distillation teacher network and the knowledge distillation student network have the same structure. In the application scenario of using a low-line-count lidar, this method makes full use of the public large-scale high-line-count dataset, avoids re-collecting the training dataset, and adopts the knowledge distillation method to enable the target student network to learn the parameters in the knowledge distillation teacher network, thereby improving the performance of the target student network.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a detection model training method, target detection method, device and storage medium. Background Art

[0002] As we all know, LiDAR sensors are widely used in mapping, positioning, and environmental perception of autonomous entities (such as robots). High-line-count LiDARs can provide accurate three-dimensional information, but they are also expensive. Therefore, in some lower-cost products, low-line-count LiDARs are usually deployed, which leads to the problem of domain differences caused by the number of lines. However, most of the existing public datasets are collected by high-line-count LiDARs. Therefore, for the application scenarios of low-line-count LiDARs, in order to avoid repeatedly re-collecting the datasets for the application scenarios, how to use high-line-count public datasets to train detection models for low-line-count application scenarios is a technical problem that needs to be solved urgently by those skilled in the art. Summary of the Invention

[0003] Embodiments of the present application provide a detection model training method, target detection method, device and storage medium.

[0004] In a first aspect, an embodiment of the present application provides a method for training a detection model, comprising:

[0005] generating second laser data according to the first laser data; wherein the number of lines of the first laser data is higher than that of the second laser data;

[0006] The knowledge distillation student network is trained based on the second laser data, the knowledge distillation teacher network and a preset loss function to obtain a target student network; wherein the knowledge distillation teacher network and the knowledge distillation student network have the same structure.

[0007] Furthermore, the method further comprises:

[0008] determining a difference between the number of lines of the first laser data and the number of lines of the second laser data;

[0009] If the difference is greater than a preset value, the target student network is used as the knowledge distillation teacher network, and the step of generating the second laser data based on the first laser data is continued until the number of lines of the second laser data is equal to the target number of lines.

[0010] Furthermore, the knowledge distillation teacher network is trained based on the first laser data.

[0011] Furthermore, generating the second laser data according to the first laser data includes:

[0012] determining the equivalent line number of the laser data to be generated relative to the first laser data;

[0013] The first laser data is downsampled according to the equivalent line number to generate second laser data.

[0014] Furthermore, determining the number of equivalent lines of the laser data to be generated relative to the first laser data includes:

[0015] Acquire a first vertical field of view angle corresponding to the first laser data and a second vertical field of view angle corresponding to the laser data to be generated;

[0016] Based on the first vertical field of view angle, the second vertical field of view angle, and the number of lines of the laser data to be generated, an equivalent number of lines of the laser data to be generated relative to the first laser data is determined.

[0017] Furthermore, the first vertical field of view angle is the vertical field of view angle corresponding to the first laser radar that obtains the first laser data; the second vertical field of view angle is the vertical field of view angle corresponding to the second laser radar that obtains the second laser data.

[0018] Further, determining the equivalent number of lines of the laser data to be generated relative to the first laser data based on the first vertical field of view angle, the second vertical field of view angle, and the number of lines of the laser data to be generated includes:

[0019] After determining the ratio of the first vertical field of view angle to the second vertical field of view angle, calculating the product of the ratio and the number of lines of laser data to be generated;

[0020] The product is determined as the isobol number.

[0021] Furthermore, downsampling the first laser data according to the equivalent line number to generate second laser data includes:

[0022] classifying each laser point in the first laser data into a line corresponding to the first laser data to obtain third laser data;

[0023] The number of lines of the third laser data is downsampled to an equivalent number of lines to obtain second laser data.

[0024] Furthermore, classifying each laser point in the first laser data into a line corresponding to the first laser data to obtain third laser data includes:

[0025] A clustering operation is performed on the first laser data based on the zenith angle of each laser point in the first laser data, thereby classifying each laser point into a line corresponding to the first laser data.

[0026] Furthermore, the loss function includes a first loss function for the knowledge distillation student network to learn the label data in the second laser data, and a second loss function for the knowledge distillation student network to learn the knowledge distillation teacher network.

[0027] Furthermore, the parameters of the second loss function include: the number of samples of the second laser data, the number of regions of interest of each sample in the second laser data, the characteristics of each region of interest in the bird's-eye view feature map of each sample of the knowledge distillation teacher network, and the characteristics of each region of interest in the bird's-eye view feature map of each sample of the knowledge distillation student network.

[0028] Furthermore, the training of the knowledge distillation student network based on the second laser data, the knowledge distillation teacher network and a preset loss function to obtain a target student network includes:

[0029] Inputting the second laser data into the knowledge distillation student network, calculating the actual output of the knowledge distillation student network using the first sub-loss function, and obtaining a first loss value of the knowledge distillation student network;

[0030] Calculating features of the regions of interest in the corresponding bird's-eye view feature maps of the knowledge distillation student network and the knowledge distillation teacher network using the second sub-loss function to obtain a second loss value of the knowledge distillation student network;

[0031] The parameters of the knowledge distillation student network are adjusted based on the first loss value and the second loss value until the knowledge distillation student network converges. In a second aspect, an embodiment of the present application provides a target detection method, comprising:

[0032] Acquire fourth laser data; wherein the fourth laser data is multi-line laser data;

[0033] Determining a target object in the fourth laser data using a preset detection model;

[0034] The detection model is obtained using the method provided in the first aspect of the embodiment of the present application.

[0035] In a third aspect, an embodiment of the present application provides a training device for a detection model, comprising:

[0036] A generating module, configured to generate second laser data based on the first laser data; wherein the number of lines of the first laser data is higher than that of the second laser data;

[0037] A training module is used to train a knowledge distillation student network based on the second laser data, a knowledge distillation teacher network and a preset loss function to obtain a target student network; wherein the knowledge distillation teacher network and the knowledge distillation student network have the same structure.

[0038] In a fourth aspect, an embodiment of the present application provides a target detection device, comprising:

[0039] An acquisition module, configured to acquire fourth laser data; wherein the fourth laser data is multi-line laser data;

[0040] a determination module, configured to determine a target object in the fourth laser data using a preset detection model;

[0041] The detection model is obtained using the method provided in the first aspect of the embodiment of the present application.

[0042] In a fifth aspect, an embodiment of the present application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the training method of the detection model provided in the first aspect of the embodiment of the present application and the steps of the target detection method provided in the second aspect.

[0043] In the sixth aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the training method of the detection model provided in the first aspect of the embodiment of the present application and the steps of the target detection method provided in the second aspect.

[0044] In the seventh aspect, a robot is used to: send fourth laser data to a server; receive a target object corresponding to the fourth laser data determined by the target detection method provided by the second aspect and sent by the server, and perform a preset task based on the target object, wherein the preset task includes moving and / or cleaning based on a planned path.

[0045] The technical solution provided in the embodiment of the present application can generate low-line-count second laser data based on high-line-count first laser data, and train the knowledge distillation student network based on the second laser data, the knowledge distillation teacher network and the preset loss function, so as to obtain the target student network. In the application scenario of using low-line-count lidar, it fully utilizes the public large-scale high-line-count data set, avoids re-collection of the training data set, and adopts the knowledge distillation method to enable the target student network to learn the parameters in the knowledge distillation teacher network, thereby improving the performance of the target student network. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A flowchart of a method for training a detection model provided in an embodiment of the present application;

[0047] Figure 2 Another flowchart of the method for training a detection model provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of another flow chart of the method for training a detection model provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of a principle of a method for training a detection model provided in an embodiment of the present application;

[0050] Figure 5 A schematic diagram of a flow chart of a target detection method provided in an embodiment of the present application;

[0051] Figure 6 A schematic diagram of the structure of a training device for a detection model provided in an embodiment of the present application;

[0052] Figure 7 A schematic diagram of the structure of a target detection device provided in an embodiment of the present application;

[0053] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of this application more clear, the technical solutions in the embodiments of this application are further described in detail through the following embodiments and in conjunction with the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain this application and are not used to limit this application.

[0055] It should be noted that the execution subject of the following method embodiment can be a training device for the detection model, which can be implemented as part or all of an electronic device through software, hardware, or a combination of software and hardware. Optionally, the electronic device can be a client, including but not limited to smart phones, tablet computers, e-book readers, and vehicle-mounted terminals. Of course, the electronic device can also be an independent server or server cluster, and the electronic device can also be an autonomous entity (such as various robots, etc.). The embodiments of this application do not limit the specific form of the electronic device. The following method embodiments are described by taking the execution subject as an electronic device as an example.

[0056] Figure 1 A flow chart of a method for training a detection model provided in an embodiment of the present application. This embodiment relates to a specific process of how an electronic device uses high-line-count laser data to train a detection model for processing low-line-count laser data. Figure 1 As shown, the method may include:

[0057] S101 : Generate second laser data according to first laser data.

[0058] Among them, the number of lines of the first laser data is higher than that of the second laser data, and the first laser data and the second laser data are both multi-line laser data, that is, the first laser data is high-line-count laser data in the source domain, which is collected by a high-line-count laser radar and is real laser data, such as 64-line laser data. The number of lines of the second laser data is lower than that of the first laser data, such as 32-line laser data or 16-line laser data.

[0059] Most of the existing public data sets are high-line-count laser data, such as high-line-count autonomous driving data sets. In the embodiments of the present application, the high-line-count laser data can be fully utilized to generate the required low-line-count laser data. The low-line-count laser data here is relative to the first laser data. Specifically, the electronic device can downsample the high-line-count laser data (i.e., the first laser data) in the source domain based on the radar point cloud distillation algorithm to obtain the required low-line-count laser data (i.e., the second laser data) in the target domain. The number of lines of the second laser data can be set based on actual needs. For example, if a low-line-count laser radar, such as a 16-line laser radar, is deployed on the actual product, and if the existing public data set is collected by a 64-line laser radar, then it is very time-consuming and labor-intensive to re-collect the training data set for each generation of products. At this time, the existing 64-line laser data can be used to generate the required 16-line laser data.

[0060] S102: Train the knowledge distillation student network based on the second laser data, the knowledge distillation teacher network, and a preset loss function to obtain a target student network.

[0061] The knowledge distillation teacher network is related to the first laser data, and the knowledge distillation teacher network and the knowledge distillation student network have the same structure.

[0062] The above-mentioned knowledge distillation teacher network and knowledge distillation student network can be deep learning convolutional neural networks, or other network structures. In an embodiment of the present application, the knowledge distillation teacher network and the knowledge distillation student network adopt the same network structure. The above-mentioned knowledge distillation teacher network is trained on relatively high-line laser data. In an optional implementation, the above-mentioned knowledge distillation teacher network is trained based on the first laser data. After obtaining the second laser data, the electronic device can use the second laser data as training data, perform distillation training on the knowledge distillation student network through the trained knowledge distillation teacher network, and use a preset loss function to backpropagate the parameters of the knowledge distillation student network until the knowledge distillation student network converges, thereby obtaining the target student network. Using the above-mentioned knowledge distillation method for training can transfer the rich information in the high-line laser data to the knowledge distillation student network, thereby improving the generalization ability of the target student network.

[0063] The technical solution provided in the embodiment of the present application can generate low-line-count second laser data based on high-line-count first laser data, and train the knowledge distillation student network based on the second laser data, the knowledge distillation teacher network and the preset loss function, so as to obtain the target student network. In the application scenario of using low-line-count lidar, it fully utilizes the public large-scale high-line-count data set, avoids re-collection of the training data set, and adopts the knowledge distillation method to enable the target student network to learn the parameters in the knowledge distillation teacher network, thereby improving the performance of the target student network.

[0064] In practical applications, the difference in the number of lines between the source domain laser data and the required target domain laser data can be small or large. For example, if the source domain laser data (i.e., the first laser data) has 64 lines and the required target domain laser data has 32 lines, the difference in the number of lines between the source and target domains is small. In this case, the knowledge distillation student network can effectively learn from the knowledge distillation teacher network. However, if the required target domain has 16 lines, the difference in the number of lines between the source and target domains is large. In this case, the knowledge distillation student network cannot effectively learn from the knowledge distillation teacher network.

[0065] To this end, in one embodiment, when the difference in the number of lines between the source domain and the target domain is less than a preset value, the detection model can be trained according to the following process, wherein the preset value can be set according to actual needs. Figure 2 As shown, the method may include:

[0066] S201. Train a knowledge distillation teacher network based on the first laser data.

[0067] Among them, the first laser data can be used as training data, the first laser data can be labeled, and then the labeled data can be used to train the knowledge distillation teacher network. Specifically, the loss function of the knowledge distillation teacher network can be used to calculate the loss value between the actual output of the knowledge distillation teacher network and the labeled result, and backpropagation training can be performed according to the loss value until the loss converges to obtain a trained knowledge distillation teacher network.

[0068] S202: Generate second laser data according to the first laser data.

[0069] The number of lines of the first laser data is higher than that of the second laser data.

[0070] S203: Train the knowledge distillation student network based on the second laser data, the knowledge distillation teacher network, and a preset loss function to obtain a target student network.

[0071] Continuing with the example of a source domain with 64 laser lines and a target domain with 32 lines, the electronic device can downsample the 64-line laser data to generate 32-line laser data. The 64-line laser data is then used to train the knowledge distillation teacher network. The knowledge distillation student network is then trained using the 32-line laser data, the trained knowledge distillation teacher network, and a preset loss function to obtain the target student network. This target student network is the detection model used to process the 32-line laser data.

[0072] In one embodiment, when the difference in the number of lines between the source domain and the target domain is greater than a preset value, the target student network can be used as the knowledge distillation teacher network, and the step of generating the second laser data according to the first laser data is continued until the number of lines of the second laser data is equal to the target number of lines. That is, the knowledge distillation student network can be gradually distilled and trained until the target number of lines is reached. Specifically, the detection model can be trained according to the following process, such as Figure 3 As shown, the method may include:

[0073] S301: Train a knowledge distillation teacher network based on the first laser data.

[0074] S302: Generate second laser data according to the first laser data.

[0075] S303: Train the knowledge distillation student network based on the second laser data, the knowledge distillation teacher network, and a preset loss function to obtain a target student network.

[0076] S304: Use the target student network as the knowledge distillation teacher network for training the knowledge distillation student network next time, and set the number of lines of the second laser data to be generated next time.

[0077] Then, continue to execute the above step S302 until the number of lines of the second laser data is equal to the target number of lines, thereby obtaining the final target student network required, that is, the detection model.

[0078] See also Figure 4 Continuing with the example of the case where the number of lines of source domain laser data is 64 and the number of lines of the required target domain is 16, the electronic device can generate 32 lines of laser data from the real 64-line laser data, and then use the 32-line laser data, the knowledge distillation teacher network trained with the 64-line laser data, and the preset loss function to perform distillation training on the knowledge distillation student network, thereby obtaining a target student network for processing 32-line laser data. Furthermore, the electronic device can use the target student network as the knowledge distillation teacher network required for the next training of the knowledge distillation student network, and generate 16 lines of laser data from the 64-line laser data, and use the 16-line laser data, the new knowledge distillation teacher network (i.e., the target student network obtained in the previous step), and the preset loss function to perform distillation training on the knowledge distillation student network, thereby obtaining a target student network for processing 16 lines of laser data.

[0079] It should be noted that, since the 32-line laser data and 16-line laser data generated above are not real laser data, the generated 32-line laser data is referred to as pseudo 32-line laser data, and the generated 16-line laser data is referred to as pseudo 16-line laser data.

[0080] In this embodiment, the difference in the number of lines between the source domain and the target domain can be first determined. If the difference is less than a preset value, the knowledge distillation student network is distilled and learned based on the second laser data and the knowledge distillation teacher network to obtain a target student network. If the difference is greater than the preset value, the knowledge distillation student network is distilled and learned based on the second laser data and the knowledge distillation teacher network to obtain a target student network. The target student network is then further used as the knowledge distillation teacher network to determine the target student network again until the number of lines of the second laser data is equal to the target number of lines, thereby gradually achieving distillation learning of the knowledge distillation student network, so that the knowledge distillation student network can better learn the features in the high-line-number laser data from the knowledge distillation teacher network, thereby further improving the performance of the target student network.

[0081] In one embodiment, optionally, the process of generating the second laser data according to the first laser data may be:

[0082] determining the equivalent line number of the laser data to be generated relative to the first laser data;

[0083] The first laser data is downsampled according to the equivalent line number to generate second laser data.

[0084] Among them, since the vertical field of view angles of laser radars with different line numbers are different, that is, the vertical field of view angles corresponding to the source domain laser data and the laser data to be generated are different, directly downsampling the source domain laser data may cause some deviations between the downsampled laser data and the real laser data with the same line number, and it is impossible to accurately simulate the required real laser data. Therefore, the equivalent line number of the laser data to be generated relative to the first laser data can be calculated first, and then the first laser data can be downsampled based on the equivalent line number to generate the second laser data.

[0085] Optionally, a first vertical field of view angle corresponding to the first laser data and a second vertical field of view angle corresponding to the laser data to be generated can be obtained, and based on the first vertical field of view angle, the second vertical field of view angle and the number of lines of the laser data to be generated, the equivalent number of lines of the laser data to be generated relative to the first laser data can be determined.

[0086] Among them, the first vertical field of view angle is the vertical field of view angle corresponding to the first laser radar that obtains the first laser data; the second vertical field of view angle is the vertical field of view angle corresponding to the second laser radar that obtains the second laser data.

[0087] Preferably, determining the equivalent number of lines of the laser data to be generated relative to the first laser data based on the first vertical field of view angle, the second vertical field of view angle, and the number of lines of the laser data to be generated includes:

[0088] After determining the ratio of the first vertical field of view angle to the second vertical field of view angle, the product of the ratio and the number of lines of the laser data to be generated is calculated; and the product is determined as the equivalent number of lines.

[0089] Specifically, the electronic device may determine the equivalent line number B of the laser data to be generated relative to the first laser data according to the following formula 1 or a variation of the following formula 1: t ′:

[0090] Formula 1:

[0091] Among them, [α s ,β s ] represents the first vertical field angle corresponding to the first laser data, [α t ,β t ] represents the second vertical field angle corresponding to the laser data to be generated, B t Indicates the number of lines of laser data to be generated.

[0092] Furthermore, when downsampling the first laser data, it is necessary to downsample each line in the first laser data. To this end, the electronic device can classify each laser point in the first laser data into a line corresponding to the first laser data, thereby obtaining third laser data, and then downsample the number of lines in the third laser data to an equivalent number of lines to obtain second laser data.

[0093] Optionally, the first laser data may be clustered based on the zenith angle of each laser point in the first laser data, thereby classifying each laser point into a line corresponding to the first laser data. Specifically, the zenith angle θ of each laser point may be calculated according to the following formula 2 or a variation of formula 2:

[0094] Formula 2:

[0095] Wherein, x, y, z are the Cartesian coordinates of each laser point in the first laser data.

[0096] After obtaining the zenith angle of each laser point, a K-Means clustering operation is performed on the zenith angles of all laser points to obtain multiple classes. The center of each class represents the zenith angle of each line of the first laser data, thereby classifying each laser point into the line corresponding to the first laser data. s Line sampling to B t ' line, avoid directly s Line sampling to B t The problem that the generated laser data deviates too much from the actual laser data is caused by the lines.

[0097] In this embodiment, by calculating the equivalent line number of the laser data to be generated relative to the first laser data, the first laser data is downsampled according to the equivalent line number, thereby reducing the deviation between the downsampled laser data and the required real laser data, so that the generated second laser data can more accurately simulate the required real laser data.

[0098] In one embodiment, optionally, the loss function includes a first loss function for the knowledge distillation student network to learn the label data in the second laser data, and a second loss function for the knowledge distillation student network to learn the knowledge distillation teacher network; the parameters of the second loss function include: the number of samples of the second laser data, the number of regions of interest of each sample in the second laser data, the features of each region of interest of the knowledge distillation teacher network in the bird's-eye view feature map of each sample, and the features of each region of interest of the knowledge distillation student network in the bird's-eye view feature map of each sample.

[0099] Among them, the first sub-loss function is used to represent the loss function of the knowledge distillation student network learning the label data in the second laser data, and the second sub-loss function is used to represent the loss function of the knowledge distillation student network learning the knowledge distillation teacher network.

[0100] Specifically, the loss function may be expressed as the following formula 3 or a variation of formula 3:

[0101] Formula 3:

[0102] Among them, L gt is the first sub-loss function, is the second sub-loss function, λ is the weight corresponding to the second sub-loss function, N represents the number of samples of the second laser data, M i represents the number of regions of interest in the i-th sample, represents the features of the jth region of interest in the i-th sample bird's-eye view feature map of the knowledge distillation teacher network, Represents the features of the jth region of interest of the knowledge distillation student network in the bird's-eye view feature map of the i-th sample.

[0103] Further, optionally, the training of the knowledge distillation student network based on the second laser data, the knowledge distillation teacher network and the preset loss function to obtain the target student network includes:

[0104] Inputting the second laser data into the knowledge distillation student network, calculating the actual output of the knowledge distillation student network using the first sub-loss function, and obtaining a first loss value of the knowledge distillation student network;

[0105] Using the second sub-loss function, calculate the features of the region of interest of the knowledge distillation student network and the knowledge distillation teacher network in the corresponding bird's-eye view feature map to obtain a second loss value of the knowledge distillation student network; wherein the bird's-eye view feature map is obtained based on the second laser data;

[0106] Parameters of the knowledge distillation student network are adjusted based on the first loss value and the second loss value until the knowledge distillation student network converges.

[0107] Among them, since the sparse three-dimensional features of the knowledge distillation teacher network and the knowledge distillation student network are located in different positions, it is impossible to simply let the knowledge distillation student network imitate the sparse three-dimensional features of the knowledge distillation teacher network. Therefore, in the embodiment of the present application, the three-dimensional features of the knowledge distillation teacher network and the knowledge distillation student network can be projected onto a two-dimensional bird's-eye view map to obtain a bird's-eye view feature map, and the knowledge distillation student network can learn the features of the knowledge distillation teacher network in the bird's-eye view feature map. In addition, since the bird's-eye view feature map is very high in dimensionality, directly regressing high-dimensional vectors can easily cause the knowledge distillation student network to have difficulty converging. At the same time, considering that there are many low-response areas in the bird's-eye view feature map, the features of low-response areas are often unimportant. Therefore, regions of interest in the bird's-eye view feature map can be extracted, and the knowledge distillation student network can perform learning operations on these regions of interest. In this way, by learning the features of the regions of interest of the knowledge distillation teacher network in the bird's-eye view feature map, the knowledge distillation student network can learn the more important parameters of the knowledge distillation teacher network, which not only accelerates the convergence speed of the target student network, but also further improves the performance of the target student network.

[0108] Figure 5 A flow chart of the target detection method provided in the embodiment of the present application. Figure 5 As shown, the method may include:

[0109] S501: Acquire fourth laser data.

[0110] The fourth laser data is multi-line laser data, that is, the fourth laser data may be data collected by a low-line-count laser radar.

[0111] S502: Determine a target object in the fourth laser data using a preset detection model.

[0112] After obtaining the fourth laser data, the fourth laser data is input into a pre-trained detection model, and the detection model is used to determine the target object in the fourth laser data. For example, the detection model is used to detect whether there is an obstacle in the fourth laser data. The detection model is obtained using the detection model training method described in any of the above embodiments.

[0113] For example, consider a cleaning robot. Considering its cost, a low-line-count LiDAR is deployed on the robot. The aforementioned training method can be used to generate a detection model and deploy it within the robot. During operation, the robot uses the low-line-count LiDAR to collect laser data from its operating environment. This data is then fed into the detection model, which then determines whether obstacles and other target objects are present in the environment, providing a basis for subsequent route planning.

[0114] Since, in the embodiment of the present application, the detection model obtained by the above-mentioned detection model training method is used to detect the target object, the accuracy of the detection result can be improved during the target object detection process.

[0115] Figure 6 A structural diagram of a training device for a detection model provided in an embodiment of the present application. Figure 6 As shown, the apparatus may include: a generating module 601 and a training module 602 .

[0116] The generating module 601 is configured to generate second laser data based on the first laser data; wherein the number of lines of the first laser data is higher than that of the second laser data;

[0117] The training module 602 is used to train the knowledge distillation student network based on the second laser data, the knowledge distillation teacher network and the preset loss function to obtain a target student network; wherein the knowledge distillation teacher network and the knowledge distillation student network have the same structure.

[0118] Based on the above embodiment, optionally, the device further includes: a processing module.

[0119] Specifically, the processing module is used to use the target student network as the knowledge distillation teacher network, and continue to execute the step of generating second laser data based on the first laser data until the number of lines of the second laser data is equal to the target number of lines.

[0120] Optionally, the knowledge distillation teacher network is trained based on the first laser data.

[0121] Based on the above embodiment, optionally, the generating module 601 may include: a determining unit and a down-sampling unit.

[0122] Specifically, the determining unit is used to determine the number of equivalent lines of the laser data to be generated relative to the first laser data;

[0123] The downsampling unit is configured to downsample the first laser data according to the equivalent line number to generate second laser data.

[0124] Based on the above embodiment, optionally, the determination unit is specifically used to obtain a first vertical field of view angle corresponding to the first laser data and a second vertical field of view angle corresponding to the laser data to be generated; based on the first vertical field of view angle, the second vertical field of view angle and the number of lines of the laser data to be generated, determine the equivalent number of lines of the laser data to be generated relative to the first laser data.

[0125] Based on the above embodiment, optionally, the downsampling unit includes: a classification subunit and a downsampling subunit.

[0126] Specifically, the classification subunit is used to classify each laser point in the first laser data into a line corresponding to the first laser data to obtain third laser data;

[0127] The downsampling subunit is configured to downsample the number of lines of the third laser data to an equivalent number of lines to obtain second laser data.

[0128] Based on the above embodiment, optionally, the classification subunit is specifically configured to perform a clustering operation on the first laser data based on the zenith angle of each laser point in the first laser data, thereby classifying each laser point into a line corresponding to the first laser data.

[0129] Optionally, the loss function includes: a first sub-loss function and a second sub-loss function, the first sub-loss function is used to represent the loss function of the knowledge distillation student network learning the label data in the second laser data, and the second sub-loss function is used to represent the loss function of the knowledge distillation student network learning the knowledge distillation teacher network.

[0130] Based on the above embodiment, optionally, the training module 602 is specifically used to input the second laser data into the knowledge distillation student network, use the first sub-loss function to calculate the actual output of the knowledge distillation student network, and obtain the first loss value of the knowledge distillation student network; use the second sub-loss function to calculate the features of the knowledge distillation student network and the knowledge distillation teacher network in the corresponding bird's-eye view feature map of the area of ​​interest to obtain the second loss value of the knowledge distillation student network; adjust the parameters of the knowledge distillation student network based on the first loss value and the second loss value until the knowledge distillation student network converges.

[0131] Figure 7 A schematic diagram of the structure of the target detection device provided in the embodiment of the present application. Figure 7 As shown, the device may include: an acquisition module 701 and a determination module 702.

[0132] Specifically, the acquisition module 701 is used to acquire fourth laser data; wherein the fourth laser data is multi-line laser data;

[0133] The determination module 702 is configured to determine a target object in the fourth laser data using a preset detection model;

[0134] The detection model is obtained by using the detection model training method described in any of the above embodiments.

[0135] In one embodiment, an electronic device is provided, whose internal structure diagram can be as follows: Figure 8As shown. The electronic device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal via a network connection. The database of the electronic device is used to store data involved in the detection model training process and the target detection process. When the computer program is executed by the processor, a detection model training method or a target detection method is implemented.

[0136] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0137] In one embodiment, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0138] generating second laser data according to the first laser data; wherein the number of lines of the first laser data is higher than that of the second laser data;

[0139] The knowledge distillation student network is trained based on the second laser data, the knowledge distillation teacher network and a preset loss function to obtain a target student network; wherein the knowledge distillation teacher network and the knowledge distillation student network have the same structure.

[0140] In one embodiment, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0141] Acquire fourth laser data; wherein the fourth laser data is multi-line laser data;

[0142] Determining a target object in the fourth laser data using a preset detection model;

[0143] The detection model is obtained using the training method of the detection model provided in any of the above embodiments.

[0144] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0145] generating second laser data according to the first laser data; wherein the number of lines of the first laser data is higher than that of the second laser data;

[0146] The knowledge distillation student network is trained based on the second laser data, the knowledge distillation teacher network and a preset loss function to obtain a target student network; wherein the knowledge distillation teacher network and the knowledge distillation student network have the same structure.

[0147] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0148] Acquire fourth laser data; wherein the fourth laser data is multi-line laser data;

[0149] Determining a target object in the fourth laser data using a preset detection model;

[0150] The detection model is obtained using the training method of the detection model provided in any of the above embodiments.

[0151] In one embodiment, a robot is provided, which can be used to: send fourth laser data to a server; receive a target object corresponding to the fourth laser data determined based on the aforementioned target detection method and sent by the server, and perform a preset task based on the target object, the preset task including moving and / or cleaning based on a planned path.

[0152] The training device, equipment and storage medium of the detection model provided in the above embodiments can execute the various processes of the training method of the detection model provided in any embodiment of the present application and have corresponding beneficial effects, and the target detection device, equipment, storage medium and robot provided in the above embodiments can execute the various processes of the target detection method provided in any embodiment of the present application and have corresponding beneficial effects. To avoid repetition, they will not be repeated here.

[0153] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0154] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0155] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A method for training a detection model, characterized in that: include: Acquire a first vertical field of view angle corresponding to the first laser data and a second vertical field of view angle corresponding to the laser data to be generated; Determining an equivalent number of lines of the laser data to be generated relative to the first laser data based on the first vertical field of view angle, the second vertical field of view angle, and the number of lines of the laser data to be generated; downsampling the first laser data according to the equivalent number of lines to generate second laser data; wherein the number of lines of the first laser data is higher than that of the second laser data; The knowledge distillation student network is trained based on the second laser data, the knowledge distillation teacher network and the preset loss function to obtain a target student network; wherein the knowledge distillation teacher network and the knowledge distillation student network have the same structure, and the knowledge distillation teacher network is trained based on the first laser data.

2. The method according to claim 1, characterized in that Also includes: determining a difference between the number of lines of the first laser data and the number of lines of the second laser data; If the difference is greater than a preset value, the target student network is used as the knowledge distillation teacher network, and the step of generating second laser data based on the first laser data is continued until the number of lines of the second laser data is equal to the target number of lines.

3. The method according to claim 1, characterized in that The first vertical field of view angle is the vertical field of view angle corresponding to the first laser radar that obtains the first laser data; the second vertical field of view angle is the vertical field of view angle corresponding to the second laser radar that obtains the second laser data.

4. The method according to claim 1, wherein Determining an equivalent number of lines of the laser data to be generated relative to the first laser data based on the first vertical field of view angle, the second vertical field of view angle, and the number of lines of the laser data to be generated includes: After determining the ratio of the first vertical field of view angle to the second vertical field of view angle, calculating the product of the ratio and the number of lines of laser data to be generated; The product is determined as the isobol number.

5. The method according to claim 1, wherein Downsampling the first laser data according to the equivalent line number to generate second laser data includes: classifying each laser point in the first laser data into a line corresponding to the first laser data to obtain third laser data; The number of lines of the third laser data is downsampled to an equivalent number of lines to obtain second laser data.

6. The method according to claim 4, characterized in that The classifying each laser point in the first laser data into a line corresponding to the first laser data to obtain third laser data includes: A clustering operation is performed on the first laser data based on the zenith angle of each laser point in the first laser data, thereby classifying each laser point into a line corresponding to the first laser data.

7. The method according to any one of claims 1 to 6, characterized in that The loss function includes a first loss function for the knowledge distillation student network to learn the label data in the second laser data, and a second loss function for the knowledge distillation student network to learn the knowledge distillation teacher network.

8. The method according to claim 7, characterized in that The parameters of the second loss function include: the number of samples of the second laser data, the number of regions of interest of each sample in the second laser data, the characteristics of each region of interest in the bird's-eye view feature map of each sample of the knowledge distillation teacher network, and the characteristics of each region of interest in the bird's-eye view feature map of each sample of the knowledge distillation student network.

9. The method according to claim 8, characterized in that The step of training the knowledge distillation student network based on the second laser data, the knowledge distillation teacher network, and a preset loss function to obtain a target student network includes: Inputting the second laser data into the knowledge distillation student network, calculating the actual output of the knowledge distillation student network using the first sub-loss function, and obtaining a first loss value of the knowledge distillation student network; Calculating features of the knowledge distillation student network and the knowledge distillation teacher network in the corresponding bird's-eye view feature map in the region of interest using a second sub-loss function to obtain a second loss value of the knowledge distillation student network; Parameters of the knowledge distillation student network are adjusted based on the first loss value and the second loss value until the knowledge distillation student network converges.

10. A target detection method, characterized in that: include: Acquire fourth laser data; wherein the fourth laser data is multi-line laser data; Determining a target object in the fourth laser data using a preset detection model; Wherein, the detection model is obtained using the method according to any one of claims 1 to 9.

11. A training device for a detection model, characterized in that: include: A generating module, configured to obtain a first vertical field of view angle corresponding to the first laser data and a second vertical field of view angle corresponding to the laser data to be generated; Determining an equivalent number of lines of the laser data to be generated relative to the first laser data based on the first vertical field of view angle, the second vertical field of view angle, and the number of lines of the laser data to be generated; downsampling the first laser data according to the equivalent number of lines to generate second laser data; wherein the number of lines of the first laser data is higher than that of the second laser data; A training module is used to train a knowledge distillation student network based on the second laser data, a knowledge distillation teacher network and a preset loss function to obtain a target student network; wherein the knowledge distillation teacher network and the knowledge distillation student network have the same structure.

12. A target detection device, characterized in that: include: An acquisition module, configured to acquire fourth laser data; wherein the fourth laser data is multi-line laser data; a determination module, configured to determine a target object in the fourth laser data using a preset detection model; Wherein, the detection model is obtained using the method according to any one of claims 1 to 9.

13. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

15. A robot, characterized in that: Used for: sending the fourth laser data to the server; Receive the target object corresponding to the fourth laser data determined by the target detection method according to claim 10 and sent by the server, and perform a preset task based on the target object, wherein the preset task includes traveling and / or cleaning based on the planned path.

Images