A target detection method, device, equipment and computer readable storage medium

Abstract

Embodiments of the present application disclose a target detection method, device and equipment and a computer readable storage medium. A teacher network model and a student network model are used to perform feature extraction on a sample picture in parallel, to obtain first category features corresponding to the teacher network model and second category features corresponding to the student network model. The first category features and the second category features are converted into same-dimension feature information. Based on the same-dimension feature information and a category label corresponding to the sample picture, parameters of the student network model are iteratively adjusted to obtain a student network model meeting a recognition requirement. Through training of the teacher network model and the student network model together, the detection accuracy of the student network model can be greatly improved. Moreover, the student network model has a simple architecture and a fast detection speed, so that a new input picture can be analyzed by using the student network model meeting the recognition requirement, and the target category contained in the picture can be quickly and accurately determined.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a target detection method and device, equipment and a computer readable storage medium. BACKGROUND

[0002] As a basic application in computer vision, target detection technology based on artificial intelligence plays a very important role in modern industrial production, especially in the fields of security, autonomous driving, Internet of Things and the like.

[0003] Therefore, people's requirements for target detection technology are also getting higher and higher, and at the same time of high precision, it should also have faster detection speed. But detection precision and detection speed are often mutually exclusive indicators, high detection precision requires large detection model, but large model means the decline of detection speed. The traditional target detection method often ensures the high precision or high detection speed of target detection according to the emphasis of actual demand, and cannot realize the balance of detection precision and detection speed.

[0004] It can be seen that how to improve the detection speed while ensuring the high precision of target detection is a problem to be solved by those skilled in the art. SUMMARY

[0005] The purpose of the embodiments of the present application is to provide a target detection method, device, equipment and computer readable storage medium, which can improve the detection speed while ensuring the high precision of target detection.

[0006] To solve the above technical problems, the embodiments of the present application provide a target detection method, comprising:

[0007] The teacher network model and the student network model are used to perform feature extraction on the sample picture in parallel to obtain first category features corresponding to the teacher network model and second category features corresponding to the student network model;

[0008] The first category features and the second category features are converted into same-dimension feature information;

[0009] Based on the same-dimension feature information and the category label corresponding to the sample picture, the parameters of the student network model are iteratively adjusted to obtain a student network model meeting the recognition requirement;

[0010] The student network model meeting the recognition requirement is used to analyze a newly input picture to determine the target category contained in the picture.

[0011] Optionally, for the construction process of the teacher network model and the student network model, the method comprises:

[0012] A teacher network model for target type identification is constructed based on a YOLO network architecture;

[0013] The teacher network model is pruned to obtain a student network model; wherein the teacher network model and the student network model both include a first module for extracting picture features, a second module for feature mining, and a third module for separating classification features.

[0014] Optionally, the feature extraction of the sample picture by the teacher network model in parallel to obtain the first category feature corresponding to the teacher network model comprises:

[0015] The first initial feature information is obtained by using the first module of the teacher network model to extract features from the sample picture;

[0016] The first deep feature information is obtained by inputting the first initial feature information into the second module of the teacher network model for convolution processing;

[0017] The first target feature information matched with the classification category is extracted from the first deep feature information by using the third module of the teacher network model; wherein the first initial feature information, the first deep feature information and the first target feature information constitute the first category feature.

[0018] Optionally, the feature extraction of the sample picture by the student network model in parallel to obtain the second category feature corresponding to the student network model comprises:

[0019] The second initial feature information is obtained by using the first module of the student network model to extract features from the sample picture;

[0020] The second deep feature information is obtained by inputting the second initial feature information into the second module of the student network model for convolution processing;

[0021] The second target feature information matched with the classification category is extracted from the second deep feature information by using the third module of the student network model; wherein the second initial feature information, the second deep feature information and the second target feature information constitute the second category feature.

[0022] Optionally, the parameter of the student network model is iteratively adjusted based on the same dimension feature information and the category label corresponding to the sample picture to obtain a student network model meeting the identification requirement, comprising:

[0023] According to the same dimension feature information and the category label corresponding to the sample picture, a loss function is determined;

[0024] adjust parameters of the student network model and the teacher network model based on the loss function;

[0025] The step of returning to the step of performing feature extraction on the sample picture in parallel by the teacher network model and the student network model to obtain the first category feature corresponding to the teacher network model and the second category feature corresponding to the student network model until the recognition accuracy and the recognition efficiency of the student network model meet the set condition, and then obtaining the student network model meeting the recognition requirement.

[0026] Optionally, the loss function is determined according to the same dimension feature information and the category label corresponding to the sample picture, and includes:

[0027] The first loss function is determined according to the converted first initial feature information and the second initial feature information.

[0028] The second loss function is determined according to the converted first deep feature information and the second deep feature information.

[0029] The third loss function is determined according to the converted first target feature information and the second target feature information, and the category label corresponding to the sample picture.

[0030] Optionally, the first loss function L fea , the second loss function L fea and the third loss function W(s, t) are calculated according to the following formula:

[0031]

[0032] wherein,

[0033] N represents the total number of sample pictures, M represents a mask matrix composed of a background and a detection target, s represents the output of the student network model, t represents the output of the teacher network model, W(s, t) represents a difference function of the student network model and the teacher network model, and α and β represent parameters of the detection target and the background; ω = 10 and ε = 2.

[0034] Optionally, the third module of the student network model extracts the second target feature information matched with the classification category from the second deep feature information includes:

[0035] The second deep feature information after convolution processing is regressed with the previously calibrated category label to adjust the parameters of the third module.

[0036] The second target feature information matched with the classification category is separated from the deep feature information after convolution processing based on the adjusted parameters and the channel corresponding to the classification category.

[0037] The embodiment of the present application also provides a target detection device, comprising an extraction unit, a conversion unit, an adjustment unit and a determination unit.

[0038] The extraction unit is configured to perform feature extraction on sample pictures in parallel by using a teacher network model and a student network model, to obtain first category features corresponding to the teacher network model and second category features corresponding to the student network model.

[0039] The conversion unit is configured to convert the first category features and the second category features into same-dimension feature information.

[0040] The adjustment unit is configured to perform iterative adjustment on parameters of the student network model based on the same-dimension feature information and a category label corresponding to the sample pictures, to obtain a student network model meeting an identification requirement.

[0041] The determination unit is configured to analyze a newly input picture by using the student network model meeting the identification requirement, to determine a target category contained in the picture.

[0042] Optionally, for a construction process of the teacher network model and the student network model, the device comprises a construction unit and a pruning unit.

[0043] The construction unit is configured to construct a teacher network model for target type identification based on a YOLO network architecture.

[0044] The pruning unit is configured to prune the teacher network model to obtain a student network model; wherein the teacher network model and the student network model both comprise a first module for extracting picture features, a second module for feature mining and a third module for separating out classification features.

[0045] Optionally, the extraction unit is configured to perform feature extraction on the sample pictures by using the first module of the teacher network model, to obtain first initial feature information; input the first initial feature information into the second module of the teacher network model for convolution processing, to obtain first deep feature information; extract first target feature information matched with a classification category from the first deep feature information by using the third module of the teacher network model; wherein the first initial feature information, the first deep feature information and the first target feature information constitute the first category features.

[0046] Optionally, the extraction unit is configured to perform feature extraction on the sample picture by using a first module of the student network model to obtain second initial feature information, perform convolution processing on the second initial feature information by using a second module of the student network model to obtain second deep feature information, and extract second target feature information matching a classification category from the second deep feature information by using a third module of the student network model, wherein the second initial feature information, the second deep feature information, and the second target feature information constitute the second category feature.

[0047] Optionally, the adjustment unit comprises a determination subunit, a parameter adjustment subunit, and an obtaining subunit.

[0048] The determination subunit is configured to determine a loss function according to the same-dimension feature information and a category label corresponding to the sample picture.

[0049] The parameter adjustment subunit is configured to adjust parameters of the student network model and the teacher network model based on the loss function, and return the extraction unit to perform the step of performing feature extraction on the sample picture in parallel by using the teacher network model and the student network model to obtain the first category feature corresponding to the teacher network model and the second category feature corresponding to the student network model.

[0050] The obtaining subunit is configured to obtain the student network model meeting the recognition requirement until the recognition accuracy and the recognition efficiency of the student network model meet a set condition.

[0051] Optionally, the determination subunit is configured to determine a first loss function according to the first initial feature information and the second initial feature information converted into the same dimension, determine a second loss function according to the first deep feature information and the second deep feature information converted into the same dimension, and determine a third loss function according to the first target feature information and the second target feature information converted into the same dimension and a category label corresponding to the sample picture.

[0052] Optionally, the determination subunit is configured to calculate the first loss function L fea , the second loss function L fea , and the third loss function W(s, t) according to the following formulas.

[0053]

[0054] wherein,

[0055] N represents the total number of sample pictures, M represents a mask matrix composed of a background and a detection target, s represents the output of the student network model, t represents the output of the teacher network model, W(s, t) represents a difference function of the student network model and the teacher network model, and a and β represent parameters of the detection target and the background; ω=10 and ε=2.

[0056] Optionally, the extraction unit comprises a regression subunit and a separation subunit.

[0057] The regression subunit is configured to regress the second deep feature information after convolution processing with a pre-labeled class label to adjust parameters of the third module.

[0058] The separation subunit is configured to separate the second target feature information matching the classification category from the deep feature information after convolution processing based on the adjusted parameters and the channel corresponding to the classification category.

[0059] Embodiments of the present application also provide an electronic device comprising:

[0060] A memory configured to store a computer program;

[0061] A processor configured to execute the computer program to implement the steps of the target detection method.

[0062] Embodiments of the present application also provide a computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of the target detection method.

[0063] As can be seen from the above technical solutions, the teacher network model and the student network model are used to perform feature extraction on sample pictures in parallel, to obtain first category features corresponding to the teacher network model and second category features corresponding to the student network model. The first category features and the second category features are both feature information related to a detection target. In order to realize collaborative training of the teacher network model and the student network model, the first category features and the second category features can be converted into same-dimension feature information; based on the same-dimension feature information and a class label corresponding to the sample pictures, parameters of the student network model are iteratively adjusted to obtain a student network model meeting recognition requirements. The model architecture of the teacher network model is more complex than that of the student network model, and the teacher network model can realize higher-precision detection but slower detection speed. By training the teacher network model and the student network model together, the detection precision of the student network model can be greatly improved, and because the architecture of the student network model is simple, the student network model has faster detection speed, so that a new input picture can be analyzed by using the student network model meeting recognition requirements, to quickly and accurately determine the target category contained in the picture, thereby ensuring high-precision target detection while improving detection speed. Attached Figure Description

[0064] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 A flowchart of a target detection method provided in an embodiment of this application;

[0066] Figure 2 An architecture diagram of a teacher network model and a student network model provided for embodiments of this application;

[0067] Figure 3 A schematic diagram of the structure of a head module provided in an embodiment of this application;

[0068] Figure 4 This is a schematic diagram of the structure of a target detection device provided in an embodiment of this application;

[0069] Figure 5 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0070] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0071] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may include steps or units not listed.

[0072] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0073] Next, a target detection method provided by the embodiments of this application will be described in detail. Figure 1 A flowchart of a target detection method provided in this application embodiment, the method including:

[0074] S101: Perform feature extraction on the sample picture in parallel using the teacher network model and the student network model to obtain first category features corresponding to the teacher network model and second category features corresponding to the student network model.

[0075] Considering that in the prior art, the single-stage YOLO series framework, especially YOLOv5, is one of the optimal solutions balancing detection accuracy and detection speed, in the embodiments of the present application, a teacher network model for target type recognition can be constructed based on the yolo network architecture.

[0076] For a network model, backbone, a main neural network in a detection model, can be set to extract various features of an input picture, including edge features, texture features, context features, etc. neck can be set to further fuse the features extracted by backbone to realize deep mining of the features, thereby improving network performance, such as a feature pyramid network.

[0077] In the embodiments of the present application, in order to realize accurate detection of a target, head can be set to map the features obtained in neck to features more suitable for target detection, so as to determine the position and category of the detected target based on the features.

[0078] However, due to the particularity of the structure of YOLOv5, the current distillation of YOLO cannot simultaneously support the three modules of backbone, neck and head.

[0079] Therefore, in the embodiments of the present application, when a teacher network model for target type recognition is constructed based on the yolo network architecture, a first module for extracting picture features, a second module for feature mining and a third module for separating classification features can be set in the teacher network model.

[0080] The student network model is obtained by pruning the teacher network model, wherein the student network model also includes the first module for extracting picture features, the second module for feature mining and the third module for separating classification features.

[0081] The method of pruning the teacher network model can be to halve the channel number of all convolution modules in the teacher network model, and the pruned network model is the student network model. Due to the influence of disk compression and other factors during saving, based on actual application analysis, the model size of the student network model obtained by pruning is 2 / 5 of the teacher network.

[0082] In actual application, the first module can be a backbone module for extracting picture features, the second module can be a neck module for feature mining, and the third module can be a head module for separating classification features.

[0083] The sample picture is input into the teacher network model and the student network model respectively, and features related to the target to be detected can be extracted respectively. In order to facilitate distinction, the features extracted by the teacher network model can be referred to as first category features, and the features extracted by the student network model can be referred to as second category features.

[0084] S102: Convert the first category features and the second category features into same-dimension feature information.

[0085] Since the student network model is a model obtained by pruning the teacher network model, the feature dimension output by the student network model is different from the feature dimension output by the teacher network model. In order to realize collaborative training of the student network model and the teacher network model, the first category features and the second category features can be converted into same-dimension feature information.

[0086] In a specific implementation, an FAT (feature adapt) module can be set to realize dimension conversion of the second category features, so that the second category features are converted into data of the same dimension as the first category features. The FAT module can include a convolution layer with a convolution kernel size of 3*3 and a convolution layer with a convolution kernel size of 1*1, and the convolution layer with a convolution kernel size of 1*1 does not need a bias term.

[0087] S103: Based on the same-dimension feature information and the category label corresponding to the sample picture, iteratively adjust the parameters of the student network model to obtain a student network model meeting the recognition requirement.

[0088] The category of the target contained in each sample picture is known information. In order to facilitate model recognition, the sample image can be labeled in advance before training, that is, a category label is set for the target category contained in the sample image.

[0089] In different application scenarios, the sample pictures used for model training contain different targets to be detected. The target to be detected can be one or multiple. Taking the application scenario of face clock-in for work as an example, each user realizes clock-in for work by face recognition, and a clock-in device recognizes the face of the user to determine the identity of the user. At this time, the target to be detected is the face.

[0090] In the embodiments of the present application, in order to realize the balance between high accuracy and high detection speed of target detection, the recognition requirement set for the student network model can include a requirement for recognition accuracy and a requirement for recognition efficiency.

[0091] In actual application, the two network models, the teacher network model and the student network model, can determine the target category possibly contained in the sample picture according to the first target feature information and the second target feature information, compare the target category with the category label corresponding to the sample picture, determine the recognition error of the network model, and thus the network model can be parameter-optimized.

[0092] It should be noted that in the embodiments of the present application, the student network model is used to analyze new pictures after the training of the model is completed, and thus in the above introduction, only the parameter of the student network model is iteratively adjusted to obtain the student network model meeting the recognition requirement. However, in actual application, the network model parameter often needs to be adjusted for multiple times to reach a relatively optimal state, and when the parameter is adjusted, the parameters of the teacher network model and the student network model are adjusted at the same time.

[0093] S104: Analyzing a newly-input picture by using the student network model meeting the recognition requirement to determine the target category contained in the picture.

[0094] When it is needed to analyze which targets are contained in a new picture, the new picture can be input into the trained student network model, and by the analysis of the student network model, it can be output which target categories are contained in the newly-input picture.

[0095] In the embodiments of the present application, when the network model is trained, the position information corresponding to each target category can be set in addition to the category label of the sample picture, so that the trained student network model can determine the position information corresponding to each target in the picture in addition to analyzing the target category contained in the picture. The position information can be the vertex coordinates of the target edge box.

[0096] It can be seen from the technical solution that the teacher network model and the student network model are used to perform feature extraction on the sample picture in parallel, and the first category feature corresponding to the teacher network model and the second category feature corresponding to the student network model are obtained. The first category feature and the second category feature are both feature information related to the target to be detected. In order to realize the collaborative training of the teacher network model and the student network model, the first category feature and the second category feature can be converted into same-dimension feature information; based on the same-dimension feature information and the category label corresponding to the sample picture, the parameters of the student network model are iteratively adjusted to obtain a student network model meeting the recognition requirement. The model architecture of the teacher network model is more complex than that of the student network model, and the teacher network model can realize high-precision detection, but the detection speed is slow. By training the teacher network model and the student network model together, the detection accuracy of the student network model can be greatly improved, and because the architecture of the student network model is simple, the detection speed is fast, so the student network model meeting the recognition requirement is used to analyze the new input picture, and the target category contained in the picture can be quickly and accurately determined, which ensures high-precision target detection and improves the detection speed.

[0097] As can be known from the above introduction, the teacher network model and the student network model can include three modules, and each module can extract feature information during feature extraction.

[0098] Taking the teacher network model as an example, in specific implementation, the first module of the teacher network model can be used to perform feature extraction on the sample picture to obtain first initial feature information; the first initial feature information is input into the second module of the teacher network model for convolution processing to obtain first deep feature information; the third module of the teacher network model is used to extract first target feature information matched with the classification category from the first deep feature information; wherein the first initial feature information, the first deep feature information and the first target feature information constitute the first category feature.

[0099] Taking the student network model as an example, in specific implementation, the first module of the student network model can be used to perform feature extraction on the sample picture to obtain second initial feature information; the second initial feature information is input into the second module of the student network model for convolution processing to obtain second deep feature information; the third module of the student network model is used to extract second target feature information matched with the classification category from the second deep feature information; wherein the second initial feature information, the second deep feature information and the second target feature information constitute the second category feature.

[0100] Considering that the first module contains multiple convolutional modules, to improve the distillation efficiency of the first module while ensuring the accuracy of feature extraction, we can utilize only some of the convolutional modules in the first module for feature extraction. Based on extensive experimental results, convolutional modules 3, 5, and 7 can be selected for feature extraction.

[0101] After passing through the backbone, the sample images enter the neck section, which consists of three branches. During distillation, the first convolutional layer of each of the three branches is selected as the teacher-student distillation module, and the dimensions are balanced using a FAT module similar to that in the backbone section.

[0102] Figure 2 This application provides an architecture diagram of a teacher network model and a student network model, as shown in the embodiments. Figure 2 In this code, "Images" represents sample images, "Teacher" represents the teacher network model, and "Student" represents the student network model. Since the student and teacher network models output feature information with different dimensions, the student network model can undergo dimensionality transformation using the FAT (Feature Adaptation) module. This transformation converts the second-category features into feature information of the same dimension as the first-category features. Each module calculates a loss function based on the same-dimensional feature information. The backbone and neck modules calculate their loss functions similarly, which can be represented by Mask-wing loss. Wingloss represents the loss function of the head module.

[0103] Taking the student network model as an example, the input sample image, after passing through the backbone and neck parts, yields three sets of feature maps, with the feature map sizes becoming 1 / 8, 1 / 16, and 1 / 32 times the size of the input image. At this point, the feature maps undergo 3x3 and 1x1 convolutions in the head module for the final feature extraction step, which includes two branches. One branch (labels) regresses the second deep feature information after convolution with pre-labeled class labels to adjust the parameters of the third module. The other branch (mask), based on the adjusted parameters and the channels corresponding to the classification class, separates the second target feature information matching the classification class from the deep feature information after convolution. The second target feature information is often presented in matrix form, and the matrix composed of the second target feature information can be used as the target feature map.

[0104] like Figure 3 The diagram shown is a structural schematic of a head module provided in an embodiment of this application. Figure 3The mask module separates the target feature map used for classification from the output feature map and uses the target feature map obtained for knowledge distillation.

[0105] Taking a target detection task including 20 classification categories as an example, after the pictures collected by a camera of 1080P specification are subjected to 1x1 convolution of the backbone and neck parts, the dimension of the branch with a smaller feature map is 1x75x60x34, wherein 1 represents the number of pictures, 60x34 represents the width and height of the feature map, and 75 represents the number of channels. The 75 channels can be divided into three groups, each group including 25 channels, wherein 20 channels correspond to 20 categories respectively, and in a specific implementation, the channels used for extracting classification features can be the 6th to 25th channels, the 31st to 50th channels, and the 56th to 75th channels. The mask module generates a 0-1 rectangular matrix by using the positions of the channels, and then multiplies the generated mask matrix with the feature map to obtain the final target feature map used for distillation, which has a size of 1x60x60x34.

[0106] In the embodiments of the present application, the performance of the network model can be evaluated by using the loss function, so as to optimize the parameters of the network model.

[0107] In a specific implementation, the loss function can be determined according to the same-dimension feature information and the category label corresponding to the sample picture; the parameters of the student network model and the teacher network model are adjusted based on the loss function; and the steps of performing feature extraction on the sample picture in parallel by using the teacher network model and the student network model to obtain the first category feature corresponding to the teacher network model and the second category feature corresponding to the student network model are returned until the recognition accuracy and efficiency of the student network model meet the set conditions, and then a student network model meeting the recognition requirements is obtained.

[0108] The student network model and the teacher network model each include three modules, and each module can extract feature information. In actual application, the loss function corresponding to each module can be calculated.

[0109] Taking the first module as an example, the first loss function can be determined according to the first initial feature information and the second initial feature information converted to the same dimension. Taking the second module as an example, the second loss function can be determined according to the first deep feature information and the second deep feature information converted to the same dimension. Taking the third module as an example, the third loss function can be determined according to the first target feature information and the second target feature information converted to the same dimension and the category label corresponding to the sample picture.

[0110] In a specific implementation, the first loss function L fea , the second loss function L feaand a third loss function W(s, t),

[0111]

[0112] wherein,

[0113] N represents the total number of sample pictures, M represents a mask matrix composed of a background and a detection target, s represents the output of the student network model, t represents the output of the teacher network model, W(s, t) represents a difference function of the student network model and the teacher network model, and alpha and beta represent parameters of the detection target and the background; omega = 10, and epsilon = 2.

[0114] Taking the neck part as an example, it is assumed that the loss generated by the background in the feature map is 1.4 times that of the target region, so alpha = 0.71 and beta = 1.0 can be set. Taking the backbone part as an example, it is assumed that the loss generated by the background is 0.5 times that of the target region, so alpha = 2.0 and beta = 1.0 can be set.

[0115] The application realizes a distillation method based on the whole process of YOLOv5 by adding a mask-conv module in the head part. Different strategies and loss functions are used in the backbone, neck and head parts of the network, which improves the detection accuracy of the student model, solves the contradiction between high detection accuracy and faster detection speed in engineering, and enables the neural network to obtain faster detection speed on the premise of ensuring effective indicators.

[0116] Figure 4 A structure diagram of a target detection device provided by the embodiment of the application, comprising an extraction unit 41, a conversion unit 42, an adjustment unit 43 and a determination unit 44.

[0117] The extraction unit 41 is configured to perform feature extraction on sample pictures in parallel by using a teacher network model and a student network model to obtain first-class features corresponding to the teacher network model and second-class features corresponding to the student network model.

[0118] The conversion unit 42 is configured to convert the first-class features and the second-class features into same-dimension feature information.

[0119] The adjustment unit 43 is configured to iteratively adjust parameters of the student network model based on the same-dimension feature information and a class label corresponding to the sample picture, so as to obtain a student network model meeting an identification requirement.

[0120] The determination unit 44 is configured to analyze a newly input picture by using the student network model meeting the identification requirement, so as to determine a target class contained in the picture.

[0121] Optionally, the device comprises a building unit and a pruning unit for the building process of the teacher network model and the student network model.

[0122] The building unit is configured to build a teacher network model for target type identification based on a YOLO network architecture.

[0123] The pruning unit is configured to prune the teacher network model to obtain a student network model; wherein the teacher network model and the student network model both comprise a first module for extracting picture features, a second module for feature mining, and a third module for separating classification features.

[0124] Optionally, the extraction unit is configured to extract features of the sample picture by using the first module of the teacher network model to obtain first initial feature information; input the first initial feature information into the second module of the teacher network model for convolution processing to obtain first deep feature information; and extract first target feature information matched with the classification category from the first deep feature information by using the third module of the teacher network model; wherein the first initial feature information, the first deep feature information, and the first target feature information constitute first category features.

[0125] Optionally, the extraction unit is configured to extract features of the sample picture by using the first module of the student network model to obtain second initial feature information; input the second initial feature information into the second module of the student network model for convolution processing to obtain second deep feature information; and extract second target feature information matched with the classification category from the second deep feature information by using the third module of the student network model; wherein the second initial feature information, the second deep feature information, and the second target feature information constitute second category features.

[0126] Optionally, the adjustment unit comprises a determination subunit, a parameter adjustment subunit, and an obtaining subunit.

[0127] The determination subunit is configured to determine a loss function according to the same-dimension feature information and the category label corresponding to the sample picture.

[0128] The parameter adjustment subunit is configured to adjust parameters of the student network model and the teacher network model based on the loss function; and return the extraction unit to perform the step of extracting features of the sample picture in parallel by using the teacher network model and the student network model to obtain the first category features corresponding to the teacher network model and the second category features corresponding to the student network model.

[0129] The obtaining subunit is configured to obtain the student network model meeting the identification requirements until the recognition accuracy and the recognition efficiency of the student network model meet the set conditions.

[0130] Optionally, the determining subunit is configured to determine a first loss function according to the first initial feature information and the second initial feature information converted to the same dimension; determine a second loss function according to the first deep feature information and the second deep feature information converted to the same dimension; and determine a third loss function according to the first target feature information and the second target feature information converted to the same dimension and the category label corresponding to the sample picture.

[0131] Optionally, the determining subunit is configured to calculate the first loss function L fea , the second loss function L fea and the third loss function W(s, t) according to the following formulas.

[0132]

[0133] wherein,

[0134] N represents the total number of sample pictures, M represents a mask matrix composed of a background and a detection target, s represents the output of the student network model, t represents the output of the teacher network model, W(s, t) represents a difference function of the student network model and the teacher network model, and a and β represent parameters of the detection target and the background; ω = 10 and ε = 2.

[0135] Optionally, the extracting unit comprises a regression subunit and a separating subunit.

[0136] The regression subunit is configured to regress the second deep feature information subjected to the convolution processing with the category label calibrated in advance, so as to adjust the parameters of the third module.

[0137] The separating subunit is configured to separate the second target feature information matched with the classification category from the deep feature information subjected to the convolution processing, based on the adjusted parameters and the channel corresponding to the classification category.

[0138] Figure 4 For the description of the features in the corresponding embodiments, please refer to the related description of the corresponding embodiments, which will not be repeated here. Figure 1 For the description of the features in the corresponding embodiments, please refer to the related description of the corresponding embodiments, which will not be repeated here.

[0139] It can be seen from the technical solution that the teacher network model and the student network model are used to perform feature extraction on the sample picture in parallel, and the first type of feature corresponding to the teacher network model and the second type of feature corresponding to the student network model are obtained. The first type of feature and the second type of feature are both feature information related to the target to be detected. In order to realize the collaborative training of the teacher network model and the student network model, the first type of feature and the second type of feature can be converted into same-dimension feature information; based on the same-dimension feature information and the class label corresponding to the sample picture, the parameters of the student network model are iteratively adjusted to obtain the student network model meeting the recognition requirement. The model architecture of the teacher network model is more complex than that of the student network model, the teacher network model can realize high-precision detection, but the detection speed is slow. By training the teacher network model and the student network model together, the detection accuracy of the student network model can be greatly improved, and because the architecture of the student network model is simple, the detection speed is fast, so the student network model meeting the recognition requirement is used to analyze the new input picture, the target class contained in the picture can be quickly and accurately determined, the detection accuracy is ensured, and the detection speed is improved.

[0140] Figure 5 A structural diagram of an electronic device provided by the embodiment of the present application is shown in FIG. 1. Figure 5 As shown in FIG. 1, the electronic device includes a memory 20 for storing a computer program.

[0141] A processor 21 is configured to execute the computer program to implement the steps of the target detection method of the above embodiment.

[0142] The electronic device provided by the embodiment of the present application can include but is not limited to a smart phone, a tablet computer, a notebook computer or a desktop computer, etc.

[0143] The processor 21 can include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 21 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), a PLA (Programmable Logic Array). The processor 21 can also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also referred to as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 can be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing content required to be displayed by the display screen. In some embodiments, the processor 21 can further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

[0144] The memory 20 can include one or more computer-readable storage media that can be non-transitory. The memory 20 can further include a high-speed random access memory, and a nonvolatile memory such as one or more disk storage devices, flash storage devices. In the present embodiment, the memory 20 is at least used to store the following computer program 201, wherein the computer program is loaded and executed by the processor 21, and can implement the related steps of the target detection method disclosed in any of the preceding embodiments. In addition, the resources stored by the memory 20 can further include an operating system 202 and data 203, and the storage mode can be temporary storage or permanent storage. The operating system 202 can include Windows, Unix, Linux, and the like. The data 203 can include, but is not limited to, sample pictures, and the like.

[0145] In some embodiments, the electronic device can further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.

[0146] Those skilled in the art can understand that the structure shown in the above embodiments does not constitute a limitation on the electronic device, and can include more or fewer components than those shown in the drawings. Figure 5 The structure shown in the above embodiments does not constitute a limitation on the electronic device, and can include more or fewer components than those shown in the drawings.

[0147] It can be understood that if the target detection method in the above embodiments is implemented in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and performs all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), an electrically erasable programmable ROM, a register, a hard disk, a removable magnetic disk, a CD-ROM, a magnetic disk or an optical disk, and various media that can store program codes.

[0148] Based on this, the embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the target detection method are realized.

[0149] The functions of each functional module of the computer readable storage medium according to the embodiment of the present application can be specifically realized according to the method in the method embodiment, and the specific implementation process can be referred to the related description of the method embodiment. Here, it will not be repeated.

[0150] The above describes in detail the target detection method, device, equipment and computer readable storage medium provided by the embodiment of the present application. The embodiments in the specification are described in a progressive manner, and each embodiment mainly describes the difference from other embodiments. The same and similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the related parts can be referred to the method part.

[0151] The skilled person can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of the two. In order to clearly show the interchangeability of hardware and software, the components and steps of each example have been described in the above description. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solutions. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0152] The above describes in detail the target detection method, device, equipment and computer readable storage medium provided by the present application. The principles and implementation modes of the present application are described by applying specific examples, and the above description of the embodiments is only used to help understand the method of the present application and its core idea. It should be pointed out that, for those skilled in the art, without departing from the principles of the present application, the present application can be improved and modified in several ways, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A target detection method, characterized in that, include: The teacher network model and the student network model are used to extract features from the sample images in parallel to obtain the first category features corresponding to the teacher network model and the second category features corresponding to the student network model. Convert the first category features and the second category features into feature information of the same dimension; Based on the same-dimensional feature information and the category labels corresponding to the sample images, the parameters of the student network model are iteratively adjusted to obtain a student network model that meets the recognition requirements; while adjusting the parameters, the parameters of the teacher network model and the student network model are also adjusted simultaneously. The newly input image is analyzed using the student network model that meets the recognition requirements to determine the target category contained in the image; The method for constructing the teacher network model and the student network model includes: Based on the YOLO network architecture, a teacher network model for target type recognition is constructed. The teacher network model is trimmed to obtain the student network model; wherein, both the teacher network model and the student network model include a first module for extracting image features, a second module for feature mining, and a third module for separating classification features; the first module, the second module, and the third module use different strategies and loss functions; The step of iteratively adjusting the parameters of the student network model based on the same-dimensional feature information and the category labels corresponding to the sample images to obtain a student network model that meets the recognition requirements includes: The loss function is determined based on the same-dimensional feature information and the category label corresponding to the sample image; Based on the loss function, adjust the parameters of the student network model and the teacher network model; Returning to the step of using the teacher network model and the student network model to extract features from the sample images in parallel, obtaining the first category features corresponding to the teacher network model and the second category features corresponding to the student network model, until the recognition accuracy and recognition efficiency of the student network model meet the set conditions, then a student network model that meets the recognition requirements is obtained; The step of determining the loss function based on the same-dimensional feature information and the category label corresponding to the sample image includes: Based on the first and second initial feature information converted to the same dimension, the first loss function is determined; Based on the first and second depth feature information converted to the same dimension, the second loss function is determined; Based on the first and second target feature information converted to the same dimension, and the category label corresponding to the sample image, a third loss function is determined.

2. The target detection method according to claim 1, characterized in that, Feature extraction of sample images using a teacher network model yields the following first-category features corresponding to the teacher network model: The first module of the teacher network model is used to extract features from the sample image to obtain first initial feature information; The first initial feature information is input into the second module of the teacher network model for convolution processing to obtain the first depth feature information; The third module of the teacher network model is used to extract the first target feature information that matches the classification category from the first deep feature information; wherein, the first initial feature information, the first deep feature information and the first target feature information constitute the first category feature.

3. The target detection method according to claim 2, characterized in that, Feature extraction of sample images using a student network model yields the following second-category features corresponding to the student network model: The first module of the student network model is used to extract features from the sample image to obtain second initial feature information; The second initial feature information is input into the second module of the student network model for convolution processing to obtain the second deep feature information; The third module of the student network model is used to extract second target feature information that matches the classification category from the second deep feature information; wherein, the second initial feature information, the second deep feature information and the second target feature information constitute the second category feature.

4. The target detection method according to claim 3, characterized in that, The extraction of second target feature information matching the classification category from the second deep feature information using the third module of the student network model includes: The second depth feature information after convolution is regressed with the pre-labeled category labels in order to adjust the parameters of the third module; Based on the adjusted parameters and the channels corresponding to the classification category, the second target feature information matching the classification category is separated from the depth feature information after convolution processing.

5. A target detection device, characterized in that, It includes an extraction unit, a conversion unit, an adjustment unit, and a determination unit; The extraction unit is used to extract features from the sample images in parallel using a teacher network model and a student network model, to obtain the first category features corresponding to the teacher network model and the second category features corresponding to the student network model. The conversion unit is used to convert the first category features and the second category features into feature information of the same dimension; The adjustment unit is used to iteratively adjust the parameters of the student network model based on the same-dimensional feature information and the category label corresponding to the sample image, so as to obtain a student network model that meets the recognition requirements; when adjusting the parameters, the parameters of the teacher network model and the student network model are adjusted simultaneously. The determining unit is used to analyze the newly input image using the student network model that meets the recognition requirements, so as to determine the target category contained in the image; Regarding the construction process of the teacher network model and the student network model, the device includes a construction unit and a pruning unit; the construction unit is used to construct a teacher network model for target type recognition based on the YOLO network architecture; the pruning unit is used to prune the teacher network model to obtain the student network model; wherein, both the teacher network model and the student network model include a first module for extracting image features, a second module for feature mining, and a third module for separating classification features; the first module, the second module, and the third module use different strategies and loss functions; The adjustment unit includes a determination subunit, a parameter adjustment subunit, and an acquisition subunit. The determination subunit determines the loss function based on the same-dimensional feature information and the category labels corresponding to the sample images. The parameter adjustment subunit adjusts the parameters of the student network model and the teacher network model based on the loss function. The return extraction unit performs parallel feature extraction on the sample images using the teacher network model and the student network model to obtain the first category features corresponding to the teacher network model and the second category features corresponding to the student network model. The acquisition subunit is used to obtain a student network model that meets the recognition requirements until the recognition accuracy and efficiency of the student network model meet the set conditions. The determination subunit determines the first loss function based on the first and second initial feature information converted to the same dimension; determines the second loss function based on the first and second deep feature information converted to the same dimension; and determines the third loss function based on the first and second target feature information converted to the same dimension and the category labels corresponding to the sample images.

6. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the target detection method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the target detection method as described in any one of claims 1 to 4.

Images