Image feature extraction method, terminal device, and computer-readable storage medium

Abstract

The application discloses an image feature extraction method, a terminal device and a computer readable storage medium. The image feature extraction method comprises: performing feature extraction of different scales on a to-be-detected image to obtain a feature map set of the to-be-detected image; the feature map set comprises at least two feature maps, and the scales of the different feature maps in the feature map set are different; determining reference values corresponding to each feature map in the feature map set; the reference value represents the importance degree of the corresponding feature map relative to other feature maps, wherein the other feature maps comprise feature maps other than the corresponding feature map in the feature map set; and updating each feature map by using the reference value corresponding to the feature map to obtain a plurality of target feature maps of the to-be-detected image. The image feature extraction method of the application assigns different reference values to feature maps of different scales, thereby updating the feature maps according to the reference values, improving the connection between the feature maps and improving the accuracy of the feature maps.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image feature extraction method, a terminal device and a computer readable storage medium. BACKGROUND

[0002] In recent years, with the development of neural network technology and the rapid improvement of computer computing power, more and more visual tasks such as pedestrian detection, face recognition, garbage detection, defect detection, license plate detection, commodity detection, remote sensing target detection, etc. can be realized by computer intelligent processing.

[0003] Target detection task has wide application value in many fields such as life, industrial production, urban construction, military reconnaissance and environmental protection by accurately positioning target position and identifying target category. However, many existing target detection algorithms based on deep learning use multi-layer feature pyramid equivalent prediction target, do not consider the importance of different levels of features, and all feature layers are equivalent to participate in prediction, so that feature learning is not focused and interlayer prediction is prone to conflict. SUMMARY

[0004] The present application provides an image feature extraction method, a terminal device and a computer readable storage medium.

[0005] To solve the above technical problems, the first technical solution provided by the present application is to provide an image feature extraction method, which comprises:

[0006] Performing feature extraction of different scales on a to-be-detected image to obtain a feature map set of the to-be-detected image; the feature map set comprises at least two feature maps, and the scales of different feature maps in the feature map set are different;

[0007] Determining reference values corresponding to each feature map in the feature map set; the reference value represents the importance of the corresponding feature map relative to other feature maps, and the other feature maps include feature maps other than the corresponding feature map in the feature map set;

[0008] Updating each feature map using the reference value corresponding to the feature map to obtain a plurality of target feature maps of the to-be-detected image.

[0009] Among them, the acquisition of the multi-scale selection attention weight of each layer feature map comprises:

[0010] The determination of the reference value corresponding to each feature map in the feature map set comprises:

[0011] Obtaining a feature vector of each feature map in the feature map set;

[0012] obtain a global feature vector based on the feature vectors of the feature maps in the feature map set;

[0013] determine reference values of the feature maps in the feature map set based on the global feature vector.

[0014] The reference values include attention weights.

[0015] The determining of the reference values of the feature maps in the feature map set based on the global feature vector comprises:

[0016] inputting the global feature vector into a preset full connection layer to obtain a feature embedding vector output by the preset full connection layer, wherein a dimension of the feature embedding vector is a number of feature maps in the feature map set;

[0017] normalizing the feature embedding vector to obtain multi-scale selection attention weights of the feature maps in the feature map set.

[0018] The obtaining of the feature vectors of the feature maps in the feature map set comprises:

[0019] performing global pooling on the feature maps in the feature map set to obtain the feature vectors of the feature maps;

[0020] The obtaining of the global feature vector based on the feature vectors of the feature maps in the feature map set comprises:

[0021] splicing the feature vectors of the feature maps to obtain the global feature vector.

[0022] The determining of the reference values of the feature maps in the feature map set based on the global feature vector comprises:

[0023] The updating of the feature maps based on the reference values corresponding to the feature maps to obtain the target feature maps of the to-be-detected image comprises:

[0024] performing weighted processing on the feature maps based on the reference values corresponding to the feature maps to obtain the target feature maps corresponding to the feature maps.

[0025] The determining of the reference values of the feature maps in the feature map set based on the global feature vector comprises:

[0026] After the obtaining of the target feature maps of the to-be-detected image, the method further comprises:

[0027] performing at least one operation of target detection, target classification or target recognition on the to-be-detected image by using the target feature maps.

[0028] To solve the above technical problem, a second technical solution provided by the present application is to provide a target detection method,

[0029] The target detection method comprises:

[0030] Based on the image feature extraction method described above, feature extraction is performed on the to-be-detected image to obtain a plurality of target feature maps of the to-be-detected image.

[0031] The plurality of target feature maps are used to perform target detection on the to-be-detected image.

[0032] The plurality of target feature maps are used to perform target detection on the to-be-detected image.

[0033] The plurality of target feature maps are input into the trained target detection model to generate a target detection frame; wherein the target detection model is trained based on a plurality of target sub-prediction losses in a training image set; the sub-prediction loss is determined based on the fitting degree between the corresponding target detection frame and the target annotation frame, and the sub-prediction loss and the corresponding fitting degree are in a positive correlation relationship.

[0034] The target detection model is obtained by training in the following manner:

[0035] The target detection model in the training is used to perform target detection on the training image set to obtain target detection frames of the plurality of targets;

[0036] Based on the fitting degree between the target detection frame and the target annotation frame of each target in the plurality of targets, a sub-prediction loss of each target is determined;

[0037] Each sub-prediction loss is used to adjust the model parameters of the target detection model in the training to obtain a trained target detection model.

[0038] The target detection model in the training is obtained by training in the following manner:

[0039] Each sub-prediction loss is weighted and summed to obtain a total loss;

[0040] The total loss is used to adjust the model parameters of the target detection model in the training.

[0041] The target detection model in the training is obtained by training in the following manner:

[0042] Each target is taken as a to-be-processed target, and the to-be-processed target is operated as follows:

[0043] The intersection over union of the target detection frame and the real annotation frame of the to-be-processed target is obtained;

[0044] acquire a dynamic center factor based on the intersection-over-union ratio;

[0045] adjust the intersection-over-union ratio by using the dynamic center factor to obtain a sub-prediction loss of the target to be processed.

[0046] The acquiring of the dynamic center factor based on the intersection-over-union ratio comprises:

[0047] acquire a preset hyperparameter, wherein the preset hyperparameter is used to control loss curve distribution;

[0048] acquire a dynamic focusing degree parameter based on the preset hyperparameter and the intersection-over-union ratio;

[0049] adjust the intersection-over-union ratio by using the dynamic focusing degree parameter to obtain the dynamic center factor.

[0050] The adjusting of the intersection-over-union ratio by using the dynamic center factor to obtain the sub-prediction loss of the target to be processed comprises:

[0051] acquire a difference between 1 and the intersection-over-union ratio;

[0052] obtain the product of the dynamic center factor and the difference to obtain the sub-prediction loss of the target to be processed.

[0053] To solve the above technical problems, a third technical solution provided by the present application is to provide a terminal device, which comprises a processor and a memory connected to the processor, wherein the memory stores program instructions; the processor is used to execute the program instructions stored in the memory to realize the image feature extraction method and / or the target detection method.

[0054] To solve the above technical problems, a fourth technical solution provided by the present application is to provide a computer readable storage medium, which stores program instructions, and the program instructions are executed to realize the image feature extraction method and / or the target detection method.

[0055] The image feature extraction method provided in the application comprises the following steps: a terminal device performs feature extraction on a to-be-detected image in different scales to obtain a feature map set of the to-be-detected image; the feature map set comprises at least two feature maps, and the scales of the different feature maps in the feature map set are different; a reference value corresponding to each feature map in the feature map set is determined; the reference value represents the importance of the corresponding feature map relative to other feature maps, wherein the other feature maps comprise feature maps other than the corresponding feature map in the feature map set; each feature map is updated by using the reference value corresponding to the feature map, and a plurality of target feature maps of the to-be-detected image are obtained. The image feature extraction method provided in the application assigns different reference values to feature maps of different scales, and then updates the feature maps according to the reference values, thereby improving the connection between the feature maps and improving the accuracy of the feature maps. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort. Among them:

[0057] Figure 1 is a flowchart of an embodiment of the image feature extraction method provided in the application;

[0058] Figure 2 is a structural diagram of the multi-scale attention network provided in the application;

[0059] Figure 3 is a flowchart of an embodiment of the target detection method provided in the application;

[0060] Figure 4 is a flowchart of another embodiment of the target detection method provided in the application;

[0061] Figure 5 is a diagram showing the sample distribution of different intersection-over-union levels provided in the application;

[0062] Figure 6 is Figure 4 is a flowchart of an embodiment of step S32 in the target detection method shown in FIG. 8;

[0063] Figure 7 is a distribution curve diagram of the dynamic center intersection-over-union loss under different hyperparameters k provided in the application;

[0064] Figure 8 is an average positioning accuracy distribution diagram of the standard intersection-over-union loss and the dynamic center intersection-over-union loss provided in the application;

[0065] Figure 9 This is a schematic diagram of the structure of an embodiment of the terminal device provided in this application;

[0066] Figure 10 This is a schematic diagram of another embodiment of the terminal device provided in this application;

[0067] Figure 11 This is a schematic diagram of the structure of the computer-readable storage medium provided in this application. Detailed Implementation

[0068] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0069] The present application will now be described in detail with reference to the accompanying drawings and embodiments.

[0070] Please see Figure 1 and Figure 2 , Figure 1 This is a schematic flowchart of an embodiment of the image feature extraction method provided in this application. Figure 2 This is a schematic diagram of the structure of the multi-scale attention network provided in this application.

[0071] like Figure 2 As shown, the multi-scale attention network provided in this application adds a multi-scale attention module to the traditional feature pyramid-based detection network. This multi-scale attention network automatically assigns different attention weights to feature maps at different levels based on the response strength of the feature maps themselves, giving the entire multi-scale attention network layer selection capability, thereby solving the problem of unfocused feature learning. The following section combines the framework and functions of the multi-scale attention network with applications in… Figure 1 The image feature extraction method shown is described in detail below:

[0072] like Figure 1 As shown, the specific steps of the image feature extraction method in this application embodiment are as follows:

[0073] Step S11: Extract features from the image to be detected at different scales to obtain a feature map set of the image to be detected; the feature map set contains at least two feature maps, and the scales of different feature maps in the feature map set are different.

[0074] In the embodiment of the present application, the terminal device uses image acquisition devices such as video monitoring instruments, traffic cameras, etc. to acquire a plurality of images to be detected. In addition, the terminal device can also use images in a mature image training set publicly available on the network as images to be detected, which will not be described here.

[0075] In the embodiment of the present application, the terminal device inputs the image to be detected into the multi-scale attention network, and the deep residual network in the multi-scale attention network extracts feature maps of the image to be detected at different scales, i.e., the deep residual network performs down-sampling of the same image to be detected at different scales to form a feature map set of the image to be detected.

[0076] Specifically, the deep residual network provided in the present application uses a skip connection to make up for the loss of texture information in down-sampling. For example, the terminal device performs down-sampling of the image to be detected at the finest scale through the deep residual network, then performs up-sampling of the feature map at the finest scale to enlarge the size, and performs 1x1 convolution channel lateral connection on the high-level feature map at the finest scale and the bottom-level feature map obtained by up-sampling to reduce parameters, and stacks them level by level to construct a feature pyramid.

[0077] Step S12: determining a reference value corresponding to each feature map in the feature map set; the reference value represents the importance of the corresponding feature map relative to other feature maps.

[0078] In the embodiment of the present application, the terminal device determines a reference value corresponding to each feature map in the feature map set; the reference value represents the importance of the corresponding feature map relative to other feature maps. The other feature maps include feature maps other than the corresponding feature map in the feature map set. In the feature map set, the higher the importance of the feature map, the greater the corresponding reference value.

[0079] Specifically, the reference value of the embodiment of the present application can also include an attention weight. Hereinafter, the determination method of the reference value of each feature map in the feature map set will be described by taking the attention weight as an example:

[0080] In the embodiment of the present application, the terminal device obtains a feature vector of each layer of feature map in the feature pyramid through a global pooling manner, i.e., a global pooling feature vector. As shown in Figure 2 The feature pyramid includes four layers of feature maps: Then, the feature vectors of the four layers of feature maps are obtained through global pooling respectively: The formula of global pooling is as follows:

[0081] V i =global pooling(Xi )

[0082] Where i∈{1,2,3,4}.

[0083] The terminal device concatenates the feature vectors V1, V2, V3, and V4 from different layers to obtain the global feature vector V. g The formula for concatenating feature vectors is as follows:

[0084] V g =concat(V1, V2, V3, V4)

[0085] The terminal device will use the global feature vector V g enter Figure 2 In a fully connected layer of the multi-scale attention network shown, feature embedding vectors are predicted for each feature map, and the predicted feature embedding vectors are normalized to obtain the multi-scale selection attention weights ω for each feature map. i (i∈{1,2,3,4}). The output dimension of the fully connected layer is the number of feature layers in the pyramid structure, meaning the number of predicted multi-scale selection attention weights is consistent with the number of feature maps in the feature pyramid. The formula for normalizing the feature embedding vector is as follows:

[0086] ω=Sigmoid(fc(V g ))

[0087] Where ω = {ω1, ω2, ω3, ω4}, and fc() is the feature embedding vector prediction function.

[0088] Through the above attention weight prediction and allocation process, the multi-scale attention network can perceive the feature distribution of all levels in the feature pyramid, and customize a multi-scale selection attention weight with global information for each feature map based on this overall information. That is, the multi-scale selection attention weight of each feature map is affected by the feature maps of other layers, thus reflecting the global information of the multi-scale selection attention weight of each feature map.

[0089] Step S13: Update each feature map using the reference values ​​corresponding to each feature map to obtain multiple target feature maps of the image to be detected.

[0090] In this embodiment of the application, the terminal device updates each feature map using the reference value corresponding to each feature map, that is, it performs weighted processing on the feature map based on the reference value to obtain multiple target feature maps of the image to be detected.

[0091] For example, after the terminal device obtains the multi-scale selection attention weights corresponding to each layer of feature maps, it assigns the multi-scale selection attention weights ω of each layer of feature maps to... iAfter broadcasting, the feature map X corresponding thereto i The multiplication is performed to implement multi-scale selection attention weighting on the feature map of each level to obtain a final feature map P with inter-layer attention i The formula of the multi-scale selection attention weighting is as follows:

[0092] P i = ω i X i

[0093] wherein i is an element of {1, 2, 3, 4}.

[0094] In the embodiment of the present application, the terminal device can perform at least one operation of target detection, target classification or target recognition on the to-be-detected image according to the multi-level feature map after the multi-scale selection attention weighting.

[0095] For example, the terminal device can input the feature map with inter-layer attention into a classification network and a positioning network to perform a classification task and a positioning task.

[0096] Through the steps S11 to S14, the multi-scale attention network of the embodiment of the present application automatically performs multi-scale selection attention weighting processing on the feature maps of different levels without increasing any hyperparameters, and realizes the inter-layer attention mechanism. This enables the multi-scale attention network to select the most important feature layer for prediction, and solves the problem of unfocused feature learning from the structure, improves the efficiency of feature learning, and improves the detection accuracy of the detection network, such as the classification network and the positioning network, for different scale targets.

[0097] In the embodiment of the present application, the terminal device performs feature extraction of different scales on the to-be-detected image to obtain a feature map set of the to-be-detected image; the feature map set includes at least two feature maps, and the scales of different feature maps in the feature map set are different; a reference value corresponding to each feature map in the feature map set is determined; the reference value represents the importance of the corresponding feature map relative to other feature maps, and the other feature maps include feature maps other than the corresponding feature map in the feature map set; each feature map is updated using the reference value corresponding to the feature map to obtain a plurality of target feature maps of the to-be-detected image. The image feature extraction method of the present application assigns different reference values to feature maps of different scales, and then updates the feature maps according to the reference values, improves the connection between the feature maps, and improves the accuracy of the feature maps.

[0098] Taking target detection on the to-be-detected image according to the multi-level feature map after the multi-scale selection attention weighting as an example, please continue to refer to Figure 3 , Figure 3 is a flowchart of an embodiment of the target detection method provided by the present application.

[0099] In the target detection method of the embodiment of the present application, a dynamic center intersection-over-union loss function is also designed, which pays more attention to the regression accuracy reduction caused by the imbalance of difficult and easy samples in the positioning task, weakens the influence of difficult samples on the regression accuracy, constrains the network from excessively deviating to difficult samples, and thus improves the overall positioning accuracy of the network.

[0100] As shown in Figure 3 , the specific steps of the target detection method of the embodiment of the present application are as follows:

[0101] Step S21: performing feature extraction on the to-be-detected image to obtain a plurality of target feature maps of the to-be-detected image.

[0102] In the embodiment of the present application, the terminal device can perform feature extraction on the to-be-detected image by using the image feature extraction method as shown in Figure 1 , and the process will not be repeated here.

[0103] Step S22: performing target detection on the to-be-detected image by using the plurality of target feature maps.

[0104] In the embodiment of the present application, the terminal device inputs the feature map with inter-layer attention to the trained target detection model, so as to generate a target detection frame on the to-be-detected image through the target detection model, thereby realizing target detection on the target in the to-be-detected image.

[0105] The target detection model is trained based on the sub-prediction loss of the plurality of targets in the training image set, the sub-prediction loss is determined based on the fitting degree between the corresponding target detection frame and the target annotation frame, and the sub-prediction loss and the corresponding fitting degree are in a positive correlation relationship. The fitting degree between the target detection frame and the target annotation frame is the overlapping degree of the target detection frame and the target annotation frame.

[0106] The training method of the target detection model will be introduced below. Please refer to Figure 4 , Figure 4 for details.

[0107] As shown in Figure 4 , the specific steps of the target detection method of the embodiment of the present application are as follows:

[0108] Step S31: performing target detection on the training image set by using the target detection model in training to obtain target detection frames of a plurality of targets.

[0109] Step S32: determining the sub-prediction loss of each target based on the fitting degree between the target detection frame of each target in the plurality of targets and the target annotation frame.

[0110] In the embodiment of the present application, the target detection model is trained based on the intersection over union loss between the target detection frame and the real label frame, so as to improve the detection effect of the target detection model.

[0111] The intersection over union loss is widely used in the training of the detection network positioning branch due to its excellent performance. The standard intersection over union loss is defined as follows:

[0112]

[0113] wherein i represents the i-th positive sample, IoU i represents the intersection over union between the i-th positive sample and its corresponding real label frame.

[0114] However, in the target detection model, the difficult samples used to train the positioning branch, i.e. the samples with lower intersection over union level, are obviously more than the simple samples, i.e. the samples with higher intersection over union level. The standard intersection over union loss treats each training sample equally, and gives each training sample the same weight 1. Obviously, this will cause the positioning branch of the target detection model to be completely controlled by the difficult samples during the training process, resulting in the target detection model being excessively biased towards the difficult samples, so that the simple samples appear negative regression, affecting the overall positioning accuracy of the target detection model. For details, please refer to Figure 5 , Figure 5 is a sample distribution diagram of different intersection over union levels provided by the present application.

[0115] As Figure 5 shown, the proportion of samples with lower intersection over union level, i.e. difficult samples, is much larger than that of samples with higher intersection over union level. Therefore, the present application proposes a dynamic center intersection over union loss (DF-IoUloss) based on the standard intersection over union loss, which is used to adjust the weight between difficult samples and simple samples, so that the contribution of difficult samples and simple samples to the target detection model tends to be balanced. For details, please refer to Figure 6 , Figure 6 The specific flowchart of step S32 in the target detection method shown in Figure 4 .

[0116] As Figure 6 shown, the specific steps of the image feature extraction method of the embodiment of the present application are as follows:

[0117] Step S321: obtaining the standard intersection over union loss between the target detection frame and the real label frame.

[0118] Step S322: obtaining a dynamic center factor based on the intersection over union between the target detection frame and the real label frame.

[0119] Step S323: adjusting the standard IoU loss by using the dynamic center factor to obtain a dynamic center IoU loss.

[0120] In the embodiments of the present application, the dynamic center IoU loss is defined as follows:

[0121]

[0122] wherein, is a dynamic focusing degree parameter.

[0123] In the dynamic center IoU loss, the positioning loss weight of each positive sample is adjusted by a dynamic center factor which can directly reflect the positioning accuracy.

[0124] Specifically, for samples with relatively high positioning accuracy, i.e., large IoU, since the proportion of these samples is relatively small, the dynamic center factor is used to increase the weight of these samples. For samples with relatively low positioning accuracy, i.e., small IoU, since the proportion of these samples is relatively large, the dynamic center factor is used to relatively reduce the weight of these samples.

[0125] Therefore, by adjusting the dynamic center factor , the contribution of samples with different IoU levels to the positioning loss will reach a relatively balanced state, thereby avoiding the situation that the positioning branch of the target detection model is excessively biased towards difficult samples. In order to further improve the adjustment ability of the dynamic center factor , the embodiments of the present application further introduce a dynamic focusing degree parameter

[0126] wherein, the dynamic focusing degree parameter is defined as follows:

[0127]

[0128] wherein, k is a hyperparameter for controlling the distribution of the loss curve.

[0129] The dynamic focusing degree parameter is calculated from the IoU value of the current sample, i.e., the value of the focusing degree parameter is determined by the IoU of the current sample. Therefore, for any determined sample, a dynamic focusing degree parameter can be customized in the above manner.

[0130] This dynamic focusing degree parameter The distribution of the positioning loss is dynamically adjusted to obtain a sample-adaptive positioning loss. It should be noted that, in order to accelerate the convergence speed of the dynamic center IoU loss, the focusing degree parameter can also be set to a fixed value, that is, the dynamic change of the focusing degree parameter is limited. In use, setting the focusing degree parameter to a fixed value will cut off the back propagation gradient, and the gradient will not be back propagated.

[0131] In order to more intuitively illustrate the advantages of the dynamic center IoU loss, the loss distribution curve under different hyperparameters k is visually displayed. Figure 7

[0132] As can be seen from Figure 7 , under different hyperparameters k, the dynamic center IoU loss is an upper convex positioning loss, which means that the gradient (tangent slope) of the positioning loss of the sample with a high IoU level is increased, and the advantages of the loss are also illustrated from the perspective of image distribution.

[0133] Step S33: Adjusting the model parameters of the target detection model in training by using the obtained each sub-prediction loss to obtain the trained target detection model.

[0134] In the embodiment of the present application, the terminal device calculates the IoU loss according to the intersection and union of the current sample and the real label, wherein the IoU and the IoU loss are in a positive correlation, that is, the larger the IoU, the larger the IoU loss. Therefore, the terminal device can adjust the contribution value of the sample to the overall IoU loss according to the size of the IoU, that is, the larger the IoU, the larger the contribution value of the corresponding sample, and the smaller the IoU, the smaller the contribution value of the corresponding sample.

[0135] In other embodiments, the terminal device can also weight and sum the obtained each sub-prediction loss to obtain the overall loss, and use the overall loss to adjust the model parameters of the target detection model in training. When the weight of each sub-prediction loss is set to 1, that is, all sub-prediction losses are directly summed to obtain the overall loss.

[0136] In the embodiment of the present application, the terminal device trains the target detection model by using the overall IoU loss obtained by adding the IoU loss corresponding to all target detection boxes in the training image, that is, updates and optimizes the model parameters in the target detection model by using each training sample and its contribution value.

[0137] In the embodiment of the present application, after a target detection model is trained by the dynamic center IoU loss, the contributions of samples with different IoU levels to the positioning branch will be balanced, thereby preventing the target detection model from excessively deviating to a certain IoU level sample.

[0138] ​​The embodiment of the present application visualizes the average precision improvement of different intersection-over-union levels of samples after a target detection model is trained by the standard intersection-over-union loss and the dynamic center intersection-over-union loss respectively. Figure 8 The average precision improvement distribution diagram of the dynamic center intersection-over-union loss visualizes the average positioning precision improvement of different intersection-over-union levels of samples after a target detection model is trained by the standard intersection-over-union loss and the dynamic center intersection-over-union loss respectively.

[0139] It can be clearly seen that when trained by the standard intersection-over-union loss, although the low intersection-over-union level (0.50-0.75) samples in the detection task obtain a larger positioning precision improvement, the high intersection-over-union level (0.75-0.95) samples basically do not obtain the positioning precision improvement, and even a negative regression occurs. As a comparison, the dynamic center intersection-over-union loss proposed in the embodiment of the present application can well alleviate this problem. As can be seen from Figure 8 , the dynamic center intersection-over-union loss has basically the same effect on the low intersection-over-union level (0.50-0.75) samples as the standard intersection-over-union loss, but has a more obvious effect on the high intersection-over-union level (0.75-0.95) samples, which can obviously reduce the risk of negative regression of the target detection model, thereby ensuring the overall positioning precision of the target detection model.

[0140] The above embodiments are only one common case of the present application, and do not limit the technical scope of the present application, so any slight modification, equivalent change or modification made according to the essence of the present application to the above content still belongs to the scope of the technical solution of the present application.

[0141] Please continue to refer to Figure 9 , Figure 9 is a structural schematic diagram of an embodiment of the terminal device provided by the present application. As shown in Figure 9 , the terminal device 30 comprises a feature extraction module 31, a parameter determination module 32 and a feature updating module 33.

[0142] The feature extraction module 31 is configured to perform feature extraction of different scales on a to-be-detected image to obtain a feature map set of the to-be-detected image; the feature map set comprises at least two feature maps, and the scales of different feature maps in the feature map set are different.

[0143] The parameter determination module 32 is configured to determine reference values corresponding to each feature map in the feature map set; the reference values represent the importance of the corresponding feature map relative to other feature maps, and the other feature maps include feature maps other than the corresponding feature map in the feature map set.

[0144] The feature updating module 33 is configured to update each feature map by using the reference value corresponding to the feature map to obtain a plurality of target feature maps of the to-be-detected image.

[0145] Please refer to Figure 10 , Figure 10 is a structural schematic diagram of another embodiment of the terminal device provided in the present application. The terminal device comprises a memory 52 and a processor 51 connected to each other.

[0146] The memory 52 is used for storing program instructions for implementing any one of the above image feature extraction methods and / or target detection.

[0147] The processor 51 is used for executing the program instructions stored in the memory 52.

[0148] The processor 51 can also be referred to as a CPU (Central Processing Unit). The processor 51 can be an integrated circuit chip with a processing capability of signaling. The processor 51 can also be a general-purpose processor, a digital signaling processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0149] The memory 52 can be a memory stick, a TF card, etc., and can store all information in the terminal device, including input raw data, computer programs, intermediate running results and final running results. It stores and retrieves information according to the specified location of the controller. With the memory, the string matching prediction device has a memory function and can work normally. The memory of the string matching prediction device can be divided into main memory (internal memory) and auxiliary memory (external memory) according to the purpose, and there is also a classification method of external memory and internal memory. The external memory is usually a magnetic medium or an optical disc, etc., which can store information for a long time. The internal memory refers to the storage component on the motherboard, which is used to store the data and programs currently being executed, but only used for temporarily storing programs and data, and the data will be lost when the power is off.

[0150] In several embodiments provided in the present application, it should be understood that the disclosed method and device can be implemented in other ways. For example, the above-described device implementation is only schematic, for example, the division of modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed mutual ones can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0151] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed to multiple network units. Part or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0152] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0153] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a system server, or a network device, etc.) or a processor to execute all or part of the steps of the methods of the various embodiments of the present application.

[0154] Please refer to Figure 11 , the structural schematic diagram of the computer readable storage medium of the present application. The storage medium of the present application stores a program file 61 capable of realizing all the image feature extraction methods and / or target detection, wherein the program file 61 can be stored in the above storage medium in the form of a software product, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods of the various embodiments of the present application. The foregoing storage device includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, etc. Various media that can store program codes, or a computer, a server, a mobile phone, a tablet, etc. Terminal device.

[0155] The above is only the embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation using the contents of the present application specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A target detection method, characterized in that, The target detection method includes: Feature extraction at different scales is performed on the image to be detected to obtain a feature map set of the image to be detected; the feature map set contains at least two feature maps, and the different feature maps in the feature map set have different scales; Determine a reference value for each feature map in the feature map set; the reference value represents the importance of the corresponding feature map relative to other feature maps, and the other feature maps include feature maps other than the corresponding feature map in the feature map set. By using the reference values ​​corresponding to each feature map, each feature map is updated to obtain multiple target feature maps of the image to be detected; Target detection is performed on the image to be detected using the multiple target feature maps; The multiple target feature maps are input into the trained target detection model to generate target detection boxes; The target detection model was trained in the following manner: The target detection model in training is used to detect targets in the training image set to obtain the target detection boxes of the multiple targets. Based on the degree of fit between the target detection box and the target annotation box of each of the multiple targets, the sub-prediction loss of each target is determined; By using the obtained sub-prediction losses, the model parameters of the target detection model in training are adjusted to obtain the trained target detection model. The step of determining the sub-prediction loss for each target based on the fit between the target detection bounding box and the target annotation bounding box of each of the multiple targets includes: Each target is treated as a target to be processed, and the following operations are performed on the targets to be processed: Obtain the intersection-union ratio (IoU) of the target detection bounding box and the ground truth bounding box of the target to be processed; Obtain preset hyperparameters, wherein the preset hyperparameters are used to control the distribution of the loss curve; Based on the preset hyperparameters and the cross-union ratio, a dynamically changing focus parameter is calculated; wherein, the larger the cross-union ratio, the larger the value of the dynamically changing focus parameter. The cross-union ratio is adjusted using the dynamically changing focus parameter to obtain the dynamic center factor; The sub-prediction loss of the target to be processed is obtained by adjusting the crossover-union ratio using the dynamic center factor.

2. The target detection method according to claim 1, characterized in that, Determining the reference value corresponding to each feature map in the feature map set includes: Obtain the feature vectors of each feature map in the feature map set; The global feature vector is obtained based on the feature vectors of each feature map in the feature map set. The reference values ​​for each feature map in the feature map set are determined based on the global feature vector.

3. The target detection method according to claim 2, characterized in that, The reference values ​​include attention weights; Determining the reference values ​​of each feature map in the feature map set based on the global feature vector includes: The global feature vector is input into a preset fully connected layer to obtain the feature embedding vector output by the preset fully connected layer, wherein the dimension of the feature embedding vector is the number of feature maps in the feature map set; The feature embedding vectors are normalized to obtain the multi-scale selection attention weights for each feature map in the feature map set.

4. The target detection method according to claim 2 or 3, characterized in that, The step of obtaining the feature vectors of each feature map in the feature map set includes: Each feature map in the feature map set is subjected to global pooling to obtain the feature vector of each feature map; The step of obtaining the global feature vector based on the feature vectors of each feature map in the feature map set includes: The feature vectors of the various feature maps are concatenated to obtain the global feature vector.

5. The method according to claim 1, characterized in that, The step of updating each feature map using the reference values ​​corresponding to each feature map to obtain multiple target feature maps of the image to be detected includes: Based on the reference values ​​corresponding to each feature map, the feature maps are weighted to obtain the target feature map corresponding to each feature map.

6. The method according to claim 1, characterized in that, After obtaining multiple target feature maps of the image to be detected, the method further includes: Using the multiple target feature maps, at least one of the following operations is performed on the image to be detected: target detection, target classification, or target recognition.

7. The target detection method according to claim 1, characterized in that, The method utilizes the multiple target feature maps to perform target detection on the image to be detected. include: The target detection model is trained based on the sub-prediction loss of multiple targets in the training image set; the sub-prediction loss is determined based on the degree of fit between the corresponding target detection box and the target annotation box, and the sub-prediction loss and the corresponding degree of fit are positively correlated.

8. The target detection method according to claim 1, characterized in that, The step of adjusting the model parameters of the target detection model during training using the obtained sub-prediction losses includes: The total loss is obtained by weighted summing of the losses from each sub-prediction. The model parameters of the target detection model during training are adjusted using the overall loss.

9. The target detection method according to claim 1, characterized in that, The step of adjusting the intersection-union ratio using the dynamic center factor to obtain the sub-prediction loss of the target to be processed includes: The sub-prediction loss of the target to be processed is obtained by multiplying the difference between the dynamic center factor and the intersection-union ratio.

10. A terminal device, characterized in that, The terminal device includes a processor and a memory connected to the processor, wherein... The memory stores program instructions; The processor is used to execute program instructions stored in the memory to implement the target detection method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that, when executed, implement the target detection method according to any one of claims 1 to 9.

Images