Training method and device of target detection model, target detection method

Abstract

Provided are a training method and device of a target detection model, and a target detection method. The training method comprises: obtaining a training sample set, comprising first and second training samples, the first and second training samples respectively comprising clear data and fuzzy data; inputting the first training sample into the target detection model; determining a first loss value of the target detection model according to an output value of the target detection model and the second training sample, using a first loss function; inputting the first loss value into a Gaussian mixture model; constructing a fuzziness matrix using the Gaussian mixture model, the fuzziness matrix being used to represent the fuzziness of the data in the first training sample and the second training sample; constructing a fuzzy loss function according to the fuzziness matrix, the fuzzy loss function being constructed to be able to distinguish between the clear data and the fuzzy data in the first training sample and the second training sample; constructing a second loss function according to the fuzzy loss function; and retraining the target detection model using the first and second training sample sets and the second loss function.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, in particular to a target detection model training method and device, a target detection method, an electronic device, a computer readable storage medium and a program product. BACKGROUND

[0002] Passive terahertz human security technology can penetrate ordinary materials such as clothes and shoes to find suspicious objects hidden on the surface. Moreover, the terahertz imaging device itself does not emit electromagnetic waves, and the imaging is achieved by receiving terahertz waves generated by the human body, which is completely free of ionizing radiation and has high safety, and is particularly suitable for human security. Therefore, it has received widespread attention in the industry in recent years. Suspicious object detection on terahertz images using various algorithms can further save human resources, improve processing efficiency, and improve detection speed.

[0003] Suspicious object detection on terahertz images using traditional methods is mainly based on optical image processing technology, which slides through the image according to manually constructed features to find matching samples. This method has slow processing speed, high complexity and poor robustness, and has not been widely used. With the development of deep learning technology in the field of image detection, researchers have also applied it to terahertz suspicious object detection. However, most of the research only applies general detection paradigms to terahertz images, and does not analyze the characteristics of terahertz images, terahertz open and non-intrusive application scenarios, and lacks consideration of suspicious object deformation, blurred labeling and background interference, which limits its performance in actual applications.

[0004] The above information disclosed in this section is only for understanding the background of the inventive concept of the present disclosure, and therefore, the above information can include information that does not constitute the prior art. SUMMARY

[0005] In view of at least one aspect of the above technical problems, a target detection model training method and device, a target detection method, an electronic device, a computer readable storage medium and a program product are provided.

[0006] In one aspect, a training method of a target detection model based on fuzzy learning is provided, comprising:

[0007] Obtaining a training sample set, the training sample set comprising a first training sample and a second training sample, the first training sample and the second training sample being used to represent the input and output of the target detection model respectively, the first training sample comprising clear data and fuzzy data, and the second training sample comprising clear data and fuzzy data;

[0008] Inputting the first training sample into the target detection model;

[0009] determining a first loss value of the target detection model by using a first loss function according to the output value of the target detection model and the second training sample;

[0010] inputting the first loss value into a Gaussian mixture model;

[0011] constructing a blur matrix by using the Gaussian mixture model, wherein the blur matrix is used to represent blur degrees of data in the first training sample and the second training sample;

[0012] constructing a blur loss function according to the blur matrix, wherein the blur loss function is constructed to be able to distinguish clear data and blur data in the first training sample and the second training sample;

[0013] constructing a second loss function according to the blur loss function; and

[0014] retraining the target detection model by using the first training sample set, the second training sample set and the second loss function.

[0015] According to some exemplary embodiments, the first loss function comprises a first bounding box loss, a first confidence loss and a first classification loss.

[0016] According to some exemplary embodiments, the first bounding box loss adopts a GIoU loss, the first confidence loss adopts a mixed entropy loss, and the first classification loss adopts a mixed entropy loss.

[0017] According to some exemplary embodiments, the first loss function is specifically calculated by the following formula:

[0018]

[0019] wherein l w is a first loss function, is a first bounding box loss, is a first confidence loss, is a first classification loss, σ box , σ obj and σ cls are weight coefficients of the first bounding box loss, the first confidence loss and the first classification loss respectively.

[0020] According to some exemplary embodiments, the mixed entropy loss comprises a cross-entropy loss and a negative entropy loss.

[0021] According to some exemplary embodiments, the mixed entropy loss is specifically calculated by the following formula:

[0022] l ME = l CE + β·lNE = -y log(y * )- β y * log(y * )

[0023] wherein, l ME represents a mixed entropy loss, l CE and l NE respectively represent a cross-entropy loss and a negative entropy loss, β is a weight coefficient of the negative entropy loss; y is a value in the second training sample, representing a labeled value; y * is an output value of the target detection model, representing a predicted value.

[0024] According to some exemplary embodiments, the fuzziness matrix is specifically represented as:

[0025]

[0026] wherein, C f is a fuzziness matrix, represents a probability that the predicted value is when the labeled value is y c , and N represents a number of row elements and column elements in the fuzziness matrix.

[0027] According to some exemplary embodiments, the fuzzy loss function is specifically represented by the following formula:

[0028] l f = - (1 - r) ((1 - y * ) 1-r y *r ) γ log(y * )- r ((1 - y * ) 1-r y *r ) γ log(1 - y * )

[0029] wherein, l f represents a fuzzy loss function, r represents a proportion of fuzzy data; γ ≥ 0 is a modulation coefficient, used for adjusting sensitivity of the target detection model to data fuzziness.

[0030] According to some exemplary embodiments, the proportion r of the fuzzy data is obtained by the following formula:

[0031] r = 1 - diag(C f )

[0032] wherein, diag(C f ) represents diagonal elements of the fuzziness matrix C f .

[0033] According to some exemplary embodiments, the second loss function is calculated by the following formula:

[0034] l = σ box · l box + σ obj · l obj + σ cls · l cls

[0035] wherein, l is the second loss function, l box is the second bounding box loss, l obj is the second confidence loss, l cls is the second classification loss, σ box ', σ obj ' and σ cls ' are weight coefficients of the second bounding box loss, the second confidence loss and the second classification loss respectively.

[0036] According to some exemplary embodiments, the second bounding box loss adopts a GIoU loss, and the second confidence loss and the second classification loss both adopt the fuzzy loss function.

[0037] According to some exemplary embodiments, the fuzzy degree matrix is constructed by using the Gaussian mixture model, specifically comprising:

[0038] For the first confidence loss and the first classification loss, a confidence fuzzy degree matrix and a classification fuzzy degree matrix are respectively constructed by using the Gaussian mixture model.

[0039] According to some exemplary embodiments, for the confidence fuzzy degree matrix, N = 2; and / or,

[0040] For the classification fuzzy degree matrix, N is equal to the total number of classes of the classification output by the target detection model.

[0041] According to some exemplary embodiments, the first training sample comprises image data acquired by scanning an imaging region by a terahertz imaging device; and / or,

[0042] The second training sample comprises image data with a labeled bounding box, wherein the labeled bounding box is used to represent the fuzzy position and / or type of the target to be detected in the image data.

[0043] In another aspect, a target detection method is provided, comprising:

[0044] acquiring image data;

[0045] inputting the image data into a target detection model, wherein the target detection model is trained by the method as described above; and

[0046] determining, according to an output of the target detection model, a target to be detected in the image data.

[0047] In yet another aspect, a training apparatus of a target detection model based on fuzzy learning is provided, comprising:

[0048] a training sample acquisition module configured to acquire a training sample set, the training sample set comprising first training samples and second training samples, the first training samples and the second training samples being respectively configured to represent inputs and outputs of the target detection model, the first training samples comprising clear data and fuzzy data, and the second training samples comprising clear data and fuzzy data;

[0049] a target detection module configured to receive the first training samples, and determine, according to an output value of the target detection model and the second training samples, a first loss value of the target detection model by using a first loss function;

[0050] a blur degree matrix construction module comprising a Gaussian mixture model, the blur degree matrix construction module being configured to receive the first loss value, and construct a blur degree matrix by using the Gaussian mixture model, wherein the blur degree matrix is configured to represent blur degrees of data in the first training samples and the second training samples;

[0051] a fuzzy loss function construction module configured to construct a fuzzy loss function according to the blur degree matrix, wherein the fuzzy loss function is configured to be able to distinguish clear data and fuzzy data in the first training samples and the second training samples;

[0052] a loss function construction module configured to construct a second loss function according to the fuzzy loss function; and

[0053] a retraining module configured to retrain the target detection model by using the first training sample set, the second training sample set, and the second loss function.

[0054] In yet another aspect, an electronic device is provided, comprising:

[0055] one or more processors;

[0056] a storage configured to store one or more programs,

[0057] wherein the one or more programs, when executed by the one or more processors, enable the one or more processors to perform the method as described above.

[0058] According to some exemplary embodiments, the electronic device is a passive terahertz imaging apparatus.

[0059] In still another aspect, a computer readable storage medium is provided, having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method as described above.

[0060] In still another aspect, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method as described above.

[0061] In embodiments of the present disclosure, by gradually extracting clear features of the target to be detected, the influence of ambiguous information is reduced, thereby improving the detection rate and classification accuracy of the trained target detection model for the target to be detected, and reducing the false positive rate. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to better understand the embodiments of the present disclosure, the embodiments of the present disclosure will be described in detail according to the following drawings:

[0063] Figure 1 is a structural schematic diagram of a passive terahertz imaging device according to an exemplary embodiment of the present disclosure.

[0064] Figures 2A to 2E is a schematic diagram of images acquired by a terahertz imaging device in different cases.

[0065] Figure 3 is a schematic flowchart of a training method of a target detection model based on fuzzy learning according to an exemplary embodiment of the present disclosure.

[0066] Figure 4 is a detailed flowchart of a training method of a target detection model based on fuzzy learning according to an exemplary embodiment of the present disclosure, in which the process of fuzzy learning is shown in more detail.

[0067] Figure 5 is a schematic diagram of CE loss and NE loss.

[0068] Figure 6A and Figure 6B shows the loss distribution under different loss functions, respectively.

[0069] Figure 7 is a schematic diagram of a fuzzy loss function according to an embodiment of the present disclosure.

[0070] Figure 8 is a structural block diagram of a training device of a target detection model according to an exemplary embodiment of the present disclosure.

[0071] Figure 9 is a structural block diagram of an electronic device suitable for implementing the training method of the target detection model or the target detection method according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0072] Specific embodiments of the present disclosure will be described in detail below, it should be noted that the embodiments described herein are only used for illustration and do not limit the present disclosure. In the following description, a large number of specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it is obvious to those skilled in the art that the specific details need not be used to practice the present disclosure. In other instances, well-known structures, materials or methods are not specifically described in order to avoid obscuring the present disclosure.

[0073] Throughout the specification, reference to "one embodiment", "an embodiment", "one example" or "an example" means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment of the disclosure. Therefore, the appearance of the phrases "in one embodiment", "in an embodiment", "one example" or "an example" in various places throughout the specification are not necessarily all referring to the same embodiment or example. In addition, particular features, structures or characteristics can be combined in any suitable combination and / or sub-combination in one or more embodiments or examples. In addition, those skilled in the art will understand that the term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0074] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the disclosure. The terms "comprising", "including", and the like as used herein mean the presence of stated features, steps, operations, and / or components but do not preclude the presence or addition of one or more other features, steps, operations, or components.

[0075] All terms used herein, including technical and scientific terms, have meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted in a manner consistent with the context of the specification, and should not be interpreted in an idealized or overly formal manner.

[0076] The inventors have found, through research, that, unlike conventional optical image target detection, passive terahertz image target detection in the non-inductive customs clearance application scenario has the problems of feature ambiguity and label ambiguity.

[0077] To this end, an embodiment of the present disclosure provides a training method of a target detection model based on fuzzy learning, comprising: obtaining a training sample set, the training sample set comprising first training samples and second training samples, the first training samples and the second training samples being used to represent the input and output of the target detection model respectively, the first training samples comprising clear data and fuzzy data, and the second training samples comprising clear data and fuzzy data; inputting the first training samples into the target detection model; determining a first loss value of the target detection model by using a first loss function according to the output value of the target detection model and the second training samples; inputting the first loss value into a Gaussian mixture model; constructing a fuzziness matrix by using the Gaussian mixture model, wherein the fuzziness matrix is used to represent the fuzziness of the data in the first training samples and the second training samples; constructing a fuzzy loss function according to the fuzziness matrix, wherein the fuzzy loss function is constructed to be able to distinguish the clear data and the fuzzy data in the first training samples and the second training samples; constructing a second loss function according to the fuzzy loss function; and retraining the target detection model by using the first training sample set, the second training sample set and the second loss function. In the embodiment of the present disclosure, the clear features of the target to be detected are extracted step by step, and the influence of the fuzzy information is reduced, so that the detection rate and classification accuracy of the trained target detection model for the target to be detected can be improved, and the false positive rate can be reduced.

[0078] Figure 1 is a structural schematic diagram of a passive terahertz imaging device according to an example embodiment of the present disclosure.

[0079] As shown in Figure 1 , the passive terahertz imaging device according to an example embodiment of the present disclosure can include a reflector 2 and its servo system, a lens 3, a detector array 4, a data acquisition and processing device 6, a display device 7 and a distribution box 5. The terahertz waves spontaneously emitted by the object to be detected and the terahertz waves reflected by the background environment are incident on the reflector 2 through the window 1 on the shell, and are reflected by the reflector 2 to the lens 3, and are received by the detector array 4 after converging through the lens 3. The detector array 4 converts the received terahertz waves into electrical signals. The data acquisition and processing device 6 is connected with the detector array 4 to receive the electrical signals from the detector array 4 and generate a millimeter wave / terahertz wave image. The display device 7 is connected with the data acquisition and processing device 6 to receive and display the terahertz wave image generated by the data acquisition and processing device 6. The distribution box 5 is configured to provide power to the entire passive terahertz imaging device.

[0080] In actual work, the servo system of the reflecting plate 2 controls the reflecting plate 2 to reciprocate, and the reciprocal period T is the reciprocal of the imaging frame rate s. When the reflecting plate 2 swings from the maximum elevation angle to the minimum depression angle, the swing angle is θ, so as to complete the scanning of the field of view with the height direction of 2θ in the depth of field range. The reflecting plate 2 forms a figure from the maximum elevation angle to the minimum depression angle. The data acquisition and processing device 6 collects data all the time in this process. The control system of the reflecting plate 2 can be provided with a position encoder, for example, to feed back the scanning position of the reflecting plate with high precision. When collecting data, the data acquisition and processing device 6 first marks the collected data according to the information of the position encoder, for distinguishing the data of the next figure, and then processes and reconstructs the collected data to generate a terahertz image. Then, the data acquisition and processing device 6 can transmit the image data to the display device 7 for display, suspicious object marking and automatic alarm on the display device 7.

[0081] For the above-mentioned passive terahertz imaging device, the inventors have found that target detection on the terahertz image obtained by the passive terahertz imaging device has feature ambiguity and marking ambiguity.

[0082] It should be noted that, in this document, various embodiments are described by taking the terahertz imaging device as an example, but the embodiments of the present disclosure are not limited to terahertz imaging, and can also be applied to other imaging modes such as millimeter wave imaging, without conflict.

[0083] Specifically, the feature ambiguity is mainly caused by the terahertz imaging characteristics and factors such as occlusion deformation, motion deformation, imaging angle deformation and environmental light and electromagnetic wave interference. The passive terahertz imaging technology is based on blackbody radiation theory, and uses a terahertz antenna to sense the human body radiation energy, so as to form a two-dimensional gray terahertz image with energy intensity information. When carrying a suspicious object (i.e. a target to be detected), due to the occlusion of the human body radiation energy, the suspicious object region will form a gray difference with the human body, so that the suspicious object information can be displayed in the image. The terahertz image has good contour information and gray information, but compared with the visible light image, the image resolution is low, and is easily affected by occlusion, motion, imaging angle and external environment, thereby causing feature ambiguity. Figures 2A to 2E The images obtained by the terahertz imaging device in different situations are respectively schematically shown. For example, Figures 2A to 2E The imaging results of the same person carrying a gun simulation object and a mobile phone simulation object in different situations are respectively shown. It can be seen that, compared with Figure 2A The ideal imaging shown in FIG. 6A, in the case of Figures 2B to 2EIn the middle, due to the blocking deformation, motion deformation, angle deformation, external interference deformation and other factors, the image features in these cases are blurred to varying degrees. For example, the features of the gun simulation object are similar to those of the mobile phone simulation object in some cases. In an open and non-susceptible application scenario, human motion, suspect carrying methods, and external environment are diverse, and image feature blurring is common, which will have some impact on target identification.

[0084] Labeling ambiguity is the ambiguity of label information generated by manual labeling. Since manual labeling can only be judged by terahertz image information, and there is strong subjectivity, when the image features are blurred, the labeling personnel cannot label the target through fixed standards. For the presence or absence of the target, the target belongs to the category, different labeling personnel may give different labeling results, and even the same labeling personnel may give different labeling results in different scenarios. This will make the labeling information ambiguous, so that the model cannot distinguish effective label information during training.

[0085] For example, a deep neural network model optimizes the empirical risk R l，D Obtain the model parameters θ, the process can be represented as:

[0086]

[0087] Where l(·) represents the loss function; (X, Y) e D represents the sample and its corresponding label; f(X, θ) represents the model prediction result. A prerequisite for formula (1) to obtain an ideal result is that the data set D is clear, clean, accurate and correct at this time. When the data and labels are ambiguous, formula (1) changes to:

[0088]

[0089] Where (X f , Y f ) e D f represent the ambiguous sample and the ambiguous label in the ambiguous data set respectively; θ f represents the parameter obtained by training the ambiguous data set. Since {X, Y} and {X f , Y f} have different distribution characteristics, it should be understood that θ f ≠ θ, the model parameters obtained by training on the ambiguous data set cannot achieve the optimal effect on the clear data set, which will affect the model performance.

[0090] It should be noted that for the function argminf(x), argmin represents the variable value when the objective function f(x) takes the minimum value.

[0091] In embodiments of the present disclosure, a training method of a target detection model based on fuzzy learning and a target detection method using the target detection model are proposed. In the training method, fuzzy data in terahertz image data is evaluated by using warm-up training and a Gaussian Mixture Model (GMM), a fuzzy matrix is estimated, fuzzy sample weights are adjusted, and clear information in training samples is extracted by step-by-step iteration, so as to reduce the influence of feature fuzziness and label fuzziness in terahertz images. Correspondingly, the target detection model obtained by using the training method can improve the detection rate of passive terahertz human suspicious object detection in a non-susceptible pass-through application scenario, reduce the false positive rate, and improve the classification accuracy.

[0092] Figure 3 is a schematic flowchart of a training method of a target detection model based on fuzzy learning according to an exemplary embodiment of the present disclosure. Figure 4 is a detailed flowchart of a training method of a target detection model based on fuzzy learning according to an exemplary embodiment of the present disclosure, in which the process of fuzzy learning is shown in more detail.

[0093] As shown in Figure 3 and Figure 4 , the training method of the target detection model based on fuzzy learning according to an exemplary embodiment of the present disclosure can include operations S310-S380, which can be executed by a processor or any electronic device including the processor.

[0094] In operation S310, a training sample set is obtained, which includes first training samples and second training samples, and the first training samples and the second training samples are respectively used to represent the input and output of the target detection model. The first training samples include clear data and fuzzy data, and the second training samples include clear data and fuzzy data.

[0095] For example, the image data actually sampled by the terahertz imaging device can be used as the first training sample. As described above, due to the influence of occlusion, motion, imaging angle and external environment, the features of part of the data in the image data actually sampled by the terahertz imaging device will not be clear, i.e., there is feature fuzziness. That is, the first training data includes clear data and fuzzy data.

[0096] For the image data actually sampled by the terahertz imaging device, manual labeling can be performed, for example, a label box is used to label the position and / or type of the target to be detected (for example, various suspicious objects). The image data with the label box is taken as the second training sample. As described above, since manual labeling can only be determined by terahertz image information, and there is strong subjectivity, when the image features are blurred, the labeler cannot label the target by fixed standards, that is, there is a labeling ambiguity. Therefore, in the second training sample, the label box is mainly used to represent the ambiguous position and / or type of the target to be detected in the image data.

[0097] In operation S320, the first training sample is input into the target detection model.

[0098] In operation S330, according to the output value of the target detection model and the second training sample, a first loss value of the target detection model is determined by using a first loss function.

[0099] In operation S320 and operation S330, the target detection model is subjected to warm-up training, in which a first loss value of the target detection model is determined by using a first loss function.

[0100] For example, in the warm-up training, 10-30 Epochs can be trained.

[0101] It should be noted that in the training of a deep machine learning model, the expression Epoch means that the entire data of the training set is used to train the model once, which can be referred to as "one generation of training". For example, when a complete data set passes through a neural network model once and returns once, this process is called one Epoch. That is, all training samples have undergone one forward propagation and one backward propagation in the neural network. It should be understood that when the number of samples in an Epoch (that is, all training samples) can be too large (for a computer), it is usually necessary to divide it into multiple small blocks, that is, into multiple Batches for training.

[0102] In an embodiment of the present disclosure, the first loss function includes a first bounding box loss, a first confidence loss, and a first classification loss.

[0103] Specifically, the first loss function is calculated by the following formula (3):

[0104]

[0105] wherein l w is the first loss function, is the first bounding box loss, is the first confidence loss, is a first classification loss, σ box , σ obj and σ cls are weight coefficients of the first bounding box loss, the first confidence loss and the first classification loss respectively.

[0106] For example, the first bounding box loss adopts a GIoU loss, the first confidence loss adopts a hybrid entropy loss, and the first classification loss adopts a hybrid entropy loss.

[0107] It should be understood that the IoU (Intersection over Union) loss is a commonly used index in target detection, which is not only used to determine positive samples and negative samples, but also used to evaluate the distance between a predicted detection box and a ground-truth. The IoU loss can be expressed as follows:

[0108]

[0109] Wherein, A and B represent two detection boxes, i.e. a predicted detection box and a ground-truth.

[0110] Since IoU is a ratio concept, it is not sensitive to the size of the target object. On this basis, the GIoU loss is proposed, which is specifically as follows:

[0111]

[0112] The above formula means: first calculate the minimum closed region area U (i.e. the area of the minimum box containing the predicted detection box and the ground-truth), then calculate the IoU, then calculate the proportion of the area not belonging to the two boxes in the closed region A c , and finally subtract the proportion from the IoU to obtain the GIoU.

[0113] In an embodiment of the present disclosure, the hybrid entropy loss includes a cross entropy (CE) loss and a negative entropy (NE) loss. For example, the hybrid entropy loss is specifically calculated by the following formula (4):

[0114] l ME = l CE + β · l NE = - y · log(y * ) - β · y * · log(y * ) (4)

[0115] Wherein, l ME represents the hybrid entropy loss, lCE and l NE respectively represent cross-entropy loss and negative entropy loss, β is a weight coefficient of the negative entropy loss; y is a value in the second training sample, representing a labeled value; y * is an output value of the target detection model, representing a predicted value.

[0116] Figure 5 are schematic diagrams of CE loss and NE loss. In Figure 5 , the solid line represents the curve of the NE loss function, and the dashed line represents the curve of the CE loss function. It can be seen that the CE loss is smaller when the model predicts correctly and larger when the model predicts incorrectly, which will make the model parameters adjust to the direction of correct prediction; the NE loss is smaller when the model predicts extreme values and larger when the model predicts intermediate values, which will make the model parameters adjust to the direction of predicting extreme values. Therefore, the mixed entropy has the commonality of both, and using the loss function containing the mixed entropy for warm-up training can make the loss distribution difference between clear samples and ambiguous samples larger, thereby strengthening the distinction of data. Figure 6A and Figure 6B respectively show the loss distribution under different loss functions. Figure 6A shows the cross-entropy loss result, Figure 6B shows the mixed entropy loss result. In Figure 6A and Figure 6B , the deep color represents the distribution of ambiguous data, and the light color represents the distribution of clear data. It can be seen that compared with cross-entropy, the loss distribution using mixed entropy is more uniform, and the distribution difference between clear data and ambiguous data is more obvious, which can better fit.

[0117] In operation S340, the first loss value is input into a Gaussian mixture model.

[0118] In an embodiment of the present disclosure, Gaussian mixture model (GMM) is used to model clear data and ambiguous data. The Gaussian mixture model can be regarded as a model composed of multiple single Gaussian models, represented as:

[0119]

[0120] wherein μ, σ, ω respectively represent expectation, variance and sub-distribution weight; Φ k , μ k , σ k , ω k respectively represent the kth sub-distribution and its corresponding expectation, variance and distribution weight. The Gaussian mixture model has better adaptability to asymmetric ambiguous distribution, and can better distinguish clear data and ambiguous data.

[0121] It should be understood that the Gaussian mixture model is used to accurately quantify things with Gaussian probability density functions (also known as normal distribution curves), and a thing is decomposed into several models based on Gaussian probability density functions.

[0122] In operation S350, a fuzziness matrix is constructed using the Gaussian mixture model, wherein the fuzziness matrix is used to represent the fuzziness of data in the first training sample and the second training sample.

[0123] In an embodiment of the present disclosure, the fuzziness matrix is specifically represented as:

[0124]

[0125] wherein C f is the fuzziness matrix, represents the probability that the predicted value is and the labeled value is y c , and N represents the number of row elements and column elements in the fuzziness matrix.

[0126] That is, in the fuzziness matrix, the element represents the probability that the predicted value is and the labeled value is y c .

[0127] For example, the i-th sample can be represented by respectively, and then is estimated by the following formula:

[0128]

[0129] wherein represents a sample subset with a predicted category n and a labeled category m; represents a sample subset with a predicted category n; represents a sample subset with a labeled category m; and τ represents a fuzziness threshold.

[0130] For example, the constructing of the fuzziness matrix using the Gaussian mixture model specifically includes: for the first confidence loss and the first classification loss, respectively constructing a confidence fuzziness matrix and a classification fuzziness matrix using the Gaussian mixture model.

[0131] It should be noted that, in the modeling of the classification loss and the confidence loss using the GMM model, the confidence loss of all candidate boxes is modeled using the GMM, and only the classification loss of the candidate box containing the target is modeled using the GMM. According to the trained GMM, a class with a higher loss mean is selected, and then the confidence fuzziness fobj and the estimation of the classification ambiguity f cls

[0132]

[0133]

[0134] wherein, represents the predicted value of the i-th bounding box.

[0135] Based on the above-obtained confidence ambiguity f obj and the classification ambiguity f cls , the confidence ambiguity matrix and the classification ambiguity matrix can be estimated respectively.

[0136] For example, for the confidence ambiguity matrix, N = 2; and / or, for the classification ambiguity matrix, N is equal to the total number of classes of the classification output by the target detection model.

[0137] Referring to Figure 4 , in the confidence ambiguity matrix C f_obj , each row can correspond to a predicted value respectively, and each column can correspond to a labeled value respectively. For example, for a sample, the confidence ambiguity matrix C f_obj is a 2*2 matrix, the element in the first row and the first column represents the probability that the predicted value is “yes” and the labeled value is “yes”, the element in the first row and the second column represents the probability that the predicted value is “no” but the labeled value is “yes”, the element in the second row and the first column represents the probability that the predicted value is “yes” but the labeled value is “no”, and the element in the second row and the second column represents the probability that the predicted value is “no” and the labeled value is “no”. That is, for the confidence ambiguity matrix C f_obj of each sample, the elements in the diagonal line represent the case that the predicted value and the labeled value are consistent, i.e., the elements in the diagonal line represent clear data.

[0138] In the classification ambiguity matrix C f_cls , each row can correspond to a predicted classification respectively, and each column can correspond to a labeled classification respectively, which can be represented by class 1, class 2, …, class N respectively. For example, for a sample, the classification ambiguity matrix C f_cls is an N*N matrix, the element in the first row and the first column represents the probability that the predicted value is “class 1” and the labeled value is “class 1”, the element in the first row and the second column represents the probability that the predicted value is “class 1” but the labeled value is “class 2”, and so on, the element in the first row and the Nth column represents the probability that the predicted value is “class 1” but the labeled value is “class N”. The elements of other rows are similar. That is, for the classification ambiguity matrix C f_cls of each sample, the elements in the diagonal line represent the case that the predicted value and the labeled value of the classification are consistent, i.e., the elements in the diagonal line represent clear data.​

[0139] In operation S360, a blur loss function is constructed according to the ambiguity matrix, where the blur loss function is constructed to be able to distinguish clear data and blur data in the first training sample and the second training sample.

[0140] In an embodiment of the present disclosure, the blur loss function is specifically constructed by the following formula (9):

[0141] l f =-(1-r)((1-y * ) 1-r y *r ) γ log(y * )-r((1-y * ) 1-r y *r ) γ log(1-y * ) (9)

[0142] Wherein, l f represents the blur loss function, r represents the proportion of blur data; γ≥0 is a modulation coefficient, used to adjust the sensitivity of the target detection model to data blur.

[0143] Wherein, the proportion r of the blur data is obtained by the following formula (10):

[0144] r=1-diag(C f ) (10)

[0145] Wherein, diag(C f ) represents the diagonal elements of the ambiguity matrix C f .

[0146] It should be noted that the above loss function only describes the case when the label is positive.

[0147] For example, Figure 7 is a schematic diagram of the blur loss function according to an embodiment of the present disclosure. In Figure 7 , the curve of the CE loss function is compared with the curve of the blur loss function according to an embodiment of the present disclosure. From Figure 7As can be seen, for high-confidence samples (for example, samples with a confidence greater than 0.5), the fuzzy loss function can increase the weight of the high-confidence samples; for low-confidence samples (for example, samples with a confidence between 0.05 and 0.5), the fuzzy loss function can reduce the weight of the low-confidence samples; and for very low-confidence samples (for example, samples with a confidence less than 0.05), the fuzzy loss function can correct the label of the sample, that is, change the annotation value of the sample. At the same time, the fuzzy loss function is affected by the parameter fuzzy ratio r. As r increases, the high-confidence interval of the fuzzy loss function will be further weighted, the low-confidence interval will be further reduced, and the very low-confidence interval will be increased. That is, in the embodiments of the present disclosure, the fuzzy loss function can adaptively distinguish fuzzy samples and clear samples according to the fuzzy data ratio, and correct the annotation of the fuzzy samples, so that the model learns clear features, thereby improving the detection performance.

[0148] In operation S370, a second loss function is constructed according to the fuzzy loss function.

[0149] In the embodiments of the present disclosure, the second loss function is specifically calculated by the following formula (11):

[0150] l = σ box ′ · l box + σ obj ′ · l obj + σ cls ′ · l cls (11)

[0151] wherein, l is the second loss function, l box is the second bounding box loss, l obj is the second confidence loss, l cls is the second classification loss, σ box ', σ robj ' and σ cls ' are weight coefficients of the second bounding box loss, the second confidence loss and the second classification loss, respectively.

[0152] For example, the second bounding box loss adopts a GIoU loss, and the second confidence loss and the second classification loss are both expressed by the fuzzy loss function, that is, calculated by the above formula (9).

[0153] By designing the above fuzzy loss function, manual processing of fuzzy data is not required, fuzzy information can be automatically extracted, and fuzzy information is not completely discarded, so that the information can be fully utilized.

[0154] In operation S380, the target detection model is retrained using the first training sample set, the second training sample set and the second loss function.

[0155] For example, the target detection model can be retrained by using a loss function shown in formula (11), and a Stochastic Gradient Descent (SGD) algorithm can be used to update parameters.

[0156] During the training process, iterative updating can be performed. For example, the GMM training step can be returned to again for iterative updating until an ideal index or a maximum training period is reached.

[0157] In the training method of the target detection model according to the embodiments of the present disclosure, the influence of terahertz image feature blurring and annotation blurring can be reduced, so as to improve the terahertz suspect detection rate and suspect classification accuracy of the target detection model and reduce the false positive rate.

[0158] It should be noted that in the embodiments of the present disclosure, the target detection model can be applied to different model structures, including but not limited to first-order / second-order target detection methods or anchor free, transformer, and other detection methods. That is, the training method according to the embodiments of the present disclosure can optimize different model structures.

[0159] In the embodiments of the present disclosure, the improvement of the above-mentioned training method on the actual application effect is mainly concentrated in the training stage, and has little influence on resource occupation and detection speed in the model application test process.

[0160] The embodiments of the present disclosure also provide a target detection method, which can include the following steps: obtaining image data; inputting the image data into a target detection model, wherein the target detection model is trained by the method as described above; and determining a target to be detected in the image data according to the output of the target detection model.

[0161] Based on the training method of the target detection model, the embodiments of the present disclosure also provide a training device of a target detection model. The following will be described in detail in combination with Figure 8 The device is described in detail.

[0162] Figure 8 is a structural block diagram of a training device of a target detection model according to an exemplary embodiment of the present disclosure.

[0163] As Figure 8 shown, the training device 700 of the target detection model includes a training sample acquisition module 710, a target detection module 720, a blurring degree matrix construction module 730, a blurring loss function construction module 740, a loss function construction module 750, and a retraining module 760.

[0164] The training sample obtaining module 710 is configured to obtain a training sample set, the training sample set comprising first training samples and second training samples, the first training samples and the second training samples being respectively used to represent inputs and outputs of the target detection model, the first training samples comprising clear data and fuzzy data, and the second training samples comprising clear data and fuzzy data. In some exemplary embodiments, the training sample obtaining module 710 can be configured to perform the operation S310 described above, and details are not repeated here.

[0165] The target detection module 720 is configured to receive the first training samples, and determine a first loss value of the target detection model by using a first loss function according to output values of the target detection model and the second training samples. In some exemplary embodiments, the target detection module 720 can be configured to perform the operations S320 and S330 described above, and details are not repeated here.

[0166] The blur matrix constructing module 730 comprises a Gaussian mixture model, and the blur matrix constructing module 730 is configured to receive the first loss value, and construct a blur matrix by using the Gaussian mixture model, wherein the blur matrix is used to represent blur degrees of data in the first training samples and the second training samples. In some exemplary embodiments, the blur matrix constructing module 730 can be configured to perform the operations S340 and S350 described above, and details are not repeated here.

[0167] The blur loss function constructing module 740 is configured to construct a blur loss function according to the blur matrix, wherein the blur loss function is constructed to be able to distinguish clear data and fuzzy data in the first training samples and the second training samples. In some exemplary embodiments, the blur loss function constructing module 740 can be configured to perform the operation S360 described above, and details are not repeated here.

[0168] The loss function constructing module 750 is configured to construct a second loss function according to the blur loss function. In some exemplary embodiments, the loss function constructing module 750 can be configured to perform the operation S370 described above, and details are not repeated here.

[0169] The retraining module 760 is configured to retrain the target detection model by using the first training sample set, the second training sample set and the second loss function. In some exemplary embodiments, the retraining module 760 can be configured to perform the operation S380 described above, and details are not repeated here.

[0170] According to an embodiment of the present disclosure, any of the training sample obtaining module 710, the target detection module 720, the blur matrix constructing module 730, the blur loss function constructing module 740, the loss function constructing module 750 and the retraining module 760 can be combined in one module, or any of the modules can be split into multiple modules. Alternatively, at least part of the function of one or more of the modules can be combined with at least part of the function of other modules, and implemented in one module. According to an embodiment of the present disclosure, at least one of the training sample obtaining module 710, the target detection module 720, the blur matrix constructing module 730, the blur loss function constructing module 740, the loss function constructing module 750 and the retraining module 760 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable manner of integrating or packaging a circuit, etc. hardware or firmware, or in any one of software, hardware and firmware three implementation ways or in any appropriate combination of any of them. Alternatively, at least one of the training sample obtaining module 710, the target detection module 720, the blur matrix constructing module 730, the blur loss function constructing module 740, the loss function constructing module 750 and the retraining module 760 can be at least partially implemented as a computer program module which can perform corresponding functions when the computer program module is run.

[0171] Figure 9 A structural block diagram of an electronic device suitable for implementing a training method of a target detection model or a target detection method according to an example embodiment of the present disclosure is schematically shown.

[0172] As Figure 9 shown, the electronic device 800 according to an embodiment of the present disclosure includes a processor 801 which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 can include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset, and / or a special-purpose microprocessor (such as an application specific integrated circuit (ASIC)), and the like. The processor 801 can also include an on-board memory for cache use. The processor 801 can include a single processing unit or a plurality of processing units for performing different actions of the method processes according to an embodiment of the present disclosure.

[0173] For example, the electronic device can be a passive terahertz imaging device.

[0174] In the RAM 803, various programs and data required for the operation of the electronic device 800 are stored. The processor 801, the ROM 802, and the RAM 803 are connected to each other via the bus 804. The processor 801 performs various operations of the method flow according to the embodiments of the present disclosure by executing the programs in the ROM 802 and / or the RAM 803. It should be noted that the programs can also be stored in one or more memories other than the ROM 802 and the RAM 803. The processor 801 can also perform various operations of the method flow according to the embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0175] According to an embodiment of the present disclosure, the electronic device 800 can further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 can further include one or more of the following components connected to the I / O interface 805: an input part 806 including a keyboard, a mouse, and the like; an output part 807 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage part 808 including a hard disk, and the like; and a communication part 809 including a network interface card such as a LAN card, a modem, and the like. The communication part 809 performs communication processing via a network such as the Internet. A drive 88 is also connected to the I / O interface 805 as necessary. A removable recording medium 811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is attached to the drive 88 as necessary, so that a computer program read out therefrom is installed in the storage part 808 as necessary.

[0176] The present disclosure also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, when the one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.

[0177] According to an embodiment of the present disclosure, the computer readable storage medium can be a nonvolatile computer readable storage medium, for example, can include, but is not limited to, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM or Flash memory), a portable compact disc read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, the computer readable storage medium can include the ROM 802 and / or the RAM 803 described above and / or one or more memory other than the ROM 802 and the RAM 803.

[0178] Embodiments of the present disclosure also include a computer program product, which includes a computer program containing program codes for executing the methods shown in the flowcharts. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the item recommendation method provided by the embodiments of the present disclosure.

[0179] The above-described functions defined in the system / device of the embodiments of the present disclosure are performed when the computer program is executed by the processor 801. According to an embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by computer program modules.

[0180] In one embodiment, the computer program can rely on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program can also be transmitted, distributed, and downloaded in the form of a signal on a network medium, and be downloaded and installed through the communication part 809, and / or be installed from the detachable medium 811. The program codes contained in the computer program can be transmitted by any appropriate network medium, including but not limited to wireless, wired, etc., or any appropriate combination thereof.

[0181] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or be installed from the detachable medium 811. When the computer program is executed by the processor 801, the above-described functions defined in the system of the embodiments of the present disclosure are performed. According to an embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0182] According to embodiments of the present disclosure, program code of the computer program for performing the methods provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages, and can be implemented in a computer program product. Specifically, the computer program can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. The programming language includes, but is not limited to, Java, C++, python, “C” language, or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, and partly on a remote computing device, or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider (ISP).

[0183] The computer program product of the present disclosure can be a computer program product, which is a machine-readable medium (or computer readable medium) having stored therein a sequence of instructions readable by a machine (e.g., a computer). The instructions in the machine-readable medium focus on the function of the computer program (e.g., in the form of a computer program product). The instructions can be executed by one or more processors to produce a computer-implemented process, such as the one(s) described above. The instructions can be stored in a computer memory, which can be a single memory or spread out over multiple memories. The machine-readable medium can be a magnetic or optical disk, a magnetic tape, a semiconductor memory, a computer disk, or any other suitable medium. The machine-readable medium can be loaded into the computer memory, or the computer memory can access the machine-readable medium through a connection, such as a communication network. The instructions represent examples of the program code means adapted to carry out the steps of the disclosed methods.

[0184] Those skilled in the art will understand that features of the various embodiments and / or claims of the present disclosure can be combined or / and integrated with one another, even though such a combination or integration is not expressly mentioned in the present disclosure. In particular, the features of the various embodiments and / or claims of the present disclosure can be combined or / and integrated with one another without departing from the spirit and teachings of the present disclosure. All such combinations and / or integrations are within the scope of the present disclosure.

[0185] The above describes embodiments of the present disclosure. However, these embodiments are merely for illustrative purposes, and are not intended to limit the scope of the present disclosure. Although each embodiment is described above separately, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Those skilled in the art can make various substitutions and modifications without departing from the scope of the present disclosure, and these substitutions and modifications should all fall within the scope of the present disclosure.

Claims

1. A method for training a target detection model based on fuzzy learning, characterized in that, The method comprises: obtaining a training sample set, the training sample set comprising first training samples and second training samples, the first training samples and the second training samples being respectively used to represent inputs and outputs of the target detection model, the first training samples comprising clear data and fuzzy data, and the second training samples comprising clear data and fuzzy data; inputting the first training samples into the target detection model; determining a first loss value of the target detection model by using a first loss function according to an output value of the target detection model and the second training samples; inputting the first loss value into a Gaussian mixture model; constructing a fuzziness matrix by using the Gaussian mixture model, wherein the fuzziness matrix is used to represent fuzziness of data in the first training samples and the second training samples; constructing a fuzzy loss function according to the fuzziness matrix, wherein the fuzzy loss function is constructed to be able to distinguish clear data and fuzzy data in the first training samples and the second training samples; constructing a second loss function according to the fuzzy loss function; and retraining the target detection model by using the first training sample set, the second training sample set and the second loss function.

2. The method of claim 1, wherein, The first loss function comprises a first bounding box loss, a first confidence loss and a first classification loss.

3. The method of claim 2, wherein, The first bounding box loss adopts a GIoU loss, the first confidence loss adopts a mixed entropy loss, and the first classification loss adopts a mixed entropy loss.

4. The method of claim 3, wherein, The first loss function is specifically calculated by the following formula: wherein, is a first loss function, is a first bounding box loss, is a first confidence loss, is a first classification loss, , and are weight coefficients of the first bounding box loss, the first confidence loss, and the first classification loss, respectively.

5. The method of claim 4, wherein, The mixed entropy loss comprises a cross-entropy loss and a negative entropy loss.

6. The method of claim 5, wherein, The mixed entropy loss is specifically calculated by the following formula: wherein, denotes a mixed entropy loss, and denotes a cross-entropy loss and a negative entropy loss, respectively, is a weight coefficient of the negative entropy loss; is a value in the second training sample, denoting a labeled value; is an output value of the target detection model, denoting a predicted value.

7. The method of claim 6, wherein, The fuzziness matrix is specifically represented by the following formula: wherein, is a matrix of ambiguities, is a reference value, is a predicted value, denotes the probability that the predicted value is the reference value is denotes the number of row elements and column elements in the ambiguity matrix.​ 8. The method of claim 7, wherein, The fuzzy loss function is specifically represented by the following formula: wherein, represents a blur loss function, represents a proportion of blurred data; is a modulation coefficient for adjusting the sensitivity of the target detection model to data blur.

9. The method of claim 2, wherein, The constructing of the fuzziness matrix by using the Gaussian mixture model specifically comprises: for the first confidence loss and the first classification loss, a confidence fuzziness matrix and a classification fuzziness matrix are respectively constructed by using the Gaussian mixture model.

10. The method of any one of claims 2-9, wherein, The second loss function is specifically calculated by the following formula: wherein, is a second loss function, is a second bounding box loss, is a second confidence loss, is a second classification loss, , and are weight coefficients of the second bounding box loss, the second confidence loss and the second classification loss, respectively, both of which are expressed by the fuzzy loss function.

11. The method of claim 10, wherein, The second bounding box loss adopts a GIoU loss.

12. The method of claim 11, wherein, For the confidence ambiguity matrix, = 2; for the classification ambiguity matrix, is equal to the total number of classes of the classification output by the target detection model.

13. The method of any one of claims 1-9, wherein, The first training samples comprise image data obtained by scanning an imaging region by a terahertz imaging device; The second training samples comprise image data with a labeled box, wherein the labeled box is used to represent a fuzzy position and / or type of a target to be detected in the image data.

14. A target detection method characterized by, The method comprises: obtaining image data; inputting the image data into a target detection model, wherein the target detection model is trained by the method according to any one of claims 1-13; and determining a target to be detected in the image data according to an output of the target detection model. 15.A device for training a target detection model based on fuzzy learning, characterized in that, The method comprises: a training sample obtaining module, configured to obtain a training sample set, the training sample set comprising first training samples and second training samples, the first training samples and the second training samples being respectively used to represent inputs and outputs of the target detection model, the first training samples comprising clear data and fuzzy data, and the second training samples comprising clear data and fuzzy data; a target detection module, configured to receive the first training sample, and determine a first loss value of the target detection model according to an output value of the target detection model and the second training sample by using a first loss function; a fuzzy degree matrix construction module, the fuzzy degree matrix construction module comprising a Gaussian mixture model, the fuzzy degree matrix construction module being configured to receive the first loss value, and construct a fuzzy degree matrix by using the Gaussian mixture model, wherein the fuzzy degree matrix is used to represent fuzzy degrees of data in the first training sample and the second training sample; a fuzzy loss function construction module, configured to construct a fuzzy loss function according to the fuzzy degree matrix, wherein the fuzzy loss function is constructed to be able to distinguish clear data and fuzzy data in the first training sample and the second training sample; a loss function construction module, configured to construct a second loss function according to the fuzzy loss function; and a retraining module, configured to retrain the target detection model by using the first training sample set, the second training sample set and the second loss function.

16. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are caused to perform the method according to any one of claims 1-14.

17. The electronic device of claim 16, wherein, The electronic device is a passive terahertz imaging device.

18. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1-14.

19. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-14.

Images