Contraband detection model training method and contraband detection method
Through the method of feature stitching and detection results optimization of sample images, the problems of low efficiency and low accuracy caused by invalid samples in deep learning model training are solved, and the training efficiency and accuracy of contraband detection models are improved.
Patent Information
- Application Number
- CN202411426897.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-14
- Publication Date
- 2024-11-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
There are a large number of invalid samples during the training process of deep learning models in contraband detection, resulting in low training efficiency and low accuracy and lack of targeting.
By obtaining the backbone network features of the sample image, performing stitching processing, reducing the number of iterations of the training process, model training is carried out based on the stitching image, and updating the sample data and loss function according to the detection results until the detection results are consistent, so as to improve training efficiency and targetedness .
The training efficiency and accuracy of the contraband detection model are improved, the impact of invalid samples is reduced, and the targetedness and effectiveness of the model are enhanced.
Smart Images

Figure CN118941565A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to image processing technology, and more particularly to a contraband detection model training method and a contraband detection method. Background Art
[0002] In many scenarios, packages need to be inspected to prevent them from containing prohibited items.
[0003] Manual identification of contraband is inefficient and cannot guarantee accuracy. With the development of deep learning, it is now possible to automatically identify contraband through deep learning.
[0004] However, there will be a large number of invalid samples in the training process of deep learning models, which will have a negative impact on the training, low training efficiency, and lack of targeted training, resulting in low accuracy of the trained model. Summary of the invention
[0005] The present application provides a contraband detection model training method and a contraband detection method to improve the efficiency and pertinence of contraband detection model training, improve the accuracy of the trained model, and improve the accuracy of contraband detection.
[0006] In a first aspect, an embodiment of the present application provides a contraband detection model training method, the contraband detection model training method comprising:
[0007] Obtain backbone network features of sample images; the sample images include contraband labels;
[0008] According to the characteristics of the backbone network, the sample images are spliced to obtain a spliced image;
[0009] The first round of training of the contraband detection model is performed based on the spliced images to obtain the model prediction results of the sample images;
[0010] Determine the detection result of the sample image according to the contraband label of the sample image and the model prediction result; the detection result includes consistency, missed detection and false detection;
[0011] If there are sample images whose detection results are inconsistent, the sample data and loss function for the next round of training are updated according to the detection results of each sample image, and the aforementioned training process is repeated according to the updated sample data and loss function until the detection results of all sample images are consistent.
[0012] In a second aspect, an embodiment of the present application provides a method for detecting contraband, the method comprising:
[0013] Acquire the image to be detected;
[0014] The image to be detected is input into a contraband detection model to obtain a contraband detection result of the image to be detected; wherein the contraband detection model is trained based on the training method of the contraband detection model of any embodiment in the present application.
[0015] In a third aspect, the embodiment of the present application further provides a contraband detection model training device, the contraband detection model training device comprising:
[0016] A backbone network feature acquisition module, used to acquire backbone network features of a sample image; the sample image includes a contraband label;
[0017] A sample image stitching module is used to stitch the sample images according to the characteristics of the backbone network to obtain a stitched image;
[0018] A model prediction result determination module is used to perform a first round of training on the contraband detection model based on the spliced image to obtain the model prediction result of the sample image;
[0019] A detection result determination module is used to determine the detection result of the sample image according to the contraband label of the sample image and the model prediction result; the detection result includes consistency, missed detection and false detection;
[0020] The model adjustment training module is used to update the sample data and loss function for the next round of training according to the detection results of each sample image if the detection results of the sample images are inconsistent, and repeat the above training process according to the updated sample data and loss function until the detection results of all sample images are consistent.
[0021] In a fourth aspect, an embodiment of the present application further provides a contraband detection device, the contraband detection device comprising:
[0022] An image acquisition module to be detected is used to acquire the image to be detected;
[0023] The contraband detection module is used to input the image to be detected into the contraband detection model to obtain the contraband detection result of the image to be detected; wherein the contraband detection model is trained based on the training method of the contraband detection model of any embodiment in the present application.
[0024] In a fifth aspect, an embodiment of the present application further provides an electronic device, the electronic device comprising:
[0025] one or more processors;
[0026] A storage device for storing one or more programs;
[0027] When one or more programs are executed by one or more processors, the one or more processors implement any contraband detection model training method provided in the embodiments of the present application, or any contraband detection method provided in the embodiments of the present application.
[0028] In a sixth aspect, an embodiment of the present application further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute any one of the contraband detection model training methods provided in the embodiments of the present application, or any one of the contraband detection methods provided in the embodiments of the present application.
[0029] The present application obtains the backbone network features of the sample image; the sample image includes a contraband label; features are extracted from the sample image; according to the backbone network features, the sample images are spliced to obtain a spliced image; the number of iterations of the training process is reduced to improve the training efficiency; the first round of training of the contraband detection model is performed based on the spliced image to obtain the model prediction result of the sample image; the detection result of the sample image is determined according to the contraband label of the sample image and the model prediction result; the detection result includes consistency, missed detection and false detection; if the detection results of the sample images are inconsistent, the sample data and loss function of the next round of training are updated according to the detection results of each sample image, and the above training process is repeated according to the updated sample data and loss function until the detection results of all sample images are consistent, the sample data of the next round of training is updated according to the detection results of each sample image, the pertinence of the samples of the next round of training is ensured, invalid samples are reduced, and the training efficiency is improved, the loss function of the next round of training is updated according to the detection results of each sample image, and the learning ability of the current training is adjusted in a targeted manner to improve the pertinence and effectiveness of the training and the accuracy of the trained model. Therefore, the technical solution of the present application solves the problem that a large number of invalid samples exist in the training process of the deep learning model, which has a negative impact on the training, low training efficiency, lack of targeted training, and leads to low accuracy of the trained model. The efficiency and targetedness of the contraband detection model training and the accuracy of the trained model are improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 is a flow chart of a contraband detection model training method in Example 1 of the present application;
[0031] Figure 2 is a flow chart of a contraband detection model training method in Embodiment 2 of the present application;
[0032] Figure 3 It is a flow chart of a contraband detection model training method in Embodiment 3 of the present application;
[0033] Figure 4is a flow chart of a contraband detection method in Embodiment 4 of the present application;
[0034] Figure 5 It is a structural schematic diagram of a contraband detection model training device in Embodiment 5 of the present application;
[0035] Figure 6 It is a structural schematic diagram of a contraband detection device in Embodiment 6 of the present application;
[0036] Figure 7 It is a structural diagram of an electronic device in Embodiment 7 of the present application. DETAILED DESCRIPTION
[0037] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.
[0038] It should be noted that the terms "first" and "second" etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0039] Embodiment 1
[0040] Figure 1 This is a flow chart of a method for training a contraband detection model provided in Example 1 of the present application. This embodiment is applicable to the situation of training a contraband detection model. The method can be executed by a contraband detection model training device, which can be implemented in software and / or hardware and is specifically configured in an electronic device where the contraband detection model is located.
[0041] See also Figure 1 The contraband detection model training method shown in the figure specifically includes the following steps:
[0042] S110, obtaining backbone network features of a sample image; the sample image includes a contraband label.
[0043] The sample image may be a sample for training a contraband detection model. For example, the sample image may be an x-ray scan image of a package to be identified. The sample image includes a contraband label, which is manually annotated and includes the type and location of the contraband.
[0044] The backbone network features may be features extracted by the backbone network in the contraband detection model. The sample image is input into the backbone network in the initial contraband detection model to obtain the backbone network features. The contraband detection model may be a network model for identifying contraband. Exemplarily, the contraband detection model may be a neural network model.
[0045] S120: splicing the sample images according to the backbone network characteristics to obtain a spliced image.
[0046] The stitching may be to arrange multiple images according to a certain spatial position and combine them into one image. For example, four images may be stitched into one stitched image. The stitched image may be a stitched image used to train the contraband detection model.
[0047] For example, the backbone network features can be classified according to certain classification principles, and sample images of the same type can be spliced to obtain a spliced image, thereby reducing the contrast between the spliced images. For example, the sample images can be clustered according to the similarity between the sample images to obtain multiple groups of clustered sample images, and the sample images clustered into one group can be spliced to obtain a spliced image.
[0048] In an optional embodiment, sample images are spliced according to the backbone network characteristics to obtain a spliced image, including: determining the similarity of each sample image according to the backbone network characteristics; clustering each sample image according to the similarity to obtain multiple groups of similar sample groups; and splicing the sample images in each similar sample group to obtain a spliced image.
[0049] The similarity of each sample image can be calculated by a similarity formula. Exemplarily, the similarity of each sample image can be determined by calculating the cosine similarity of each sample image. For example, for each sample image, the backbone feature of the sample image is matrix-dot-multiplied with the backbone feature of another sample image to obtain the similarity of each sample image with the other sample images.
[0050] Clustering is performed according to the similarity of each sample image to obtain multiple groups of similar sample groups. Exemplarily, clustering can be performed by K-mean clustering (a professional term, a clustering method) method to obtain multiple groups of similar sample groups. The number of sample images in the similar sample group can be preset. For example, the number of sample images in the similar sample group can be 4. The sample images in each similar sample group are spliced according to a preset arrangement to obtain a spliced image. Exemplarily, the arrangement can be a four-square grid, a nine-square grid, or a horizontal parallel arrangement, etc., which is not specifically limited in this application. For example, the number of sample images in the similar sample group can be 4, and the arrangement can be a four-square grid. The splicing process can be to reduce each sample image to a quarter of the original size, and splice the four sample images in the form of a four-square grid to obtain a spliced image.
[0051] The similarity of each sample image is determined according to the characteristics of the backbone network; clustering is performed according to the similarity of each sample image to obtain multiple groups of similar sample groups; the sample images in each similar sample group are spliced to obtain a spliced image, in which the backbone network characteristics of each sample image are similar, thereby reducing the contrast between the spliced images and avoiding data jumps. By training with spliced images, the number of training iterations can be reduced without losing the characteristics of contraband, the speed of each round of iteration can be accelerated, and the efficiency of training can be improved.
[0052] S130: Perform a first round of training on the contraband detection model based on the spliced image to obtain a model prediction result of the sample image.
[0053] The spliced image is input into the contraband detection model for the first round of training, including forward propagation and back propagation, and the output is the model prediction result of the sample image. The model prediction result includes the type and location of the contraband.
[0054] Forward propagation can obtain the loss value, prediction result and backbone network features of the forward propagation backbone network for each spliced image. The loss value obtained by forward propagation is back-propagated to update the weights of the contraband detection model. When the sample images are spliced, splicing information can be formed. The splicing information can include the image area, image name and contraband identification of each sample image. Based on the splicing information, the prediction result of the spliced image and the feature value of the forward propagation backbone network can be mapped to a single sample image to obtain the prediction result and backbone network features of a single sample image in the spliced image. Obtaining the backbone network features can facilitate the direct acquisition of the backbone network features in the next round of training, thereby reducing the amount of calculation of the contraband detection model.
[0055] S140, determining the detection result of the sample image according to the contraband label of the sample image and the model prediction result; the detection result includes consistency, missed detection and false detection.
[0056] If the prohibited item label of the sample image is consistent with the prohibited item type of the model prediction result and the position overlap meets the preset requirements, it can be considered that the prohibited item is accurately detected, and the detection result is consistent; if the prohibited item type is consistent but the position overlap does not meet the preset requirements, the detection result is missed detection. If an object other than the prohibited item corresponding to the prohibited item label is detected, the detection result is a false detection, and if the prohibited item corresponding to the prohibited item label is not detected, the detection result is missed detection.
[0057] S150. If there are sample images whose detection results are inconsistent, the sample data and loss function for the next round of training are updated according to the detection results of each sample image, and the aforementioned training process is repeated according to the updated sample data and loss function until the detection results of all sample images are consistent.
[0058] If the detection results of sample images are inconsistent, that is, there is at least one of missed detection and false detection, then the sample data and loss function for the next round of training are updated according to the detection results of each sample image, and the sample images with missed detection and false detection are used as training samples for the next round. The contraband detection model is trained in a targeted manner to improve the accuracy of the contraband detection model.
[0059] The loss function can be used to adjust the learning ability of the contraband detection model. According to the detection results of each sample image, the loss function of the next round of training is updated, and targeted training is performed for the lack of learning ability in the current round. For example, when there is a category misdetection, the weight of the category loss function can be increased, and when there is a position misdetection, the weight of the coordinate loss function can be increased.
[0060] The aforementioned training process S110 to S140 is repeated according to the updated sample data and loss function until the detection results of all sample images are consistent, the model converges, and the training is completed.
[0061] The technical solution of this embodiment is as follows: the backbone network features of the sample image are obtained; the sample image includes a contraband label; the features of the sample image are extracted; the sample images are spliced according to the backbone network features to obtain a spliced image; the number of iterations of the training process is reduced to improve the training efficiency; the contraband detection model is trained for the first round based on the spliced image to obtain the model prediction result of the sample image; the detection result of the sample image is determined according to the contraband label of the sample image and the model prediction result; the detection result includes consistency, missed detection and false detection; if the detection results of the sample images are inconsistent, the sample data and loss function of the next round of training are updated according to the detection results of each sample image, and the above training process is repeated according to the updated sample data and loss function until the detection results of all sample images are consistent, the sample data of the next round of training is updated according to the detection results of each sample image, the pertinence of the samples of the next round of training is ensured, invalid samples are reduced, and the training efficiency is improved, the loss function of the next round of training is updated according to the detection results of each sample image, and the learning ability of the current training is adjusted in a targeted manner to improve the pertinence and effectiveness of the training and the accuracy of the trained model. Therefore, the technical solution of the present application solves the problem that a large number of invalid samples exist in the training process of the deep learning model, which has a negative impact on the training, low training efficiency, lack of targeted training, and leads to low accuracy of the trained model. The efficiency and targetedness of the contraband detection model training and the accuracy of the trained model are improved.
[0062] Embodiment 2
[0063] Figure 2 This is a flow chart of a contraband detection model training method provided in Example 2 of the present application. The technical solution of this embodiment is further refined on the basis of the above technical solution.
[0064] Furthermore, “determine the detection result of the sample image based on the contraband label of the sample image and the model prediction result” is refined as: “match the contraband label for each model prediction result. If the contraband label is matched, determine whether the contraband label of the sample image is consistent with the model prediction result; otherwise, determine the label remaining amount and the predicted remaining amount based on the contraband label and the model prediction result; determine the detection result of the sample image based on the label remaining amount and the predicted remaining amount” to determine the detection result of the sample image.
[0065] See also Figure 2 A contraband detection model training method shown includes:
[0066] S210, obtaining backbone network features of a sample image; the sample image includes a contraband label.
[0067] S220: splicing the sample images according to the backbone network characteristics to obtain a spliced image.
[0068] S230: Perform a first round of training on the contraband detection model based on the spliced image to obtain a model prediction result of the sample image.
[0069] S240, matching the prohibited item label for each model prediction result, and if the prohibited item label is matched, determining whether the prohibited item label of the sample image is consistent with the model prediction result.
[0070] After obtaining the prediction results of each model, match the prohibited item label for each predicted contraband in the prediction results. The same sample image may include multiple contraband, and accordingly, the model prediction results and contraband labels in the same sample image may be multiple. To match the prohibited item label for each model prediction result, all model prediction results and contraband labels may be traversed to determine whether there is a contraband label of the same type as the model prediction result. If so, further determine whether the contraband label of the sample image is consistent with the model prediction result to determine the detection result. When the model prediction results or contraband labels are traversed, the matching ends.
[0071] In an optional embodiment, determining whether the contraband label of the sample image is consistent with the model prediction result includes: determining the intersection-over-union ratio of the contraband label and the model prediction result; determining whether the intersection-over-union ratio is greater than a preset intersection-over-union ratio threshold; if so, determining that the detection result is consistent, otherwise, determining that the detection result is a missed detection.
[0072] Consistent is used to indicate that the prohibited item label of the sample image and the model prediction result are of the same type, and the overlap of the position range meets the preset threshold. Therefore, to determine whether the prohibited item label of the sample image and the model prediction result are consistent, it is necessary to determine whether the prohibited item label and the model prediction result are of the same type, and then determine whether the overlap of the position range of the prohibited item label and the model prediction result meets the preset threshold.
[0073] In target detection, the intersection-and-union ratio can be used to measure the degree of overlap between the predicted box and the true box. The larger the intersection-and-union ratio, the higher the degree of overlap between the two bounding boxes. After matching the model prediction result with a contraband label of the same type, it is necessary to use the intersection-and-union ratio to determine whether the contraband label and the model prediction result are consistent. The intersection-and-union ratio threshold may be a threshold determined in advance based on experience or experiments, and this application does not make specific limitations on this. For example, the intersection-and-union ratio threshold may be 0.5. If the intersection-and-union ratio is greater than the preset intersection-and-union ratio threshold, the detection result is determined to be consistent, otherwise, the detection result is determined to be missed.
[0074] By determining the intersection-and-union ratio of the contraband label and the model prediction result; judging whether the intersection-and-union ratio is greater than the preset intersection-and-union ratio threshold; if so, determining that the detection result is consistent, otherwise, determining that the detection result is a missed detection, and further determining whether the model prediction result is accurate through the intersection-and-union ratio, thereby improving the accuracy of the judgment of the detection result.
[0075] S250: Otherwise, determine the remaining quantity of the label and the predicted remaining quantity according to the contraband label and the model prediction result.
[0076] Otherwise, the model prediction result does not match the contraband label of the same type. The remaining label quantity is used to indicate the number of contraband labels that are not matched to the model prediction result. The remaining prediction quantity is used to indicate the number of model prediction results that are not matched to the contraband label. If the model prediction results are traversed, the number of remaining contraband labels is counted to obtain the remaining label quantity; if the contraband labels are traversed, the number of remaining model prediction results is counted to obtain the remaining prediction quantity.
[0077] S260: Determine the detection result of the corresponding position of the sample image according to the label remaining amount and the predicted remaining amount.
[0078] If the remaining number of tags is greater than zero, the detection result of the corresponding position of the remaining contraband tags is determined to be a missed detection. If the predicted remaining number is greater than zero, the detection result of the corresponding position of the remaining model prediction result is determined to be a false detection.
[0079] Optionally, perform threshold calculation for each category of contraband: count the detection rate of each category of contraband; sort the model prediction results of the same category from large to small according to the confidence of the model prediction results; update the confidence threshold based on the detection rate threshold, sorting results and detection rate.
[0080] The confidence level is the confidence level of the output of the contraband detection model, that is, the confidence level of each model prediction result. The detection rate can be the ratio of the model prediction results with consistent detection results to the prediction results of all models in this category, which can be used to identify the accuracy probability of detecting contraband in this category. Exemplarily, the confidence threshold can be 0.8.
[0081] The model prediction results of the same category are sorted from large to small according to the confidence of the model prediction results. If the detection rate determined according to the confidence threshold is greater than the detection rate threshold, there is no need to update the confidence threshold. Otherwise, the confidence threshold is updated according to the detection rate threshold, the sorting result and the detection rate. The model prediction results of the same category are sorted from large to small according to the confidence of the model prediction results, and the minimum confidence value that meets the detection rate is determined. If the minimum confidence value is less than the confidence threshold, the minimum confidence value is set as the confidence threshold, otherwise the confidence threshold is not modified.
[0082] According to the confidence threshold, the model prediction results of each training set image are traversed. If the false detection and missed detection of this sample image are 0, this sample image and all the detected contraband in this sample image are marked as ready for deletion, otherwise this image is recorded as the sample data for the next round of model training. After the traversal is completed, the number of ready-to-delete images is greater than the preset deletion probability threshold, and the process returns to S240-S260, and the detection rate threshold is adjusted first, then the intersection-over-union ratio threshold is adjusted, and finally the confidence threshold is adjusted until the ready-to-delete data is less than the preset deletion probability threshold. Exemplarily, the preset deletion probability threshold can be set to one quarter.
[0083] S270. If there are sample images whose detection results are inconsistent, the sample data and loss function for the next round of training are updated according to the detection results of each sample image, and the aforementioned training process is repeated according to the updated sample data and loss function until the detection results of all sample images are consistent.
[0084] The technical solution of this embodiment matches the prohibited goods label for each model prediction result. If the prohibited goods label is matched, it is determined whether the prohibited goods label of the sample image is consistent with the model prediction result; otherwise, the label remaining amount and the predicted remaining amount are determined according to the prohibited goods label and the model prediction result; the detection result of the sample image is determined according to the label remaining amount and the predicted remaining amount, and the detection result of each model prediction result in the sample image is accurately determined, so as to facilitate the subsequent targeted training of the contraband detection model according to the detection results.
[0085] Embodiment 3
[0086] Figure 3 This is a flow chart of a contraband detection model training method provided in Example 3 of the present application. The technical solution of this embodiment is further refined on the basis of the above technical solution.
[0087] Furthermore, “if there are sample images with inconsistent detection results, update the sample data and loss function for the next round of training according to the detection results of each sample image” is refined as: “delete the sample images with consistent detection results, and perform target enhancement on the sample images with missed detection and false detection results to obtain updated sample data for the next round of training; update the loss function for the next round of training according to the model prediction results and the corresponding detection results” to update the sample images and loss function for the next round of training.
[0088] See also Figure 3 A contraband detection model training method shown includes:
[0089] S310, obtaining backbone network features of a sample image; the sample image includes a contraband label.
[0090] S320: splicing the sample images according to the backbone network characteristics to obtain a spliced image.
[0091] S330: Perform a first round of training on the contraband detection model based on the spliced image to obtain a model prediction result of the sample image.
[0092] S340, determining the detection result of the sample image according to the contraband label of the sample image and the model prediction result; the detection result includes consistency, missed detection and false detection.
[0093] S350, deleting sample images with consistent detection results, and performing target enhancement on sample images with missed detection and false detection results, to obtain updated sample data for the next round of training.
[0094] For sample images with consistent detection results, it can be considered that the contraband detection model has accurately learned all the knowledge in the sample image in the current round of training. Therefore, the sample images with consistent detection results are deleted to reduce invalid training data in the next round of training.
[0095] Target enhancement can be to enhance the image of the area in the sample image where the detection result is missed detection or false detection, so as to strengthen the learning of the contraband detection model in the next round of training. For example, the area in the sample image where the detection result is missed detection or false detection can be copied to the position without contraband in other images to obtain a new sample image.
[0096] The formula for the copy operation is as follows:
[0097] ;
[0098] Among them, img_new represents the new pixel value of the spliced image area, fn represents missed detection, and fp represents false detection. Indicates the target pixel value of fn or fp, Indicates the pixel value of the stitched image area.
[0099] S360: Update the loss function for the next round of training based on the model prediction results and the corresponding detection results.
[0100] The reasons for missed detection and false detection are determined based on the model prediction results and the corresponding detection results, and the loss function of the next round of training is updated according to the reasons for missed detection and false detection, so as to strengthen the learning ability of the strong model for contraband detection on false detection and missed detection areas. Exemplarily, the reasons for missed detection and false detection may include category detection errors and target detection errors. Category detection errors may include non-detection and detection errors, and correspondingly, target detection errors may also include non-detection and detection errors.
[0101] In an optional embodiment, the loss function for the next round of training is updated based on the model prediction results and the corresponding detection results, including: if the model prediction result is that it does not match the target whose detection result is missed or misdetected, then the weights of the category loss function and the coordinate loss function of the loss function for the next round of training are updated.
[0102] If the model predicts that the target is not matched and the detection result is missed or misdetected, the weights of the category loss function and the coordinate loss function of the loss function for the next round of training are updated according to the following formula:
[0103] ;
[0104] in, Represents the category loss function, and the cross entropy loss function can be used. Represents the coordinate loss function, using the l1_smmoth (professional term, a loss function) loss function.
[0105] If the model predicts that the target is not matched and the detection result is missed or misdetected, the weights of the category loss function and the coordinate loss function of the loss function of the next round of training are updated, which can strengthen the learning of the category or coordinates of the unmatched missed or misdetected targets.
[0106] In an optional embodiment, the loss function for the next round of training is updated according to the model prediction result and the corresponding detection result, including: if the model prediction result is to match the target whose detection result is a missed detection, then according to the missed detection type, the weights of the category loss function and the coordinate loss function of the loss function for the next round of training are updated; if the model prediction result is to match the target whose detection result is a false detection, then the loss function for the next round of training is updated to the category loss function.
[0107] If the model prediction result matches the contraband target with missed detection attributes, different weight calculations need to be loaded for the category and coordinates respectively, which can be divided into the following four cases:
[0108] When the categories are consistent, the intersection-over-union matching value is greater than the intersection-over-union threshold, and the confidence is less than the confidence threshold, the model's learning ability for category confidence is increased, and the loss function is updated to the following formula:
[0109]
[0110] ;
[0111] Among them, p represents the category confidence of the target, represents the category loss obtained in this case, represents the total loss obtained in this case, Represents the regression loss, calculated by the l1_smmoth function.
[0112] When the categories are consistent and the IoU is greater than the IoU threshold, the model's ability to learn coordinates is increased, and the loss function is updated to the following formula:
[0113] ;
[0114] ;
[0115] Among them, iou represents the intersection-over-union ratio of the true target and the predicted target, Represents the regression loss, calculated by the l1_smmoth function, represents the coordinate loss obtained in this case, represents the category loss, Represents the total loss obtained in this case.
[0116] When the categories are consistent, the intersection-over-union ratio is less than the intersection-over-union ratio threshold, and the confidence is less than the confidence threshold, the learning ability of confidence and coordinates is enhanced, and the loss function is updated to the following formula:
[0117] ;
[0118] in, represents the category loss obtained in the first case above, represents the coordinate loss obtained in the second case above, Represents the total loss obtained in this case.
[0119] When the IoU is greater than the IoU threshold, the categories are inconsistent, and the learning ability of the category branch is increased. The loss function is updated as follows:
[0120] ;
[0121] in, represents the category loss obtained in the first case above, Represents the regression loss, calculated by the l1_smmoth function, Represents the total loss obtained in this case.
[0122] If the model prediction result does not match the target whose detection result is a false positive, the loss function of the next round of training is updated to the category loss function, and the loss function is updated to the following formula:
[0123] ;
[0124] Among them, p represents the category confidence of the target, Represents the category loss function in this case.
[0125] If the model prediction result is to match the target whose detection result is missed, the weights of the category loss function and the coordinate loss function of the loss function for the next round of training are updated according to the missed detection type; if the model prediction result is to match the target whose detection result is false detection, the loss function for the next round of training is updated to the category loss function, and the loss function is updated according to the causes of missed detection and false detection, so as to strengthen the learning of the category and coordinates of the matched missed or falsely detected targets.
[0126] S370, repeating the aforementioned training process according to the updated sample data and loss function until the detection results of all sample images are consistent.
[0127] The technical solution of this embodiment is to obtain updated sample data for the next round of training by deleting sample images with consistent detection results and performing target enhancement on sample images with missed detection and false detection results; using the prediction results of this round to mine effective information, enhance the learning ability of the contraband detection model for effective data and shorten the training time; and update the loss function for the next round of training based on the model prediction results and the corresponding detection results, enhance the learning ability of the contraband detection model for falsely detected and missed targets.
[0128] Embodiment 4
[0129] Figure 4 A flowchart of a method for detecting prohibited items is provided for Embodiment 4 of the present application. This embodiment is applicable to the case of detecting prohibited items. The method can be executed by a contraband detection device, which can be implemented using software and / or hardware and specifically configured in a contraband detection device, for example, a detection device at a courier station.
[0130] See also Figure 4 The contraband detection method shown in the figure specifically comprises the following steps:
[0131] S410: Acquire an image to be detected.
[0132] The image to be detected may be an image that needs to be detected for contraband. Exemplarily, the image to be detected may be an image obtained by x-ray scanning. For example, it may be an image obtained by x-ray scanning of an item by a security inspection machine in places where security inspection of items is required, such as express logistics stations, train stations, airports, and hospitals.
[0133] S420. Input the image to be detected into a contraband detection model to obtain a contraband detection result of the image to be detected; wherein the contraband detection model is trained based on the contraband detection model training method provided in any embodiment of the present application.
[0134] In this embodiment, the contraband detection model is trained based on the training method of the contraband detection model provided in any embodiment of the present application. The image to be detected is input into the contraband detection model to obtain the contraband detection result of the image to be detected. The contraband detection result may include the location of the contraband and the category of the contraband. The location of the contraband can be marked by a box to facilitate further verification and inspection by relevant staff.
[0135] With the development of the express delivery industry, the number of express parcels is increasing. Manual detection of whether a parcel contains contraband not only consumes a lot of manpower, but also results in a large number of misjudgments. Therefore, a trained contraband detection model can be applied to parcel detection to achieve automatic detection of contraband and improve the efficiency and accuracy of detection. In addition, for places such as high-speed railway stations, subway stations, airports, and hospitals that require security checks on items, the contraband detection method in the embodiment can be used to detect contraband on the image to be detected.
[0136] The technical solution of this embodiment is to obtain an image to be detected; input the image to be detected into a contraband detection model to obtain a contraband detection result of the image to be detected; wherein the contraband detection model is trained based on the training method of the contraband detection model provided in any embodiment of the present application, which can improve the efficiency and accuracy of contraband detection and reduce the occurrence of missed detection and false detection.
[0137] Embodiment 5
[0138] Figure 5 The figure shows a structural diagram of a contraband detection model training device provided in the fifth embodiment of the present application. This embodiment is applicable to the case of training a contraband detection model. The specific structure of the contraband detection model training device is as follows:
[0139] A backbone network feature acquisition module 510 is used to acquire backbone network features of a sample image; the sample image includes a contraband label;
[0140] The sample image stitching module 520 is used to stitch the sample images according to the backbone network characteristics to obtain a stitched image;
[0141] A model prediction result determination module 530 is used to perform a first round of training on the contraband detection model based on the spliced image to obtain a model prediction result of the sample image;
[0142] The detection result determination module 540 is used to determine the detection result of the sample image according to the contraband label of the sample image and the model prediction result; the detection result includes consistency, missed detection and false detection;
[0143] The model adjustment training module 550 is used to update the sample data and loss function for the next round of training according to the detection results of each sample image if the detection results of the sample images are inconsistent, and repeat the above training process according to the updated sample data and loss function until the detection results of all sample images are consistent.
[0144] The technical solution of this embodiment is as follows: the backbone network features of the sample image are obtained; the sample image includes a contraband label; the features of the sample image are extracted; the sample images are spliced according to the backbone network features to obtain a spliced image; the number of iterations of the training process is reduced to improve the training efficiency; the contraband detection model is trained for the first round based on the spliced image to obtain the model prediction result of the sample image; the detection result of the sample image is determined according to the contraband label of the sample image and the model prediction result; the detection result includes consistency, missed detection and false detection; if the detection results of the sample images are inconsistent, the sample data and loss function of the next round of training are updated according to the detection results of each sample image, and the above training process is repeated according to the updated sample data and loss function until the detection results of all sample images are consistent, the sample data of the next round of training is updated according to the detection results of each sample image, the pertinence of the samples of the next round of training is ensured, invalid samples are reduced, and the training efficiency is improved, the loss function of the next round of training is updated according to the detection results of each sample image, and the learning ability of the current training is adjusted in a targeted manner to improve the pertinence and effectiveness of the training and the accuracy of the trained model. Therefore, the technical solution of the present application solves the problem that a large number of invalid samples exist in the training process of the deep learning model, which has a negative impact on the training, low training efficiency, lack of targeted training, and leads to low accuracy of the trained model. The efficiency and targetedness of the contraband detection model training and the accuracy of the trained model are improved.
[0145] Optionally, the sample image stitching module 520 includes:
[0146] A similarity determination unit, used to determine the similarity of each sample image according to the backbone network characteristics;
[0147] A similar sample group determination unit is used to cluster the sample images according to their similarity to obtain multiple similar sample groups;
[0148] The sample image stitching unit is used to stitch the sample images in each similar sample group to obtain a stitched image.
[0149] Optionally, the detection result determination module 540 includes:
[0150] The model prediction result matching unit is used to match the contraband label for each model prediction result. If the contraband label is matched, it is determined whether the contraband label of the sample image is consistent with the model prediction result;
[0151] An unmatched result determination unit, used to determine the label remaining amount and the predicted remaining amount according to the contraband label and the model prediction result;
[0152] The detection result determination unit is used to determine the detection result of the sample image according to the label remaining amount and the predicted remaining amount.
[0153] Optionally, a model prediction result matching unit includes:
[0154] An intersection-and-union ratio determination subunit is used to determine the intersection-and-union ratio of the contraband label and the model prediction result;
[0155] The I / O comparison subunit is used to determine whether the I / O ratio is greater than a preset I / O ratio threshold;
[0156] The detection result determination subunit is used to determine that the detection result is consistent if yes, and otherwise, determine that the detection result is missed.
[0157] Optionally, the model adjustment training module 550 includes:
[0158] A sample data updating unit is used to delete sample images with consistent detection results, and to perform target enhancement on sample images with missed detection and false detection results, so as to obtain updated sample data for the next round of training;
[0159] The loss function updating unit is used to update the loss function of the next round of training according to the model prediction results and the corresponding detection results.
[0160] Optional, loss function update unit, including:
[0161] The unmatched loss function updating subunit is used to update the weights of the category loss function and the coordinate loss function of the loss function of the next round of training if the model prediction result is that the target whose detection result is missed or misdetected is not matched.
[0162] Optional, loss function update unit, including:
[0163] The missed detection loss function updating subunit is used to update the weights of the category loss function and the coordinate loss function of the next round of training according to the missed detection type if the model prediction result matches the target whose detection result is missed detection;
[0164] The false positive loss function updating subunit is used to update the loss function of the next round of training to the category loss function if the model prediction result matches the target whose detection result is a false positive.
[0165] The contraband detection model training device provided in the embodiments of the present application can execute the contraband detection model training method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects for executing the contraband detection model training method.
[0166] Embodiment 6
[0167] Figure 6 1 is a schematic diagram of the structure of a contraband detection device provided according to Embodiment 6 of the present application. This embodiment is applicable to the case of detecting contraband. The device can be implemented in software and / or hardware, and can be integrated into an electronic device that carries the contraband detection function, such as a security inspection platform. Figure 6 As shown, the contraband detection of this embodiment may include:
[0168] The image acquisition module 610 is used to acquire the image to be detected;
[0169] The contraband detection module 620 is used to input the image to be detected into the contraband detection model to obtain the contraband detection result of the image to be detected; wherein the contraband detection model is trained based on the training method of the contraband detection model provided in any embodiment of the present application.
[0170] The technical solution of the embodiment of the present application is to obtain an image to be detected; input the image to be detected into a contraband detection model to obtain a contraband detection result of the image to be detected; wherein the contraband detection model is trained based on the training method of the contraband detection model provided in any embodiment of the present application, which can improve the efficiency and accuracy of contraband detection and reduce the occurrence of missed detection and false detection.
[0171] Embodiment 7
[0172] Figure 7 A schematic diagram of the structure of an electronic device provided in Embodiment 7 of the present application is shown in FIG. Figure 7 As shown, the electronic device includes a processor 710, a memory 720, an input device 730, and an output device 740; the number of the processor 710 in the electronic device can be one or more. Figure 7 A processor 710 is taken as an example; the processor 710, the memory 720, the input device 730 and the output device 740 in the electronic device can be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.
[0173] The memory 720, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the contraband detection model training method in the embodiment of the present application (for example, the backbone network feature acquisition module 510, the sample image splicing module 520, the model prediction result determination module 530, the detection result determination module 540 and the model adjustment training module 550). The processor 710 executes various functional applications and data processing of the electronic device by running the software programs, instructions and modules stored in the memory 720, that is, realizing the above-mentioned contraband detection model training method.
[0174] The memory 720 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 720 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 720 may further include a memory remotely arranged relative to the processor 710, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0175] The input device 730 may be used to receive input character information and generate key signal input related to user settings and function control of the electronic device. The output device 740 may include a display device such as a display screen.
[0176] According to an embodiment of the present invention, the present invention also provides an electronic device, a readable storage medium and a computer program product.
[0177] Embodiment 8
[0178] Embodiment 8 of the present application also provides a storage medium containing computer executable instructions, which are used to execute a contraband detection model training method when executed by a computer processor, the method comprising: obtaining a backbone network feature of a sample image; the sample image comprises a contraband label; according to the backbone network feature, the sample image is spliced to obtain a spliced image; based on the spliced image, a first round of training is performed on the contraband detection model to obtain a model prediction result of the sample image; according to the contraband label of the sample image and the model prediction result, the detection result of the sample image is determined; the detection result comprises consistency, missed detection and false detection; if there are sample images whose detection results are inconsistent, then according to the detection results of each sample image, the sample data and loss function of the next round of training are updated, and the aforementioned training process is repeated according to the updated sample data and loss function until the detection results of all sample images are consistent.
[0179] Embodiment eight of the present application also provides a storage medium containing computer executable instructions, which, when executed by a computer processor, is used to execute a contraband detection model training method, the method comprising: obtaining an image to be detected; inputting the image to be detected into a contraband detection model to obtain a contraband detection result of the image to be detected; wherein the contraband detection model is trained based on the contraband detection model training method provided in any embodiment of the present application.
[0180] Of course, the storage medium containing computer executable instructions provided in an embodiment of the present application, whose computer executable instructions are not limited to the method operations described above, can also execute related operations in the contraband detection model training method provided in any embodiment of the present application.
[0181] Through the above description of the implementation method, the technicians in the relevant field can clearly understand that the present application can be implemented with the help of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for an electronic device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0182] It is worth noting that in the embodiment of the above-mentioned contraband detection model training device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.
[0183] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.
Claims
1. A contraband detection model training method, It is characterized in that include: Obtaining backbone network features of a sample image; the sample image includes a contraband label; According to the backbone network characteristics, the sample images are spliced to obtain a spliced image; Performing a first round of training on the contraband detection model based on the spliced image to obtain a model prediction result of the sample image; Determine the detection result of the sample image according to the contraband label of the sample image and the model prediction result; the detection result includes consistency, missed detection and false detection; If the detection results of some sample images are inconsistent, the sample data and loss function for the next round of training are updated according to the detection results of each sample image, and the above training process is repeated according to the updated sample data and loss function until the detection results of all sample images are consistent.
2. The method according to claim 1, It is characterized in that The step of splicing the sample images according to the backbone network characteristics to obtain a spliced image includes: Determining the similarity of each of the sample images according to the backbone network characteristics; Clustering is performed according to the similarity of each of the sample images to obtain multiple groups of similar sample groups; The sample images in each of the similar sample groups are spliced together to obtain the spliced image.
3. The method according to claim 1, It is characterized in that Determining the detection result of the sample image according to the contraband label of the sample image and the model prediction result includes: Matching a contraband label for each of the model prediction results, and if a contraband label is matched, determining whether the contraband label of the sample image is consistent with the model prediction result; Otherwise, determining the remaining quantity of the label and the predicted remaining quantity according to the contraband label and the prediction result of the model; A detection result of the sample image is determined according to the label remaining amount and the predicted remaining amount.
4. The method according to claim 3, It is characterized in that The determining whether the contraband label of the sample image is consistent with the model prediction result includes: Determining an intersection-over-union ratio of the contraband label and the model prediction result; Determine whether the intersection-over-union ratio is greater than a preset intersection-over-union ratio threshold; If so, the detection result is determined to be consistent, otherwise, the detection result is determined to be missed.
5. The method according to claim 1, It is characterized in that If the detection results of the sample images are inconsistent, the sample data and loss function for the next round of training are updated according to the detection results of the sample images, including: Delete the sample images with consistent detection results, and perform target enhancement on the sample images with missed detection and false detection results to obtain updated sample data for the next round of training; Update the loss function for the next round of training based on the model prediction results and the corresponding test results.
6. The method according to claim 5, It is characterized in that The updating of the loss function for the next round of training according to the model prediction results and the corresponding detection results includes: If the model prediction result is that the target is not matched and the detection result is missed or misdetected, the weights of the category loss function and the coordinate loss function of the loss function for the next round of training are updated.
7. The method according to claim 5, It is characterized in that The updating of the loss function for the next round of training according to the model prediction results and the corresponding detection results includes: If the model predicts that the target is matched but missed in the detection result, the weights of the category loss function and the coordinate loss function of the loss function for the next round of training are updated according to the missed detection type; If the model prediction result matches the target whose detection result is a false positive, the loss function of the next round of training is updated to the category loss function.
8. A method for detecting contraband, include: Acquire the image to be detected; The image to be detected is input into a contraband detection model to obtain a contraband detection result of the image to be detected; wherein the contraband detection model is trained based on the contraband detection model training method described in any one of claims 1 to 7.
9. A contraband detection model training device, It is characterized in that include: A backbone network feature acquisition module, used to acquire backbone network features of a sample image; the sample image includes a contraband label; A sample image stitching module, used for stitching the sample images according to the backbone network characteristics to obtain a stitched image; A model prediction result determination module, used to perform a first round of training on the contraband detection model based on the spliced image to obtain a model prediction result of the sample image; A detection result determination module, used to determine the detection result of the sample image according to the contraband label of the sample image and the model prediction result; the detection result includes consistency, missed detection and false detection; The model adjustment training module is used to update the sample data and loss function for the next round of training according to the detection results of each sample image if the detection results of the sample images are inconsistent, and repeat the above training process according to the updated sample data and loss function until the detection results of all sample images are consistent.
10. A contraband detection device, It is characterized in that include: An image acquisition module to be detected is used to acquire the image to be detected; A contraband detection module is used to input the image to be detected into a contraband detection model to obtain a contraband detection result of the image to be detected; wherein the contraband detection model is trained based on the contraband detection model training method according to any one of claims 1 to 7.
11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the contraband detection model training method according to any one of claims 1 to 7, or the contraband detection method according to claim 8 is implemented.
12. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the contraband detection model training method as described in any one of claims 1 to 7, or the contraband detection method as described in claim 8 is implemented.
13. A computer program product, It is characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the contraband detection model training method according to any one of claims 1 to 7, or the contraband detection method according to claim 8.
Citation Information
Patent Citations
Product detection method, device and equipment and storage medium
CN115659183A
Semi-supervised signal point detection model training method, signal point detection method and device
CN115690100A
Multi-target detection method and device, electronic equipment and storage medium
CN116385850A
Deep learning model training method, target object detection method and device
CN116663650A
Parking space detection method and device, model training method and device and electronic equipment
CN117037110A
Cited By
A method for detecting prohibited article risk based on association relationship
CN120299012B
