Object detection model training method and apparatus, image detection method and apparatus

Abstract

This application discloses a method and apparatus for training an object detection model, and an image detection method and apparatus. The model training method includes: acquiring a first training sample set, where each first training sample is a first image stitched together from at least one foreground image and at least one background image; training a first object detection model using the first training sample set, and adjusting the model parameters of the first object detection model according to a first loss function to obtain a second object detection model; acquiring a second training sample set, where each second training sample includes: a foreground image, a background image, and a second image composed of a foreground image and a background image; performing distillation learning on the second object detection model using the second training sample set, and adjusting the model parameters of the second object detection model according to a fourth loss function to obtain a third object detection model. This application solves the technical problem of high false detection rates in object detection models in related technologies.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and more specifically, to a method and apparatus for training an object detection model and an image detection method and apparatus. Background Technology

[0002] With the widespread application of deep learning object detection methods in various industrial fields, object detection networks have developed relatively rapidly, and the accuracy of object detection models has become increasingly higher. However, in X-ray image detection scenarios, the following problems still exist: Currently, X-ray image detection models typically employ one-stage object detection models. However, because one-stage object detection models do not filter candidate boxes but instead sample them using positive and negative samples, they are particularly sensitive to target features and neglect the learning of background features. This results in a high detection rate but also a high false detection rate. While two-stage object detection models can reduce the false detection rate to some extent, they are time-consuming and have high hardware requirements, making them unsuitable for large-scale industrial applications.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a target detection model training method and apparatus, and an image detection method and apparatus, to at least solve the technical problem of high false detection rate in target detection models in related technologies.

[0005] According to one aspect of the embodiments of this application, a method for training an object detection model is provided, comprising: acquiring a first training sample set, wherein each first training sample is a first image stitched together from at least one foreground image and at least one background image, wherein the foreground image contains a detection target and the background image does not contain a detection target; training a first object detection model using the first training sample set, and adjusting the model parameters of the first object detection model according to a first loss function to obtain a second object detection model, wherein the first loss function includes at least: a second loss function reflecting the difference between the predicted result of the foreground image and the actual result of the foreground image and a third loss function reflecting the difference between the predicted result of the background image and the actual result of the background image; acquiring a second training sample set, wherein each second training sample includes: a foreground image, a background image, and a second image composed of a mixture of the foreground image and the background image; performing distillation learning on the second object detection model using the second training sample set, and adjusting the model parameters of the second object detection model according to a fourth loss function to obtain a third object detection model, wherein the fourth loss function includes at least: a fifth loss function reflecting the difference between the predicted result of the foreground image and the predicted result of the second image.

[0006] Optionally, obtaining the first training sample set includes: obtaining an image set including multiple foreground images and multiple background images; selecting the same number of foreground images and background images from the image set and stitching them together to obtain a first image, and using the first image as a first training sample; and sequentially determining multiple first training samples to obtain the first training sample set.

[0007] Optionally, the process of determining the first loss function includes: for each first training sample, inputting the first training sample into the first object detection model to obtain the first prediction result output by the first object detection model, wherein the first prediction result includes at least: foreground predicted object box, foreground predicted object category, background predicted object box, and background predicted object category; constructing a sixth loss function based on the difference between the foreground predicted object box and the foreground labeled object box, and constructing a seventh loss function based on the difference between the foreground predicted object category and the foreground labeled object category, and using the sum of the sixth loss function and the seventh loss function as the second loss function; constructing a third loss function based on the difference between the background predicted object category and the background labeled object category; determining the first weight and the second weight corresponding to the second loss function and the third loss function respectively, and determining the first loss function based on the first weight, the second loss function, the second weight, and the third loss function.

[0008] Optionally, the first weight and second weight corresponding to the second loss function and the third loss function are determined respectively, including: determining the first sigmoid value of the smoothed average absolute error loss function of the foreground predicted target box and the foreground labeled target box, and determining the first perfect intersection-union loss function value of the foreground predicted target box and the foreground labeled target box; determining the third weight based on the first sigmoid value and the first perfect intersection-union loss function value; determining the optimal predicted target box in the first image using a preset target detection method, determining the second perfect intersection-union loss function value of the foreground predicted target box and the optimal predicted target box, and determining the second sigmoid value of the second perfect intersection-union loss function value and the foreground predicted target category as the fourth weight; determining the product of the third weight and the fourth weight as the first weight; and determining the second weight based on the average value of the loss function values ​​of each background predicted target box and the loss function values ​​of each background predicted target box.

[0009] Optionally, obtaining a second training sample set includes: obtaining a foreground image set and a background image set; selecting a foreground image from the foreground image set and a background image from the background image set; mixing the foreground image and the background image to obtain a second image; using the foreground image, the background image, and the second image together as a second training sample; and sequentially determining multiple second training samples to obtain a second training sample set.

[0010] Optionally, the process of determining the fourth loss function includes: for each second training sample, inputting the images in the second training sample into the second object detection model in sequence to obtain the second prediction result output by the second object detection model, wherein the second prediction result includes at least: foreground image prediction result, background image prediction result, and second image prediction result; constructing a fifth loss function based on the difference between the foreground image prediction result and the second image prediction result; constructing an eighth loss function based on the difference between the foreground image prediction result and the true result of the foreground image; constructing a ninth loss function based on the difference between the background image prediction result and the true result of the background image; and determining the fourth loss function based on the fifth loss function, the eighth loss function, and the ninth loss function.

[0011] Optionally, the images in the second training sample are sequentially input into the second target detection model, including: performing weak data processing on the foreground image and the second image, wherein the weak data processing includes at least one of the following: rotation, brightness adjustment; and sequentially inputting the weakly processed foreground image, the second image, and the background image into the second target detection model.

[0012] Optionally, the bottleneck layer of the second object detection model employs a global attention mechanism, constructing a fifth loss function based on the difference between the prediction results of the foreground image and the prediction results of the second image. This includes: obtaining the first contextual relationship of each feature of the foreground image in the bottleneck layer and the second contextual relationship of each feature of the second image in the bottleneck layer; constructing a tenth loss function based on the difference between the first and second contextual relationships; constructing an eleventh loss function based on the difference between the predicted target category of the foreground image and the predicted target category of the second image; constructing a complete intersection-union loss function based on the difference between the predicted target box of the foreground image and the predicted target box of the second image, and using the product of the prediction result score of the foreground image and the complete intersection-union loss function as the twelfth loss function; and determining the fifth loss function based on the tenth, eleventh, and twelfth loss functions.

[0013] According to one aspect of the embodiments of this application, an image detection method is provided, comprising: acquiring an image to be detected; inputting the image to be detected into a target detection model to obtain a detection result output by the target detection model, wherein the target detection model is trained by the target detection model training method described above; and issuing an alarm message when the detection result indicates that a target object exists in the image to be detected.

[0014] According to another aspect of the embodiments of this application, a target detection model training apparatus is also provided, comprising: a first acquisition module, configured to acquire a first training sample set, wherein each first training sample is a first image stitched together from at least one foreground image and at least one background image, wherein the foreground image contains a detection target and the background image does not contain a detection target; a first training module, configured to train a first target detection model using the first training sample set and adjust the model parameters of the first target detection model according to a first loss function to obtain a second target detection model, wherein the first loss function includes at least: a second loss function reflecting the difference between the predicted result of the foreground image and the actual result of the foreground image and a third loss function reflecting the difference between the predicted result of the background image and the actual result of the background image; a second acquisition module, configured to acquire a second training sample set, wherein each second training sample includes: a foreground image, a background image, and a second image composed of a mixture of the foreground image and the background image; a second training module, configured to perform distillation learning on the second target detection model using the second training sample set and adjust the model parameters of the second target detection model according to a fourth loss function to obtain a third target detection model, wherein the fourth loss function includes at least: a fifth loss function reflecting the difference between the predicted result of the foreground image and the predicted result of the second image.

[0015] According to another aspect of the embodiments of this application, an image detection apparatus is also provided, comprising: a third acquisition module for acquiring an image to be detected; a detection module for inputting the image to be detected into a target detection model to obtain a detection result output by the target detection model, wherein the target detection model is trained by the target detection model training method described above; and an alarm module for issuing an alarm message when the detection result indicates that a target object exists in the image to be detected.

[0016] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-described target detection model training method or image detection method by running the computer program.

[0017] According to another aspect of the embodiments of this application, an electronic device is also provided, the electronic device including: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described target detection model training method or image detection method through the computer program.

[0018] The target detection model trained in this application is a one-stage target detection model, but the model training process is divided into two stages: In the first stage, the foreground and background images are stitched together. By modifying the loss function, the loss weight for the background part of the feature map is increased, improving the model's focus on background features. In the second stage, the foreground image, background image, and the image formed by the foreground and background are simultaneously input into the model for training. Distillation learning is performed using the foreground image and the mixed image to extract common features from the target features of both, reducing invalid information in the target information and improving the model's ability to extract effective features, thereby improving the model's detection accuracy and reducing the false detection rate. This application's solution effectively solves the technical problem of high false detection rates in target detection models in related technologies. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 This is a schematic diagram of the structure of an optional computer terminal according to an embodiment of this application;

[0021] Figure 2 This is a flowchart illustrating an optional target detection model training method according to an embodiment of this application;

[0022] Figure 3 This is a schematic flowchart of an optional image detection method according to an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of an optional target detection model training device according to an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of an optional image detection device according to an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] To better understand the embodiments of this application, the following is a translation and explanation of some nouns or terms that appear in the description of the embodiments of this application:

[0028] One-stage object detection models: These models directly detect and locate objects on the input image. They work by sliding a fixed-size window across the image and using a classifier or regressor to determine if the window contains an object and estimate its location. One-stage detection models typically use a single network and can perform object detection in real-time, but their accuracy is relatively low.

[0029] Two-stage object detection models divide the object detection task into two stages. The first stage is candidate region extraction, which generates candidate boxes that may contain the target using region generation networks or other methods. The second stage classifies and regresses these candidate boxes to determine the target's category and precise location. Two-stage detection models typically use two networks or modules, offering higher accuracy but slower speed.

[0030] Mosaic data augmentation method: Mosaic is a new method proposed in YOLOv4, suitable for object detection. The main idea is to stitch four images together into one image as training samples, and the coordinates of the bounding boxes are also changed accordingly during stitching. Its main operations are as follows: 1) Randomly crop each image to obtain A; 2) Resize A to the output image size to obtain B; 3) Randomly crop a region C of a specified size from B; 4) Paste C into the corresponding position in the output image.

[0031] Mixup data augmentation method: Mixup is to merge samples and labels in the same way to obtain a new training sample.

[0032] The Hungarian Algorithm is a combinatorial optimization algorithm that solves the task assignment problem in polynomial time. It is widely used in operations research to solve the data association problem in multi-target tracking.

[0033] Example 1

[0034] According to the embodiments of this application, a method for training an object detection model is first provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0035] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a method for training an object detection model is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0036] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0037] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the target detection model training method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the vulnerability detection method of the application described above. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0038] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0039] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0040] Under the above operating environment, embodiments of this application provide a method for training an object detection model, such as... Figure 2 As shown, the method includes at least steps S202-S208, wherein:

[0041] Step S202: Obtain a first training sample set, wherein each first training sample is a first image stitched together from at least one foreground image and at least one background image, wherein the foreground image contains the detection target and the background image does not contain the detection target.

[0042] As an optional implementation, when obtaining the first training sample set, an image set including multiple foreground images and multiple background images can be obtained first; then, the same number of foreground images and background images are selected from the image set and stitched together to obtain a first image, which is used as a first training sample; multiple first training samples are determined in sequence to obtain the first training sample set.

[0043] Taking the stitching of four images as the first training sample as an example, firstly, select one image from the image set and determine the type of the image; if the image is a foreground image, continue to obtain one foreground image and two background images from the image set; if the image is a background image, continue to obtain one background image and two foreground images from the image set; use the mosaic data augmentation method to stitch the four obtained images to obtain a first image as the first training sample.

[0044] To accelerate image processing efficiency and reduce memory usage and training time, the orientation of the images can be adaptively adjusted before image stitching. For example, if the width of the first image is greater than its height, the subsequent three images should maintain the same orientation, with width greater than height; otherwise, the images should be rotated 90°.

[0045] Step S204: Train the first target detection model using the first training sample set, and adjust the model parameters of the first target detection model according to the first loss function to obtain the second target detection model. The first loss function includes at least: a second loss function reflecting the difference between the predicted result of the foreground image and the actual result of the foreground image, and a third loss function reflecting the difference between the predicted result of the background image and the actual result of the background image.

[0046] As an optional implementation method, the first loss function can be determined in the following way:

[0047] For each first training sample, the first training sample is input into the first object detection model to obtain the first prediction result output by the first object detection model. The first prediction result includes at least: foreground predicted object box, foreground predicted object category, background predicted object box, and background predicted object category.

[0048] A sixth loss function, Loss, is constructed based on the difference between the predicted foreground bounding box and the labeled foreground bounding box. fg_bbox A seventh loss function, Loss, is constructed based on the difference between the predicted target category and the labeled target category in the foreground. fg_cls The sum of the sixth and seventh loss functions is used as the second loss function, Loss. fg ;

[0049] A third loss function, Loss, is constructed based on the difference between the predicted target category and the labeled target category in the background. bg_cls ;

[0050] Determine the second loss function Loss respectively fg and the third loss function Loss bg_cls The corresponding first weight fg Second weight bg And based on the first weightfg Second loss function Loss fg Second weight bg and the third loss function Loss bg_cls The first loss function is determined as follows:

[0051] Loss one =weight fg ×(Loss fg_cls +Loss fg_bbox )+weight bg ×Loss bg_cls

[0052] Optionally, in determining the first weight fg To maintain a linear relationship between the target box score and the category score, the target box weights can be adjusted. bbox and category weight cls The specific calculation method for combining them is as follows:

[0053] Since the target bounding box is composed of the target area and the target distance, the smooth L1 loss function lacks constraints on the area and is easily affected by the target size. The Complete Intersection over Union (CIOU) loss function, on the other hand, does not constrain the target coordinates, leading to the CIOU value being the same even when the target is in different positions. Therefore, we can determine the first sigmoid value (sigmoid(smooth L1 Loss)) of the smooth L1 Loss function for both the predicted and labeled foreground bounding boxes, and determine the first CIOU value. Based on the first sigmoid value (sigmoid(smooth L1 Loss)) and the first CIOU value, we determine the third weight, i.e., the target box weight.

[0054] weight bbox =sigmoid(smoothL1 Loss)+ciou

[0055] Calculate category weights cls To prevent the negative effects of artificial targets, this embodiment first uses a preset target detection method to determine the optimal predicted target box in the first image. This target detection method can be the Hungarian method. Then, the second perfect intersection-union loss function value (ciou) between the foreground predicted target box and the optimal predicted target box is determined, denoted as GT. soft Determine the second complete intersection-union loss function value GT softThe second sigmoid value of the target class P in the foreground prediction is the fourth weight, i.e., the class weight is...

[0056] weight cls =sigmoid(P,GT) soft )

[0057] Determine the third weight bbox With the fourth weight cls The product is the first weight.

[0058] weight fg =weight cls ×weight bbox

[0059] Optionally, in determining the second weight bg At that time, the loss function value of the target box can be predicted based on each background. bg The average value avg(Loss) sum_bg and the loss function value of each background predicted target box. bg Determine the second weight as

[0060]

[0061] In the first stage of training, an image stitching method was used to assign different weights to the foreground and background target boxes. This enhanced the model's learning of difficult targets on the stitched foreground image and also enhanced the model's learning of easily misdetected targets on the background image.

[0062] Step S206: Obtain a second training sample set, wherein each second training sample includes: a foreground image, a background image, and a second image composed of a foreground image and a background image.

[0063] As an optional implementation, when obtaining the second training sample set, a foreground image set and a background image set can be obtained first; then, a foreground image is selected from the foreground image set, and a background image is selected from the background image set. Data augmentation methods such as mixup are used to mix the foreground image and the background image to obtain a second image. The foreground image, the background image, and the second image are used together as a second training sample. Multiple second training samples are determined in sequence to obtain the second training sample set.

[0064] Step S208: Distillation learning is performed on the second target detection model using the second training sample set, and the model parameters of the second target detection model are adjusted according to the fourth loss function to obtain the third target detection model. The fourth loss function includes at least a fifth loss function that reflects the difference between the prediction result of the foreground image and the prediction result of the second image.

[0065] As an optional implementation, the fourth loss function can be determined in the following way:

[0066] For each second training sample, the images in the second training sample are sequentially input into the second object detection model to obtain the second prediction result output by the second object detection model. The second prediction result includes at least: the foreground image prediction result, the background image prediction result, and the second image prediction result.

[0067] A fifth loss function, Loss, is constructed based on the difference between the foreground image prediction result and the second image prediction result. two_dil ;

[0068] An eighth loss function, Loss, is constructed based on the difference between the predicted foreground image and the actual foreground image. two_fg ;

[0069] A ninth loss function, Loss, is constructed based on the difference between the predicted background image and the actual background image. two_bg ;

[0070] Based on the fifth loss function Loss two_dil The eighth loss function, Loss two_fg and the ninth loss function Loss two_bg The fourth loss function is determined as follows:

[0071] Loss two =Loss two_dil +Loss two_fg +Loss two_bg

[0072] Optionally, when inputting the images from the second training samples into the second target detection model in sequence, the foreground image and the second image can be subjected to weak data processing, such as rotation and brightness adjustment; the weakly processed foreground image, the second image, and the background image are then input into the second target detection model in sequence.

[0073] The bottleneck layer of the second object detection model employs a global attention mechanism. When constructing the fifth loss function based on the difference between the foreground image prediction result and the second image prediction result, it can be done in the following way:

[0074] Obtain the first contextual relationship of each feature of the foreground image in the bottleneck layer. The second contextual relationship (GC) of each feature in the bottleneck layer of the second image. mixup According to the first context relationship Second context relationship GC (feature_map) mixupThe difference between the two loss functions constructs a tenth loss function, which can be the mean absolute error loss function, specifically...

[0075]

[0076] Then, based on the predicted target category P of the foreground image, fg The predicted target category P of the second image mixup The difference is used to construct an eleventh loss function, which can be the cross-entropy loss function, specifically...

[0077] Loss cls_dil =CE(P mixup ,P fg )

[0078] Predicted bounding boxes (bboxes) based on the foreground image fg Predicted target bounding boxes (bboxes) of the second image mixup The difference constructs a complete intersection-union loss function ciou(bboxes) mixup bboxs fg To reduce the impact of false positives, the prediction score P of the foreground image can be calculated. fg As its weight, the prediction score P of the foreground image is... fg The loss function ciou(bboxes) with the perfect intersection-union ratio mixup bboxs fg The product of these two functions is used as the twelfth loss function, specifically:

[0079] Loss bbox_dil =ciou(bboxs) mixup bboxs fg )×P fg

[0080] Finally, based on the tenth loss function Loss fpm_distil Eleventh Loss Function cls_dil and the twelfth loss function Loss bbox_dil The fifth loss function is determined as follows:

[0081] Loss two_dil =Loss fpm_distil +Loss cls_dil +Loss bbox_dil

[0082] Optionally, to further improve the model's detection accuracy, corresponding weights can be added to the eighth and ninth loss functions. The process of determining the weight values ​​can refer to the first weight in the first training phase. fg Second weight bgThe process of determining [the value] will not be elaborated upon here.

[0083] In the second stage of training, image mixing and distillation methods are used to enable the model to effectively extract common features of the target, which can improve the robustness and accuracy of the model.

[0084] The target detection model trained in this application is a one-stage target detection model, but the model training process is divided into two stages: In the first stage, the foreground and background images are stitched together. By modifying the loss function, the loss weight for the background part of the feature map is increased, improving the model's focus on background features. In the second stage, the foreground image, background image, and a mixed foreground and background image are simultaneously input into the model for training. Distillation learning is performed using the foreground image and the mixed image to extract common features from the target features of both, reducing invalid information in the target information and improving the model's ability to extract effective features, thereby improving the model's detection accuracy and reducing the false detection rate. This application's solution effectively solves the technical problem of high false detection rates in target detection models in related technologies.

[0085] Example 2

[0086] Based on the target detection model training method provided in Example 1, this application also provides an image detection method, which can also be implemented in... Figure 1 The steps are executed in the computer terminal 10 (or similar computing device such as a mobile device) shown. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0087] Figure 3 This is a flowchart illustrating an optional image detection method according to an embodiment of this application, as shown below. Figure 3 As shown, the method includes at least steps S302-S306, wherein:

[0088] Step S302: Obtain the image to be detected.

[0089] The proposed solution is mainly used for detecting hazardous materials using X-ray images; therefore, the image to be detected can be a real-time acquired X-ray image.

[0090] Step S304: Input the image to be detected into the target detection model to obtain the detection result output by the target detection model. The target detection model is trained by the target detection model training method in Example 1.

[0091] Since the object detection model training method has been described in detail in Example 1, it will not be repeated here.

[0092] Step S306: When the detection result indicates that a target object exists in the image to be detected, an alarm message is issued.

[0093] Optionally, when the detection result indicates the presence of hazardous materials in the X-ray image to be detected, an alarm message can be issued to the staff, who will then inspect the corresponding items.

[0094] In this embodiment, since the target detection model is a one-stage target detection model, the detection time is short; and the model has high detection accuracy and low false detection rate. In industrial application scenarios, it can greatly improve the target detection efficiency and has strong applicability.

[0095] Example 3

[0096] According to embodiments of this application, a target detection model training apparatus for implementing the target detection model training method in Embodiment 1 is also provided, such as... Figure 4 As shown, the target detection model training device includes at least a first acquisition module 41, a first training module 42, a second acquisition module 43, and a second training module 44, wherein:

[0097] The first acquisition module 41 is used to acquire a first training sample set, wherein each first training sample is a first image composed of at least one foreground image and at least one background image, wherein the foreground image contains a detection target and the background image does not contain a detection target.

[0098] As an optional implementation, when acquiring the first training sample set, the first acquisition module may first acquire an image set including multiple foreground images and multiple background images; then select the same number of foreground images and background images from the image set and stitch them together to obtain a first image, and use the first image as a first training sample; and so on, determine multiple first training samples to obtain the first training sample set.

[0099] The first training module 42 is used to train the first object detection model using the first training sample set and adjust the model parameters of the first object detection model according to the first loss function to obtain the second object detection model. The first loss function includes at least: a second loss function reflecting the difference between the predicted result of the foreground image and the real result of the foreground image and a third loss function reflecting the difference between the predicted result of the background image and the real result of the background image.

[0100] Optionally, the process of determining the first loss function includes: for each first training sample, inputting the first training sample into the first object detection model to obtain the first prediction result output by the first object detection model, wherein the first prediction result includes at least: foreground predicted object box, foreground predicted object category, background predicted object box, and background predicted object category; constructing a sixth loss function based on the difference between the foreground predicted object box and the foreground labeled object box, and constructing a seventh loss function based on the difference between the foreground predicted object category and the foreground labeled object category, and using the sum of the sixth loss function and the seventh loss function as the second loss function; constructing a third loss function based on the difference between the background predicted object category and the background labeled object category; determining the first weight and the second weight corresponding to the second loss function and the third loss function respectively, and determining the first loss function based on the first weight, the second loss function, the second weight, and the third loss function.

[0101] Optionally, when the first training module determines the first weight and the second weight corresponding to the second loss function and the third loss function, it determines the first sigmoid value of the smoothed average absolute error loss function between the foreground predicted target box and the foreground labeled target box, and determines the first perfect intersection-union loss function value between the foreground predicted target box and the foreground labeled target box. The third weight is determined based on the first sigmoid value and the first perfect intersection-union loss function value. The optimal predicted target box in the first image is determined using a preset target detection method. The second perfect intersection-union loss function value between the foreground predicted target box and the optimal predicted target box is determined. The second sigmoid value of the second perfect intersection-union loss function value and the second sigmoid value of the foreground predicted target category are determined as the fourth weight. The product of the third weight and the fourth weight is determined as the first weight. The second weight is determined based on the average of the loss function values ​​of each background predicted target box and the loss function values ​​of each background predicted target box.

[0102] The second acquisition module 43 is used to acquire a second training sample set, wherein each second training sample includes: a foreground image, a background image, and a second image composed of the foreground image and the background image.

[0103] As an optional implementation, when acquiring the second training sample set, the second acquisition module may first acquire a foreground image set and a background image set; select a foreground image from the foreground image set and a background image from the background image set, and stitch the foreground image and the background image together to obtain a second image, and use the foreground image, the background image and the second image together as a second training sample; and sequentially determine multiple second training samples to obtain the second training sample set.

[0104] The second training module 44 is used to perform distillation learning on the second object detection model using the second training sample set, and adjust the model parameters of the second object detection model according to the fourth loss function to obtain the third object detection model. The fourth loss function includes at least a fifth loss function that reflects the difference between the prediction result of the foreground image and the prediction result of the second image.

[0105] Optionally, the process of determining the second loss function includes: for each second training sample, inputting the images in the second training sample into the second object detection model in sequence to obtain the second prediction result output by the second object detection model, wherein the second prediction result includes at least: foreground image prediction result, background image prediction result, and second image prediction result; constructing a fifth loss function based on the difference between the foreground image prediction result and the second image prediction result; constructing an eighth loss function based on the difference between the foreground image prediction result and the true result of the foreground image; constructing a ninth loss function based on the difference between the background image prediction result and the true result of the background image; and determining a fourth loss function based on the fifth loss function, the eighth loss function, and the ninth loss function.

[0106] Optionally, when the second training module inputs the images from the second training samples into the second target detection model in sequence, it first performs weak data processing on the foreground image and the second image, wherein the weak data processing includes at least one of the following: rotation, brightness adjustment; and then inputs the weakly processed foreground image, the second image, and the background image into the second target detection model in sequence.

[0107] Optionally, the bottleneck layer of the second object detection model adopts a global attention mechanism. When the second training module constructs the fifth loss function, it obtains the first contextual relationship of each feature of the foreground image in the bottleneck layer and the second contextual relationship of each feature of the second image in the bottleneck layer. It constructs the tenth loss function based on the difference between the first and second contextual relationships. It constructs the eleventh loss function based on the difference between the predicted target category of the foreground image and the predicted target category of the second image. It constructs the full intersection-union loss function based on the difference between the predicted target box of the foreground image and the predicted target box of the second image, and uses the product of the prediction result score of the foreground image and the full intersection-union loss function as the twelfth loss function. The fifth loss function is determined based on the tenth, eleventh, and twelfth loss functions.

[0108] It should be noted that each module in the target detection model training device in this application corresponds one-to-one with each implementation step of the target detection model training method in Example 1. Since Example 1 has been described in detail, some details not shown in this embodiment can be referred to Example 1, and will not be elaborated further here.

[0109] Example 4

[0110] According to an embodiment of this application, an image detection apparatus for implementing the image detection method in Embodiment 2 is also provided, such as... Figure 5 As shown, the image detection device includes at least a third acquisition module 51, a detection module 52, and an alarm module 53, wherein:

[0111] The third acquisition module 51 is used to acquire the image to be detected.

[0112] The proposed solution is mainly used for detecting hazardous materials using X-ray images; therefore, the image to be detected can be a real-time acquired X-ray image.

[0113] The detection module 52 is used to input the image to be detected into the target detection model and obtain the detection result output by the target detection model. The target detection model is trained by the target detection model training method in Example 1.

[0114] Since the object detection model training method has been described in detail in Example 1, it will not be repeated here.

[0115] The alarm module 53 is used to issue an alarm message when the detection result indicates that a target object exists in the image to be detected.

[0116] Optionally, when the detection result indicates that there are hazardous materials in the X-ray image to be detected, the alarm module can send an alarm message to the staff, who can then inspect the corresponding items.

[0117] It should be noted that each module in the image detection device in this embodiment corresponds one-to-one with each implementation step of the image detection method in embodiment 2. Since embodiment 2 has been described in detail, some details not shown in this embodiment can be referred to embodiment 2, and will not be elaborated further here.

[0118] Example 5

[0119] According to an embodiment of this application, a non-volatile storage medium is also provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the target detection model training method in Embodiment 1 by running the computer program.

[0120] According to an embodiment of this application, a processor is also provided for running a computer program, wherein the computer program executes the target detection model training method in Embodiment 1 during runtime.

[0121] According to an embodiment of this application, an electronic device is also provided, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the target detection model training method of Embodiment 1 through the computer program.

[0122] Specifically, the computer program executes the following steps: acquiring a first training sample set, wherein each first training sample is a first image stitched together from at least one foreground image and at least one background image, the foreground image containing the target to be detected, and the background image not containing the target to be detected; training a first target detection model using the first training sample set, and adjusting the model parameters of the first target detection model according to a first loss function to obtain a second target detection model, wherein the first loss function includes at least: a second loss function reflecting the difference between the predicted result of the foreground image and the actual result of the foreground image, and a third loss function reflecting the difference between the predicted result of the background image and the actual result of the background image; acquiring a second training sample set, wherein each second training sample includes: a foreground image, a background image, and a second image composed of a mixture of the foreground image and the background image; performing distillation learning on the second target detection model using the second training sample set, and adjusting the model parameters of the second target detection model according to a fourth loss function to obtain a third target detection model, wherein the fourth loss function includes at least: a fifth loss function reflecting the difference between the predicted result of the foreground image and the predicted result of the second image.

[0123] Optionally, the computer program may also perform the following steps: acquiring an image to be detected; inputting the image to be detected into a target detection model to obtain the detection result output by the target detection model, wherein the target detection model is trained by the target detection model training method in Example 1; and issuing an alarm message when the detection result indicates that a target object exists in the image to be detected.

[0124] The sequence numbers of the above embodiments are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0125] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0126] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0127] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0128] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0129] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0130] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for training an object detection model, characterized in that, include: Obtain a first training sample set, wherein each first training sample is a first image stitched together from at least one foreground image and at least one background image, wherein the foreground image contains a detection target and the background image does not contain a detection target; The first target detection model is trained using the first training sample set, and the model parameters of the first target detection model are adjusted according to the first loss function to obtain the second target detection model. The first loss function includes at least: a second loss function that reflects the difference between the predicted result of the foreground image and the real result of the foreground image, and a third loss function that reflects the difference between the predicted result of the background image and the real result of the background image. Obtain a second training sample set, wherein each second training sample includes: a foreground image, a background image, and a second image formed by mixing the foreground image and the background image; The second object detection model is distilled and learned using the second training sample set, and the model parameters of the second object detection model are adjusted according to the fourth loss function to obtain the third object detection model. The fourth loss function includes at least a fifth loss function that reflects the difference between the prediction result of the foreground image and the prediction result of the second image.

2. The method according to claim 1, characterized in that, Obtain the first training sample set, including: Obtain an image set that includes multiple foreground images and multiple background images; The same number of foreground and background images are selected from the image set and stitched together to obtain a first image, which is then used as a first training sample. Multiple first training samples are determined sequentially to obtain the first training sample set.

3. The method according to claim 1, characterized in that, The process of determining the first loss function includes: For each of the first training samples, the first training sample is input into the first object detection model to obtain the first prediction result output by the first object detection model, wherein the first prediction result includes at least: foreground predicted object box, foreground predicted object category, background predicted object box, and background predicted object category; A sixth loss function is constructed based on the difference between the predicted foreground bounding box and the labeled foreground bounding box, and a seventh loss function is constructed based on the difference between the predicted foreground target category and the labeled foreground target category. The sum of the sixth loss function and the seventh loss function is used as the second loss function. The third loss function is constructed based on the difference between the predicted target category and the labeled target category in the background. First weights and second weights corresponding to the second loss function and the third loss function are determined respectively, and the first loss function is determined based on the first weights, the second loss function, the second weights and the third loss function.

4. The method according to claim 3, characterized in that, Determining the first weight and the second weight corresponding to the second loss function and the third loss function respectively includes: The first sigmoid value of the smoothed average absolute error loss function of the foreground predicted target box and the foreground labeled target box is determined, and the first perfect intersection-union loss function value of the foreground predicted target box and the foreground labeled target box is determined. The third weight is determined based on the first sigmoid value and the first perfect intersection-union loss function value. The optimal predicted target box in the first image is determined using a preset target detection method. The second complete intersection-union ratio (CIU) loss function value between the foreground predicted target box and the optimal predicted target box is determined. The second CIU loss function value and the second sigmoid value of the foreground predicted target category are determined as the fourth weight. The product of the third weight and the fourth weight is determined to be the first weight; The second weight is determined based on the average value of the loss function of each of the background prediction target boxes and the loss function value of each of the background prediction target boxes.

5. The method according to claim 1, characterized in that, Obtain the second training sample set, including: Obtain the foreground image set and the background image set; Select a foreground image from the foreground image set and a background image from the background image set. Mix the foreground image and the background image to obtain a second image. Use the foreground image, the background image, and the second image together as a second training sample. Multiple second training samples are determined sequentially to obtain the second training sample set.

6. The method according to claim 1, characterized in that, The process of determining the fourth loss function includes: For each of the second training samples, the images in the second training samples are sequentially input into the second object detection model to obtain the second prediction result output by the second object detection model. The second prediction result includes at least: foreground image prediction result, background image prediction result, and second image prediction result. The fifth loss function is constructed based on the difference between the foreground image prediction result and the second image prediction result; An eighth loss function is constructed based on the difference between the predicted foreground image and the actual foreground image. A ninth loss function is constructed based on the difference between the predicted background image and the actual background image. The fourth loss function is determined based on the fifth loss function, the eighth loss function, and the ninth loss function.

7. The method according to claim 6, characterized in that, The images from the second training samples are sequentially input into the second object detection model, including: The foreground image and the second image are subjected to data weakening processing, wherein the data weakening processing includes at least one of the following: rotation, brightness adjustment; The foreground image and the second image after weak data processing, along with the background image, are sequentially input into the second target detection model.

8. The method according to claim 6, characterized in that, The bottleneck layer of the second object detection model employs a global attention mechanism, constructing the fifth loss function based on the difference between the foreground image prediction result and the second image prediction result, including: Obtain the first contextual relationship of each feature of the foreground image in the bottleneck layer and the second contextual relationship of each feature of the second image in the bottleneck layer, and construct the tenth loss function based on the difference between the first contextual relationship and the second contextual relationship; An eleventh loss function is constructed based on the difference between the predicted target category of the foreground image and the predicted target category of the second image; A complete intersection-union loss function is constructed based on the difference between the predicted target box of the foreground image and the predicted target box of the second image, and the product of the prediction result score of the foreground image and the complete intersection-union loss function is used as the twelfth loss function; The fifth loss function is determined based on the tenth loss function, the eleventh loss function, and the twelfth loss function.

9. An image detection method, characterized in that, include: Acquire the image to be detected; The image to be detected is input into the target detection model to obtain the detection result output by the target detection model, wherein the target detection model is trained by the target detection model training method according to any one of claims 1 to 8; When the detection result indicates that a target object exists in the image to be detected, an alarm message is issued.

10. A target detection model training device, characterized in that, include: The first acquisition module is used to acquire a first training sample set, wherein each first training sample is a first image stitched together from at least one foreground image and at least one background image, wherein the foreground image contains a detection target and the background image does not contain a detection target; The first training module is used to train the first target detection model using the first training sample set, and adjust the model parameters of the first target detection model according to the first loss function to obtain the second target detection model. The first loss function includes at least: a second loss function reflecting the difference between the predicted result of the foreground image and the real result of the foreground image, and a third loss function reflecting the difference between the predicted result of the background image and the real result of the background image. The second acquisition module is used to acquire a second training sample set, wherein each second training sample includes: a foreground image, a background image, and a second image formed by mixing the foreground image and the background image; The second training module is used to perform distillation learning on the second object detection model using the second training sample set, and adjust the model parameters of the second object detection model according to the fourth loss function to obtain the third object detection model. The fourth loss function includes at least a fifth loss function that reflects the difference between the prediction result of the foreground image and the prediction result of the second image.

11. An image detection device, characterized in that, include: The third acquisition module is used to acquire the image to be detected; A detection module is used to input the image to be detected into a target detection model and obtain the detection result output by the target detection model, wherein the target detection model is trained by the target detection model training method according to any one of claims 1 to 8; The alarm module is used to issue an alarm message when the detection result indicates that a target object exists in the image to be detected.

12. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored computer program, wherein the device containing the non-volatile storage medium executes the target detection model training method according to any one of claims 1 to 8 or the image detection method according to claim 9 by running the computer program.

13. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute, via the computer program, the object detection model training method of any one of claims 1 to 8 or the image detection method of claim 9.

Images