Training of image detection model, image detection method and device

Abstract

Embodiments of the present application provide a training method and device of an image detection model, and an image detection method and device, the image detection model comprising a first feature extraction network, a second feature extraction network and a detection label prediction network; the first feature extraction network is a common network for n image detection tasks, the second feature extraction network comprises n feature extraction sub-networks, and the detection label prediction network comprises n label prediction sub-networks; the feature extraction sub-networks, the label prediction sub-networks and the image detection tasks correspond one by one; wherein image detail feature extraction is performed through the first feature extraction network, and the extracted image feature information is respectively input into the n feature extraction sub-networks, so as to perform image region global feature extraction and label prediction through the feature extraction sub-network and the label prediction sub-network corresponding to each image detection task, and complete the n image detection tasks, thereby improving the model training efficiency and the image detection efficiency and reducing the model maintenance cost.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a training method and apparatus for an image detection model. Background Technology

[0002] Currently, with the continuous development of image processing and artificial intelligence technologies, there are increasingly more application scenarios requiring quality inspection of target images. For example, in advertising scenarios, to ensure that the advertisements delivered to users are compliant, it is usually necessary to perform quality inspection on the advertising materials (i.e., the image frames in the advertisements to be delivered). This includes detecting whether the advertising materials contain traces of image tampering (e.g., image manipulation by Photoshop) and whether the text contained in the advertising materials conforms to preset specifications. However, for cases involving multiple image quality inspection items, it is necessary to train multiple neural network models separately, with each neural network model corresponding to one image quality inspection item. That is, each neural network model completes a different image detection task. This not only results in long model training time during the model training phase but also inevitably leads to image detection efficiency issues during the model application phase. Summary of the Invention

[0003] The purpose of this application is to provide a training method and apparatus for an image detection model, which can improve model training efficiency and image detection efficiency, and reduce model maintenance costs.

[0004] To achieve the above technical solution, the embodiments of this application are implemented as follows:

[0005] In a first aspect, embodiments of this application provide a method for training an image detection model, the method comprising:

[0006] Obtain an image sample dataset; the image sample dataset includes N target image samples;

[0007] Based on N target image samples, the image detection model to be trained is iteratively trained to obtain the trained image detection model;

[0008] The image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks. The second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, the label prediction sub-networks, and the image detection tasks correspond one-to-one. The specific implementation methods for each model training iteration are as follows:

[0009] For each target image sample: the first feature extraction network extracts features based on the target image sample to obtain first image feature information; each feature extraction sub-network extracts features based on the first image feature information to obtain second image feature information; each label prediction sub-network predicts labels based on the first image feature information and the second image feature information to obtain target predicted labels;

[0010] Based on the predicted target label and the true target label, a sub-loss set corresponding to the second feature extraction network is determined; the true target label corresponds to the image detection task corresponding to the feature extraction sub-network, and the sub-loss set includes n sub-loss values ​​corresponding to the feature extraction sub-network respectively.

[0011] The image detection model is updated with parameters based on the sub-loss values ​​corresponding to each feature extraction sub-network.

[0012] Secondly, an image detection method is provided in the embodiments of this application, the method comprising:

[0013] Acquire the image to be detected;

[0014] The image to be detected is input into the trained image detection model to perform image detection, and the image detection result of the image to be detected is obtained.

[0015] The image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks. The second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, the label prediction sub-networks, and the image detection tasks correspond one-to-one. The specific implementation methods for each image detection are as follows:

[0016] For each of the images to be detected: the first feature extraction network extracts features based on the image to be detected to obtain third image feature information; each feature extraction sub-network extracts features based on the third image feature information to obtain fourth image feature information; each label prediction sub-network predicts labels based on the third image feature information and the fourth image feature information to obtain image sub-detection results;

[0017] Based on the image sub-detection results output by each of the label prediction sub-networks, the image detection result of the image to be detected is obtained.

[0018] Thirdly, an image detection model training apparatus provided in this application embodiment includes:

[0019] An image sample acquisition module is used to acquire an image sample dataset; the image sample dataset includes N target image samples;

[0020] The detection model training module is used to iteratively train the image detection model to be trained based on N target image samples to obtain the trained image detection model.

[0021] The image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks. The second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, the label prediction sub-networks, and the image detection tasks correspond one-to-one. The specific implementation methods for each model training iteration are as follows:

[0022] For each target image sample: the first feature extraction network extracts features based on the target image sample to obtain first image feature information; each feature extraction sub-network extracts features based on the first image feature information to obtain second image feature information; each label prediction sub-network predicts labels based on the first image feature information and the second image feature information to obtain target predicted labels; based on the target predicted labels and the target true labels, a sub-loss set corresponding to the second feature extraction network is determined; the target true labels correspond to the image detection task corresponding to the feature extraction sub-network, and the sub-loss set includes sub-loss values ​​corresponding to n feature extraction sub-networks respectively; the parameters of the image detection model are updated based on the sub-loss values ​​corresponding to each feature extraction sub-network.

[0023] Fourthly, an image detection apparatus is provided in the embodiments of this application, the apparatus comprising:

[0024] The image acquisition module is used to acquire the image to be detected;

[0025] The image detection module is used to input the image to be detected into the trained image detection model to perform image detection and obtain the image detection result of the image to be detected;

[0026] The image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks. The second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, the label prediction sub-networks, and the image detection tasks correspond one-to-one. The specific implementation methods for each image detection are as follows:

[0027] For each of the images to be detected: the first feature extraction network extracts features based on the image to be detected to obtain third image feature information; each feature extraction sub-network extracts features based on the third image feature information to obtain fourth image feature information; each label prediction sub-network predicts labels based on the third image feature information and the fourth image feature information to obtain image sub-detection results; based on the image sub-detection results output by each label prediction sub-network, the image detection result of the image to be detected is obtained.

[0028] Fifthly, an embodiment of this application provides a computer device, the device comprising:

[0029] A processor; and a memory arranged to store computer-executable instructions configured to be executed by the processor, the executable instructions including steps for performing the methods described in the first or second aspect.

[0030] Sixthly, embodiments of this application provide a storage medium for storing computer-executable instructions that cause a computer to perform steps in the methods described in the first or second aspect.

[0031] As can be seen, in this embodiment, the image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks, the second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, label prediction sub-networks, and image detection tasks correspond one-to-one. Specifically, the first feature extraction network extracts image detail features, and the image feature information output by the first feature extraction network is input to each of the n feature extraction sub-networks to perform global feature extraction of the image region through the feature extraction sub-network corresponding to each image detection task. Then, the image feature information output by the feature extraction sub-networks is input to the corresponding label prediction sub-network for label prediction. In other words, the image detection model includes a shared network (i.e., the first feature extraction network) and branch networks (i.e., feature extraction sub-networks and label prediction sub-networks) corresponding to each image detection task. The shared network extracts image detail features from the image. The image detail features are used as input data for each branch network. Each branch network then learns differentiated image detection knowledge based on these image detail features and the target ground truth labels corresponding to its respective image detection task. Since the target ground truth labels are different for different image detection tasks, the parameter values ​​of each branch network after multiple rounds of parameter iteration are also determined by its respective detection task. This allows a single image detection model to first extract image detail features that can be shared across multiple detection tasks using a common network. Then, different branch networks within this image detection model complete n image detection tasks based on these image detail features. Thus, n image detection tasks can be achieved with a single image detection model. Therefore, not only is iterative training of n image detection models unnecessary during the model training phase, improving model training efficiency, but it is also unnecessary to use n image detection models to complete an image detection task during the model application phase, improving image detection efficiency. Furthermore, only one image detection model needs to be maintained subsequently, reducing model maintenance costs. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in one or more of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a schematic diagram of a first flowchart of a training method for an image detection model provided in an embodiment of this application;

[0034] Figure 2This is a schematic diagram of a second process for training an image detection model provided in an embodiment of this application;

[0035] Figure 3 A schematic diagram illustrating the first implementation principle of the training method for the image detection model provided in this application embodiment;

[0036] Figure 4 A schematic diagram illustrating the second implementation principle of the training method for the image detection model provided in the embodiments of this application;

[0037] Figure 5 A schematic diagram illustrating the third implementation principle of the training method for the image detection model provided in the embodiments of this application;

[0038] Figure 6 A schematic flowchart of the image detection method provided in the embodiments of this application;

[0039] Figure 7 This is a schematic diagram illustrating the first implementation principle of the image detection method provided in the embodiments of this application;

[0040] Figure 8 This is a schematic diagram illustrating the second implementation principle of the image detection method provided in the embodiments of this application;

[0041] Figure 9 A schematic diagram of the module composition of the training device for the image detection model provided in the embodiments of this application;

[0042] Figure 10 This is a schematic diagram of the module composition of the image detection device provided in the embodiments of this application;

[0043] Figure 11 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0044] To enable those skilled in the art to better understand the technical solutions in one or more of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of one or more of this application, and not all embodiments. Based on the embodiments of one or more of this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this application.

[0045] It should be noted that, unless otherwise specified, one or more embodiments and features described in this application can be combined with each other. The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0046] This application provides one or more embodiments of a training method and apparatus for an image detection model. Considering the situation where multiple image detection tasks are involved in the image detection process, it is necessary to train a separate image detection model for each task. For example, multiple image detection tasks may include text box detection and image tampering detection. Correspondingly, image detection model 1 and image detection model 2 need to be pre-trained. Image detection model 1 is used to delineate text boxes contained in the image to be detected, and image detection model 2 is used to identify whether there are traces of image tampering in the image to be detected. However, in this application embodiment, the receptive field of image feature extraction differs due to the different kernel sizes of the convolutional layers; that is, the receptive field of image feature extraction is related to the kernel size of the convolutional layers. Furthermore, since there are certain commonalities among multiple image detection tasks—for example, both text box detection and image tampering detection are segmentation tasks—image detail features can be shared among multiple image detection tasks. However, the boundaries of text boxes and image tampering are different. Therefore, the image... The feature extraction process is divided into two sub-processes. The first sub-process uses small regions as extraction units and a shared network to extract image detail features. These detail features can be shared across multiple image detection tasks. The second sub-process uses large regions as extraction units and different feature extraction sub-networks to extract global features of the image region. Then, the parameters of the corresponding branch networks (i.e., feature extraction sub-networks and label prediction sub-networks) are trained based on the sub-loss values ​​corresponding to each image detection task, and the parameters of the shared network (i.e., the first feature extraction network) are trained based on the total loss value. The kernel size of the shared network is smaller than that of the feature extraction sub-networks. Therefore, the receptive field of the shared network is relatively small, and the feature extraction process focuses on extracting image features from small regions (image detail features). The receptive fields of each feature extraction sub-network are relatively large, and the feature extraction process focuses on extracting image features from large regions (i.e., global features of the image region). In other words, the shared network can use convolutional layers with smaller kernel sizes, while the feature extraction sub-networks in the branch networks can use convolutional layers with larger kernel sizes.In other words, this technical solution constructs a new image detection model, which includes a common network (i.e., the first feature extraction network) and branch networks (i.e., feature extraction sub-networks and label prediction sub-networks) corresponding to each image detection task. The common network and feature extraction sub-networks use convolutional layers with different kernel sizes for image feature extraction. The kernel size of the common network is smaller than that of the feature extraction sub-networks. The common network extracts image detail features from the image, which serve as input data for each branch network. Then, each branch network learns differentiated image detection knowledge based on these image detail features and the target ground truth labels corresponding to its respective image detection task. Since the target ground truth labels are different for different image detection tasks, each branch network, after multiple rounds of parameter iteration updates... The parameter values ​​are also determined by the respective detection tasks, prompting different branch networks to learn their respective image detection tasks. This allows a single image detection model to first extract image detail features that can be shared across multiple detection tasks using a common network. Then, different branch networks within this model complete n image detection tasks based on these detail features. Thus, a single image detection model can accomplish n image detection tasks. Therefore, not only is iterative training of n image detection models unnecessary during the model training phase, improving training efficiency, but it also eliminates the need for separate training of n models during application, further enhancing efficiency. Furthermore, only one image detection model needs to be maintained subsequently, reducing maintenance costs.

[0047] Figure 1 This is a schematic diagram of a first flowchart illustrating a training method for an image detection model provided in one or more embodiments of this application. Figure 1 The method described above can be executed by an electronic device equipped with an image detection model training device. This electronic device can be a terminal device or a designated server. The hardware device used for image detection model training (i.e., the electronic device equipped with the image detection model training device) and the hardware device for image detection (i.e., the electronic device equipped with the image detection device) can be the same as or different from each other. Figure 1 As shown, the training method for this image detection model includes at least the following steps:

[0048] S102, Obtain the image sample dataset; wherein, the image sample dataset includes N target image samples;

[0049] Specifically, the N target image samples can come from open-source image datasets or from image data generated within a preset historical time period under a preset application scenario, where N is an integer greater than 1. For example, the preset application scenario is an application scenario for quality inspection of advertising materials. Since the demand for advertising materials is relatively large, the advertising materials are usually designed by outsourced personnel. Therefore, it is necessary to perform image detection on the advertising materials to be promoted, and only those that meet the requirements will be promoted. Advertising materials refer to advertising materials that are placed on various traffic-driving channels for business promotion. Correspondingly, the N target image samples can be advertising materials placed within a preset historical time period.

[0050] S104, Based on the above N target image samples, perform iterative training on the image detection model to be trained to obtain the trained image detection model;

[0051] Specifically, after obtaining N target image samples, the model parameters in the image detection model to be trained are iteratively updated based on the N target image samples until the current model training result meets the preset model training termination condition, thus obtaining an image detection model for multi-image detection tasks; wherein, the preset model training termination condition may include either: the current model training round number equals the total training round number, or the model loss function converges.

[0052] In specific implementation, regarding the model iterative training process in step S104 above, the specific implementation process of model iterative training is explained below. Since the processing procedure is the same for each model training iteration, a detailed explanation is given using any single model training as an example. Specifically, if the above image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network; the first feature extraction network is a shared network for n image detection tasks, the second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks, and the above feature extraction sub-networks, label prediction sub-networks, and image detection tasks correspond one-to-one; as follows: Figure 2 As shown, each model training iteration can be implemented using the following steps S1041 to S1043:

[0053] S1041, for each target image sample: the first feature extraction network extracts features based on the target image sample to obtain first image feature information; each feature extraction sub-network extracts features based on the first image feature information to obtain second image feature information; each label prediction sub-network predicts labels based on the first image feature information and the second image feature information to obtain target predicted labels;

[0054] Specifically, both the first feature extraction network and the feature extraction sub-networks can be convolutional layers used for feature extraction. The kernel size of the first feature extraction network is smaller than that of the feature extraction sub-networks. The kernel sizes of the n feature extraction sub-networks can be the same. Correspondingly, the receptive field of the first feature extraction network is smaller than that of the feature extraction sub-networks. The receptive fields of the n feature extraction sub-networks can be the same. The receptive field refers to the size of the region mapped by a pixel on the feature map output by each layer of the convolutional neural network to the input image. That is, one point on the feature map corresponds to a region on the input image.

[0055] Specifically, the first and second image feature information mentioned above can both be image feature maps output by different convolutional layers after feature extraction. The projection area of ​​a pixel on the target image sample of the first image feature map (i.e., the first image feature information) output by the first feature extraction network is smaller than the projection area of ​​a pixel on the target image sample of the second image feature map (i.e., the second image feature information) output by the feature extraction sub-network. That is, the size of the first image feature map is larger than the size of the second image feature map. In other words, each pixel in the first image feature map corresponds to a small region of image features in the target image sample (i.e., the first feature extraction network is used to extract image detail features), and each pixel in the second image feature map corresponds to a large region of image features in the target image sample (i.e., each feature extraction sub-network in the second feature extraction network is used to extract global features of the image region). Since different image detection tasks correspond to different target ground truth labels, the sub-loss values ​​used when iteratively updating the parameters of different feature extraction sub-networks are different, and thus the parameter values ​​of different feature extraction sub-networks are also different. Therefore, the second image feature information output by different feature extraction sub-networks corresponds to the image detection task.

[0056] Specifically, since different image detection tasks correspond to different target ground truth labels, the sub-loss values ​​used when iterating and updating the parameters of different label prediction sub-networks are also different. As a result, the parameter values ​​of different label prediction sub-networks are also different, and the target prediction labels output by different label prediction sub-networks are also corresponding to the image detection task.

[0057] For example, taking an image detection process involving three image detection tasks (where n equals 3), the second feature extraction network includes feature extraction sub-network 1, feature extraction sub-network 2, and feature extraction sub-network 3, and the detection label prediction network includes label prediction sub-network 1, label prediction sub-network 2, and label prediction sub-network 3. Specifically, for the determination process of the target prediction label corresponding to each image detection task, firstly, the first feature extraction network performs feature extraction based on the target image sample to obtain the first image feature information; then, the first image feature information is input into feature extraction sub-network 1, feature extraction sub-network 2, and feature extraction sub-network 3 respectively, and feature extraction sub-network 1, based on the aforementioned first image feature information... The first image feature information is extracted to obtain second image feature information 1; the label prediction subnetwork 1 performs label prediction based on the first image feature information and second image feature information 1 to obtain target predicted label 1; the feature extraction subnetwork 2 performs feature extraction based on the first image feature information to obtain second image feature information 2; the label prediction subnetwork 2 performs label prediction based on the first image feature information and second image feature information 2 to obtain target predicted label 2; the feature extraction subnetwork 3 performs feature extraction based on the first image feature information to obtain second image feature information 3; the label prediction subnetwork 3 performs label prediction based on the first image feature information and second image feature information 3 to obtain target predicted label 3.

[0058] It should be noted that, in the process of label prediction based on image feature information, the label prediction subnetwork may predict the target predicted label based solely on the second image feature information output by the corresponding feature extraction subnetwork; alternatively, in specific implementation, it may also predict the target predicted label based on the first image feature information output by the first feature extraction network and the second image feature information output by the corresponding feature extraction subnetwork; the specific implementation can be set according to actual needs, and all are within the protection scope of this application.

[0059] S1042, Based on the above-mentioned target prediction label and target real label, determine the sub-loss set corresponding to the above-mentioned second feature extraction network; wherein, the target real label corresponds to the image detection task corresponding to the above-mentioned feature extraction sub-network, and the above-mentioned sub-loss set includes sub-loss values ​​corresponding to n feature extraction sub-networks respectively.

[0060] Specifically, for each image detection task, a sub-loss value is determined based on the target predicted label and the target true label corresponding to the image detection task. This sub-loss value is then used as the sub-loss value of the feature extraction sub-network corresponding to the image detection task. The set of sub-loss values ​​of each feature extraction sub-network is then used as the sub-loss set corresponding to the second feature extraction network. The sub-loss value corresponding to image detection task i is determined based on the loss components i of multiple target image samples used in each round of model training. That is, for each target image sample, the loss component i of the target image sample under image detection task i is calculated based on the target predicted label i and the target true label i of the target image sample. Then, the sub-loss value corresponding to image detection task i is determined based on the loss components i of multiple target image samples.

[0061] For example, taking the second feature extraction network, which includes feature extraction sub-network 1, feature extraction sub-network 2, and feature extraction sub-network 3, and the detection label prediction network, which includes label prediction sub-network 1, label prediction sub-network 2, and label prediction sub-network 3, as an example, for each image detection task 1, based on the target predicted label 1 and the target true label 1 corresponding to image detection task 1, a sub-loss value 1 is determined, and this sub-loss value 1 is determined as the sub-loss value of the feature extraction sub-network 1 corresponding to image detection task 1; wherein, the sub-loss value 1 is determined based on the loss components 1 of multiple target image samples used in each round of model training, that is, for each target image sample, based on the target predicted label 1 and the target true label 1 of the target image sample, the loss component 1 of the target image sample under image detection task 1 is calculated, and then the sub-loss value 1 corresponding to image detection task 1 is determined based on the loss components 1 of multiple target image samples; and, for each image detection task 2, based on the target predicted label 2 and the target true label 2 corresponding to image detection task 2, a sub-loss value 2 is determined, and this sub-loss value 2 is determined as the sub-loss value of the feature extraction sub-network 2 corresponding to image detection task 2; wherein, the sub-loss value 1 is determined based on the target predicted label 2 and the target true label 2 corresponding to image detection task 2; wherein, the sub-loss value 1 is determined based on the target predicted label 2 and the target true label 2 corresponding to image detection task 2. Value 2 is determined based on the loss components 2 of multiple target image samples used in each round of model training. That is, for each target image sample, based on the target predicted label 2 and the target true label 2 of the target image sample, the loss component 2 of the target image sample under image detection task 2 is calculated, and then the sub-loss value 2 corresponding to image detection task 2 is determined based on the loss components 2 of multiple target image samples. For each image detection task 3, based on the target predicted label 3 and the target true label 3 corresponding to image detection task 3, a sub-loss value 3 is determined, and this sub-loss value 3 is determined as the sub-loss value of the feature extraction sub-network 3 corresponding to image detection task 3. Here, the sub-loss value 3 is determined based on the loss components 3 of multiple target image samples used in each round of model training. That is, for each target image sample, based on the target predicted label 3 and the target true label 3 of the target image sample, the loss component 3 of the target image sample under image detection task 3 is calculated, and then the sub-loss value 3 corresponding to image detection task 3 is determined based on the loss components 3 of multiple target image samples. The set of the above sub-loss values ​​1, 2, and 3 is determined as the sub-loss set corresponding to the second feature extraction network.

[0062] S1043, based on the sub-loss values ​​corresponding to each feature extraction sub-network, updates the parameters of the image detection model to be trained.

[0063] In the image detection model, the common network (i.e., the first feature extraction network) needs to consider the sub-loss values ​​corresponding to each image detection task. Therefore, the parameters of the first feature extraction network are iteratively updated based on the sub-loss values ​​corresponding to each image detection task. Correspondingly, for the branch networks (i.e., the feature extraction sub-network and the label prediction sub-network) in the image detection model, since the target true label and target predicted label are different for different image detection tasks, the sub-loss values ​​corresponding to each image detection task are also different. Based on the sub-loss values ​​corresponding to each image detection task, the parameters of the corresponding feature extraction sub-network and label prediction sub-network are iteratively updated, prompting different branch networks to learn different image detection knowledge, thereby enabling different branch networks to learn different image detection tasks.

[0064] For example, taking the second feature extraction network as including feature extraction sub-network 1, feature extraction sub-network 2, and feature extraction sub-network 3, and the label prediction network as including label prediction sub-network 1, label prediction sub-network 2, and label prediction sub-network 3 as an example, the parameters of the first feature extraction network are iteratively updated based on the above sub-loss values ​​1, 2, and 3; and the parameters of feature extraction sub-network 1 and label prediction sub-network 1 are iteratively updated based on the above sub-loss value 1, the parameters of feature extraction sub-network 2 and label prediction sub-network 2 are iteratively updated based on the above sub-loss value 2, and the parameters of feature extraction sub-network 3 and label prediction sub-network 3 are iteratively updated based on the above sub-loss value 3.

[0065] Specifically, after determining the sub-loss set, the gradient descent method is used to iteratively update the parameters of the first feature extraction network based on the sub-loss values ​​corresponding to each image detection task; and for each image detection task, the gradient descent method is used to iteratively update the parameters of the feature extraction sub-network and label prediction sub-network corresponding to that image detection task based on the sub-loss values ​​corresponding to that image detection task.

[0066] It should be noted that the process of iteratively updating the parameters of the first feature extraction network, the feature extraction sub-network, and the label prediction sub-network using the gradient descent method can be found in the existing process of optimizing model parameters through backpropagation using the gradient descent method, and will not be repeated here.

[0067] In this embodiment, the image detection model includes a shared network (i.e., a first feature extraction network) and branch networks (i.e., feature extraction sub-networks and label prediction sub-networks) corresponding to each image detection task. The shared network extracts image detail features from the image, which serve as input data for each branch network. Each branch network then learns differentiated image detection knowledge based on these image detail features and the target ground truth labels corresponding to its respective image detection task. Since the target ground truth labels are different for different image detection tasks, the parameter values ​​of each branch network after multiple rounds of parameter iteration updates are also determined by its respective detection task. This allows an image detection model to first extract image detail features that can be shared by multiple detection tasks using a shared network, and then different branch networks in the image detection model to complete n image detection tasks based on these image detail features. In this way, n image detection tasks can be achieved with one image detection model. Therefore, not only is iterative training of n image detection models not required during the model training phase, thus improving model training efficiency, but it is also not required to use n image detection models to complete an image detection task during the model application phase, thus improving image detection efficiency. Furthermore, only one image detection model needs to be maintained subsequently, thereby reducing model maintenance costs.

[0068] In one specific embodiment, the image detection model described above includes a first feature extraction network, a second feature extraction network, and a detection label prediction network; the first feature extraction network is a shared network for n image detection tasks, the second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks, with each feature extraction sub-network, label prediction sub-network, and image detection task corresponding one-to-one; as shown... Figure 3 As shown, a schematic diagram illustrating the first specific implementation principle of the image detection model training method is presented, mainly including:

[0069] Obtain an image sample dataset containing N target image samples;

[0070] Randomly select X target image samples from N target image samples; input the X target image samples into the first feature extraction network of the image detection model to be trained;

[0071] For each target image sample, the first feature extraction network extracts features based on the target image sample to obtain first image feature information; the feature extraction sub-network 1 extracts features based on the first image feature information to obtain second image feature information 1, and the feature extraction sub-network n extracts features based on the first image feature information to obtain second image feature information n; the label prediction sub-network 1 predicts labels based on the first image feature information and second image feature information 1 to obtain target predicted label 1, and the label prediction sub-network n predicts labels based on the first image feature information and second image feature information n to obtain target predicted label n;

[0072] Based on the target predicted label 1 and target true label 1 corresponding to each target image sample, the sub-loss value 1 corresponding to image detection task 1 is determined; and the parameters of feature extraction sub-network 1 and label prediction sub-network 1 are iteratively updated based on the sub-loss value 1; wherein, the sub-loss value 1 is determined based on the loss components 1 of the X target image samples used in this round of model training, that is, for each target image sample, based on the target predicted label 1 and target true label 1 of the target image sample, the loss component 1 of the target image sample under image detection task 1 is calculated, and then the sub-loss value 1 corresponding to image detection task 1 is determined based on the loss components 1 of X target image samples;

[0073] Based on the target predicted label n and the target true label n corresponding to each target image sample, the sub-loss value n corresponding to the image detection task n is determined; and the parameters of the feature extraction sub-network n and the label prediction sub-network n are iteratively updated based on the sub-loss value n; wherein, the sub-loss value n is determined based on the loss components n of the X target image samples used in this round of model training, that is, for each target image sample, based on the target predicted label n and the target true label n of the target image sample, the loss component n of the target image sample under the image detection task n is calculated, and then the sub-loss value n corresponding to the image detection task n is determined based on the loss components n of the X target image samples.

[0074] The target loss value is determined based on the sub-loss value 1 and the loss value n; and the parameters of the first feature extraction network are iteratively updated based on the target loss value.

[0075] The trained image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network after parameter iteration updates. The second feature extraction network includes n feature extraction sub-networks after parameter iteration updates, and the detection label prediction network includes n label prediction sub-networks after parameter iteration updates.

[0076] It should be noted that the target image sample can be directly input into the first feature extraction network of the image detection model to be trained; alternatively, the target image sample can be preprocessed first, and then the preprocessed target image sample can be input into the first feature extraction network of the image detection model to be trained. The preprocessing may include image scaling, random cropping to a specified size, etc. The settings can be made according to actual needs, and all are within the scope of protection of this application.

[0077] Specifically, the first feature extraction network may include only one convolutional layer or multiple convolutional layers with different kernel sizes. If the first feature extraction network includes multiple convolutional layers, the process of extracting features from the target image sample to obtain the first image feature information can be achieved by performing multiple feature extractions sequentially. That is, the input of the first convolutional layer in the first feature extraction network is the target image sample, the image feature information output by the previous convolutional layer is used as the input of the next convolutional layer, and the output of the last convolutional layer (i.e., the last convolutional layer) in the first feature extraction network is the first image feature information, thereby improving the image feature information input to each branch network to contain more valuable image features. Specifically, the first feature extraction network includes m convolutional layers connected in sequence; the kernel size of the subsequent convolutional layer is larger than the kernel size of the previous convolutional layer; and the kernel size of the last convolutional layer in the first feature extraction network is smaller than the kernel size of the feature extraction sub-network; n feature extraction sub-networks are respectively connected to the last convolutional layer in the first feature extraction network.

[0078] Correspondingly, the first feature extraction network in S1041 above performs feature extraction based on the target image samples to obtain the first image feature information, specifically including:

[0079] (1) The first convolutional layer in the first feature extraction network extracts features based on the target image sample to obtain initial image feature information; wherein, the initial image feature information is used as the image feature information output by the previous convolutional layer of the second convolutional layer;

[0080] (2) In the first feature extraction network, the non-first and non-last convolutional layers extract features based on the image feature information output by the previous convolutional layer to obtain intermediate image feature information; wherein, the intermediate image feature information is used as the image feature information output by the previous convolutional layer of the p-th convolutional layer, 2 < p ≤ m;

[0081] (3) The last convolutional layer in the first feature extraction network performs feature extraction based on the image feature information output by the previous convolutional layer to obtain the first image feature information.

[0082] Specifically, taking a value of 4 for m as an example, the first feature extraction network mentioned above includes convolutional layer 1, convolutional layer 2, convolutional layer 3, and convolutional layer 4 connected in sequence; the convolutional kernel size from high to low is: convolutional layer 4, convolutional layer 3, convolutional layer 2, and convolutional layer 1; convolutional layer 1 performs feature extraction based on the target image sample to obtain initial image feature information; the initial image feature information is input to convolutional layer 2, and convolutional layer 2 performs feature extraction based on the initial image feature information to obtain intermediate image feature information a; intermediate image feature information a is input to convolutional layer 2, and convolutional layer 3 performs feature extraction based on intermediate image feature information a to obtain intermediate image feature information b; intermediate image feature information b is input to convolutional layer 4, and convolutional layer 4 performs feature extraction based on intermediate image feature information b to obtain first image feature information.

[0083] Specifically, in the process of label prediction based on image feature information, the label prediction subnetwork can predict the target predicted label based solely on the second image feature information output by the corresponding feature extraction subnetwork; alternatively, it can also predict the target predicted label based on the first image feature information output by the first feature extraction network and the second image feature information output by the corresponding feature extraction subnetwork; more preferably, when the first feature extraction network includes multiple convolutional layers, it can also predict the target predicted label based on the first image feature information output by the last convolutional layer in the first feature extraction network, the image feature information output by at least one of the multiple non-last convolutional layers in the first feature extraction network, and the second image feature information output by the corresponding feature extraction subnetwork; specifically, each label prediction subnetwork in S1041 predicts the target predicted label based on the aforementioned first image feature information and second image feature information, specifically including:

[0084] Step 1: Each label prediction subnetwork merges the first image feature information, the preceding image feature information, and the second image feature information output by the corresponding feature extraction subnetwork to obtain the target image feature information; wherein, the preceding image feature information includes at least one of the image feature information output by the first (m-1) convolutional layers in the first feature extraction network, that is, the preceding image feature information includes the image feature information output by at least one of the multiple non-tail convolutional layers in the first feature extraction network;

[0085] Step 2: Each label prediction subnetwork performs label prediction based on the target image feature information to obtain the target predicted label; wherein, the target predicted label corresponds to the image detection task corresponding to the label prediction subnetwork; specifically, firstly, the corresponding probability map and threshold map are obtained based on the target image feature information, then differentiable binarization is performed based on the probability map and threshold map to obtain the corresponding binary map, and then the target predicted label corresponding to the image detection task is extracted based on the binary map.

[0086] Specifically, taking the value of m equal to 4 as an example, the first feature extraction network mentioned above includes convolutional layer 1, convolutional layer 2, convolutional layer 3, and convolutional layer 4 connected in sequence; the image feature information output by convolutional layer 1 is the initial image feature information mentioned above, the image feature information output by convolutional layer 2 is the intermediate image feature information a mentioned above, the image feature information output by convolutional layer 3 is the intermediate image feature information b mentioned above, and the image feature information output by convolutional layer 4 is the first image feature information mentioned above. Correspondingly, the aforementioned pre-image feature information includes at least one of the initial image feature information, intermediate image feature information a, and intermediate image feature information b.

[0087] Specifically, regarding the process of merging multiple image feature information to obtain target image feature information, since the convolution kernel sizes of multiple network layers and feature extraction sub-networks in the first feature extraction network are different, the sizes of the aforementioned first image feature information (i.e., the first image feature map), the preceding image feature information (i.e., the preceding image feature map), and the second image feature information (i.e., the second image feature map) are also different. Therefore, it is necessary to first upsample the first image feature information, the preceding image feature information, and the second image feature information to obtain multiple upsampled feature maps of the same size, and then merge the multiple upsampled feature maps to obtain the target image feature information.

[0088] For example, taking the second feature extraction network as including feature extraction subnetwork 1, feature extraction subnetwork 2, and feature extraction subnetwork 3, and the label prediction network as including label prediction subnetwork 1, label prediction subnetwork 2, and label prediction subnetwork 3, and the preceding image feature information as including initial image feature information, intermediate image feature information a, and intermediate image feature information b, specifically, label prediction subnetwork 1 first upsamples the initial image feature information, intermediate image feature information a, intermediate image feature information b, first image feature information, and second image feature information 1 to obtain multiple upsampled feature maps of the same size, and then merges the multiple upsampled feature maps to obtain target image feature information 1; then, label prediction is performed based on target image feature information 1 to obtain target predicted label 1; label prediction subnetwork... 2. First, the initial image feature information, intermediate image feature information a, intermediate image feature information b, first image feature information, and second image feature information 2 are upsampled to obtain multiple upsampled feature maps of the same size. Then, the multiple upsampled feature maps are merged to obtain target image feature information 2. Then, label prediction is performed based on target image feature information 2 to obtain target predicted label 2. Label prediction subnetwork 3 first, the initial image feature information, intermediate image feature information a, intermediate image feature information b, first image feature information, and second image feature information 3 are upsampled to obtain multiple upsampled feature maps of the same size. Then, the multiple upsampled feature maps are merged to obtain target image feature information 3. Then, label prediction is performed based on target image feature information 3 to obtain target predicted label 3.

[0089] In one specific embodiment, the first feature extraction network includes m convolutional layers connected sequentially; the kernel size of each subsequent convolutional layer is larger than the kernel size of the preceding convolutional layer; in the above... Figure 3 On the basis of, such as Figure 4 As shown, a schematic diagram illustrating the second specific implementation principle of the image detection model training method is presented, mainly including:

[0090] The specific implementation methods for the first feature extraction network to extract features from target image samples and obtain first image feature information are as follows:

[0091] In the first feature extraction network, convolutional layer 1 extracts features based on target image samples to obtain initial image feature information; wherein, this initial image feature information serves as the image feature information input to convolutional layer 2;

[0092] In the first feature extraction network, convolutional layer j extracts features based on the image feature information output by convolutional layer (j-1) to obtain intermediate image feature information; wherein, the intermediate image feature information is used as the input image feature information of convolutional layer (j+1), and 2≤j<m;

[0093] In the first feature extraction network, convolutional layer m extracts features based on the image feature information output by convolutional layer (m-1) to obtain the first image feature information.

[0094] It should be noted that, when the first feature extraction network includes m convolutional layers, the process for determining the target predicted label can be as follows: The target predicted label can be obtained by predicting the label based solely on the first image feature information output by the m-th convolutional layer and the second image feature information output by the corresponding feature extraction sub-network; or the target predicted label can be obtained by predicting the label based on the initial image feature information output by the 1st convolutional layer, the intermediate image feature information output by the j-th convolutional layer, the first image feature information output by the m-th convolutional layer, and the second image feature information output by the corresponding feature extraction sub-network. In this case, for the above... Figure 4 For the sake of simplicity, this paper only illustrates the process of predicting the target label based on the first image feature information output by the m-th convolutional layer and the second image feature information output by the corresponding feature extraction subnetwork. This does not constitute a limitation of this application, and other feasible implementation methods are all within the protection scope of this application.

[0095] In specific implementation, for the application scenario of advertising, the above target image sample includes the sample of the advertising material, the above n image detection tasks can include text box detection tasks and image tampering detection tasks; the above target real labels can include text box position real labels and image tampering area real labels, and the above target prediction labels can include text box position prediction labels and image tampering area prediction labels.

[0096] Correspondingly, in S1042 above, based on the predicted target label and the true target label, the sub-loss set corresponding to the second feature extraction network is determined, specifically including:

[0097] Step 1: Based on the predicted text box location labels and the true text box location labels, determine the first sub-loss value. This first sub-loss value corresponds to the text box detection task. Specifically, the first sub-loss value is determined based on the text box detection loss components of multiple target image samples used in each round of model training. That is, for each target image sample, based on the predicted text box location labels and the true text box location labels of that target image sample, calculate the text box detection loss component of that target image sample under the text box detection task, and then determine the first sub-loss value corresponding to the text box detection task based on the text box detection loss components of multiple target image samples.

[0098] Step 2: Based on the predicted and true labels of the image tampering regions, determine the second sub-loss value. The first sub-loss value corresponds to the image tampering detection task. Specifically, the first sub-loss value is determined based on the image tampering detection loss components of multiple target image samples used in each round of model training. That is, for each target image sample, based on the predicted and true labels of the image tampering regions, calculate the image tampering detection loss component of the target image sample under the image tampering detection task, and then determine the second sub-loss value corresponding to the image tampering detection task based on the image tampering detection loss components of multiple target image samples.

[0099] Step 3: Based on the first and second sub-loss values ​​mentioned above, determine the sub-loss set corresponding to the second feature extraction network.

[0100] Specifically, regarding the determination process of the first and second sub-loss values ​​mentioned above, the corresponding sub-loss values ​​can be calculated based on the predicted label, the true label, and the commonly used cross-entropy loss function; alternatively, other loss functions can be used to calculate the corresponding sub-loss values. This application does not limit the specific methods used, and all of these methods are within the scope of protection of this application.

[0101] Furthermore, in the process of extracting features from the feature extraction subnetwork to obtain the second image feature information, if the above n image detection tasks include text box detection tasks and image tampering detection tasks, then the above n feature extraction subnetworks include a first extraction subnetwork and a second extraction subnetwork.

[0102] Correspondingly, each feature extraction sub-network in S1041 above performs feature extraction based on the first image feature information to obtain the second image feature information, specifically including:

[0103] (1) The first extraction sub-network performs feature extraction based on the first image feature information to obtain text box representation information; the text box representation information is determined as the second image feature information output by the first extraction sub-network;

[0104] (2) The second extraction sub-network performs feature extraction based on the first image feature information to obtain image tampering characterization information; the image tampering characterization information is determined as the second image feature information output by the second extraction sub-network.

[0105] Specifically, since different image detection tasks correspond to different target ground truth labels, the parameter values ​​of different feature extraction sub-networks are also different after multiple rounds of model training. Therefore, the second image feature information output by different feature extraction sub-networks corresponds to the image detection task. For example, the second image feature information output by the first extraction sub-network corresponds to the text box detection task, that is, the first extraction sub-network focuses on learning how to extract feature information from the image that helps to identify the area where the text box is located. The second image feature information output by the second extraction sub-network corresponds to the image tampering detection task, that is, the second extraction sub-network focuses on learning how to extract feature information from the image that helps to identify the area where the image tampering traces are located.

[0106] Furthermore, in the process of obtaining the target predicted label by performing label prediction for the label prediction subnetwork, if the above n image detection tasks include text box detection tasks and image tampering detection tasks, then the above n label prediction subnetworks include a first prediction subnetwork and a second prediction subnetwork.

[0107] Correspondingly, in step two above, each label prediction sub-network performs label prediction based on the aforementioned target image feature information to obtain the target predicted label, specifically including:

[0108] (1) The first prediction sub-network performs label prediction based on the above target image feature information to obtain the text box position prediction label; the above text box position prediction label is determined as the target prediction label corresponding to the above text box detection task.

[0109] (2) The second prediction sub-network performs label prediction based on the above target image feature information to obtain the image tampering region prediction label; the above image tampering region prediction label is determined as the target prediction label corresponding to the above image tampering region detection task.

[0110] Specifically, since different image detection tasks correspond to different target ground truth labels, the parameter values ​​of different feature extraction sub-networks are also different after multiple rounds of model training. The second image feature information output by different feature extraction sub-networks corresponds to the image detection task. Based on this, the target image feature information obtained by different label prediction sub-networks through feature merging based on the second image feature information also corresponds to the image detection task. Therefore, the target predicted label obtained by label prediction based on the target image feature information also corresponds to the image detection task. For example, the target predicted label output by the first prediction sub-network corresponds to the text box detection task, that is, the first prediction sub-network focuses on learning how to extract the area where the text box is located based on the target image feature information. The target predicted label output by the second prediction sub-network corresponds to the image tampering detection task, that is, the second prediction sub-network focuses on learning how to extract the area where the image tampering traces are located based on the target image feature information.

[0111] In one specific embodiment, the first feature extraction network includes convolutional layer 1, convolutional layer 2, convolutional layer 3, and convolutional layer 4 connected in sequence; the convolutional kernel sizes, from high to low, are: convolutional layer 4, convolutional layer 3, convolutional layer 2, and convolutional layer 1; the target image samples include projected material samples; the n image detection tasks include text box detection and image tampering detection tasks; the target ground truth labels include text box location ground truth labels and image tampering region ground truth labels; the target predicted labels include text box location predicted labels and image tampering region predicted labels. The n feature extraction subnetworks include a first extraction subnetwork (i.e., convolutional layer 51) and a second extraction subnetwork (i.e., convolutional layer 52). The kernel sizes of convolutional layers 51 and 52 are the same, and the kernel sizes of both convolutional layers 51 and 52 are larger than the kernel size of convolutional layer 4. The n label prediction subnetworks include a first prediction subnetwork and a second prediction subnetwork. The first prediction subnetwork includes an image feature merging layer 1 and a detection label prediction layer 1, and the second prediction subnetwork includes an image feature merging layer 2 and a detection label prediction layer 2. Figure 5 As shown, a schematic diagram illustrating the third specific implementation principle of the image detection model training method is presented, mainly including:

[0112] Obtain an image sample dataset containing N target image samples;

[0113] Randomly select X target image samples from N target image samples; input the X target image samples into convolutional layer 1 of the image detection model to be trained;

[0114] For each target image sample, convolutional layer 1 extracts features based on the target image sample to obtain initial image feature information; convolutional layer 2 extracts features based on the initial image feature information to obtain intermediate image feature information a; convolutional layer 3 extracts features based on intermediate image feature information a to obtain intermediate image feature information b; convolutional layer 4 extracts features based on intermediate image feature information b to obtain first image feature information; then, convolutional layer 51 extracts features based on the first image feature information to obtain second image feature information 1; image feature merging layer 1 upsamples the initial image feature information, intermediate image feature information a, intermediate image feature information b, first image feature information, and second image feature information 1 to obtain multiple upsampled feature maps of the same size. The image feature prediction layer 1 performs label prediction based on the target image feature information 1 to obtain the text box position prediction label; the convolutional layer 52 performs feature extraction based on the first image feature information to obtain the second image feature information 2; the image feature merging layer 2 upsamples the initial image feature information, intermediate image feature information a, intermediate image feature information b, first image feature information and second image feature information 2 to obtain multiple upsampled feature maps of the same size, and merges the multiple upsampled feature maps to obtain the target image feature information 2; the detection label prediction layer 2 performs label prediction based on the target image feature information 2 to obtain the image tampering region prediction label;

[0115] Based on the predicted and true labels of the text box positions corresponding to each target image sample, the first sub-loss value corresponding to the text box detection task is determined; and the parameters of convolutional layer 51, image feature merging layer 1 and detection label prediction layer 1 are iteratively updated based on the first sub-loss value.

[0116] Based on the predicted label and the true label of the image tampering region corresponding to each target image sample, the second sub-loss value corresponding to the image tampering detection task is determined; and the parameters of convolutional layer 52, image feature merging layer 2 and detection label prediction layer 2 are iteratively updated based on the second sub-loss value.

[0117] The target loss value is determined based on the first and second sub-loss values; and the parameters of convolutional layer 1, convolutional layer 2, convolutional layer 3, and convolutional layer 4 are iteratively updated based on the target loss value.

[0118] It should be noted that each feature extraction subnetwork may include one convolutional layer, or two or more convolutional layers, depending on actual needs; this application does not impose any limitations on this. Furthermore, in the above... Figure 5The illustration shows the process of predicting the target label based on the image feature information output by each convolutional layer in the first feature extraction network (i.e., initial image feature information, intermediate image feature information a, intermediate image feature information b, and first image feature information) and the second image feature information output by the corresponding feature extraction sub-network. This does not constitute a limitation of this application. It is also possible to predict the target label based solely on the first image feature information and the corresponding second image feature information, all of which are within the scope of protection of this application.

[0119] Furthermore, regarding the process of updating model parameters, if the above n image detection tasks include text box detection tasks and image tampering detection tasks, then the sub-loss values ​​corresponding to each of the above feature extraction sub-networks include the above first sub-loss value and second sub-loss value.

[0120] Correspondingly, in S1043 above, based on the sub-loss values ​​corresponding to each feature extraction sub-network, the parameters of the image detection model to be trained are updated, specifically including:

[0121] (1) Based on the first sub-loss value mentioned above, update the parameters of the feature extraction sub-network and the label prediction sub-network corresponding to the text box detection task; specifically, use the gradient descent method to update the parameters of the first extraction sub-network and the first prediction sub-network based on the first sub-loss value.

[0122] (2) Based on the second sub-loss value mentioned above, the parameters of the feature extraction sub-network and the label prediction sub-network corresponding to the image tampering detection task are updated; specifically, the gradient descent method is used to update the parameters of the second extraction sub-network and the second prediction sub-network based on the second sub-loss value.

[0123] (3) Based on the first sub-loss value and the second sub-loss value mentioned above, determine the target loss value; and update the parameters of the first feature extraction network based on the target loss value; specifically, the target loss value can be the sum of the first sub-loss value and the second sub-loss value, or it can be a weighted sum of the first sub-loss value and the second sub-loss value, that is, the sum of the product of the first weight coefficient and the first sub-loss value, and the product of the second weight coefficient and the second sub-loss value; then, use the gradient descent method to update the parameters of the first feature extraction network based on the target loss value.

[0124] Furthermore, regarding the process of acquiring the image sample dataset, it can be done by directly acquiring existing image samples, meaning that all image samples in the image sample dataset are existing image samples. However, considering that the model training process requires a large number of image samples, when the number of image samples with true labels is relatively small, it is also possible to first acquire a small number of existing image samples, and then perform data augmentation processing based on the existing image samples to obtain more image samples. Specifically, in S102 above, acquiring the image sample dataset includes:

[0125] Step 1: Obtain the initial sample dataset; the initial sample dataset includes M initial image samples;

[0126] Step two: Perform data augmentation processing on the above M initial image samples to obtain a data augmentation sample set; the above data augmentation sample set includes P data augmentation image samples, P≥(NM); specifically, data augmentation is used to change the ground truth labels of the initial image samples to generate new image samples; wherein, the data augmentation method may include at least one of the following enhancement processes: deletion, copying, or moving the image region corresponding to the ground truth label in the initial image sample; for example, if the ground truth label includes the ground truth label of the text box position, then at least one of the following enhancement processes can be performed on the existing text box in the initial image sample: deletion, copying, or moving.

[0127] Step 3: Based on the above P data-enhanced image samples and the above M initial image samples, generate an image sample dataset containing N target image samples; specifically, if P = (NM), then each initial image sample is used as a target image sample, and each data-enhanced image sample is used as a target image sample; if P > (NM), then N target image samples can be selected from the M initial image samples and the P data-enhanced image samples.

[0128] Specifically, after obtaining N target image samples, the model parameters of the image detection model to be trained can be iteratively updated based on the N target image samples to obtain the trained image detection model. In practice, for each round of model training, X target image samples can be randomly selected from the N target image samples as input to the first feature extraction network, and the parameters of the first feature extraction network, the second feature extraction network, and the detection label prediction network can be updated. After multiple rounds of parameter updates, the trained image detection model can be obtained.

[0129] In practical implementation, considering the case of n image detection tasks including text box detection and image tampering detection, enhancing the text boxes in the image samples is equivalent to adding image tampering traces. Therefore, the image tampering region's true label can be added to the image samples based on the text box processing. That is, the data enhancement process of the text boxes simultaneously completes the data enhancement of the image tampering region, without adding additional image tampering traces. It can also obtain a large number of new image samples containing image tampering traces and increase the complexity of the image tampering true label in the image samples, thereby ensuring the parameter training accuracy of the branch network corresponding to the image tampering detection task. Step two above, based on the above M initial image samples, performs data enhancement processing to obtain a data-enhanced sample set, specifically including:

[0130] For each of the aforementioned initial image samples, based on the text box marking information of the aforementioned initial image samples, at least one target text box is randomly selected from the aforementioned initial image samples;

[0131] The target text box is subjected to preset processing to obtain a data-enhanced image sample; the preset processing includes at least one of moving the text box, deleting the text box, and copying the text box; the target real label of the data-enhanced image sample is determined based on the original real label and the data enhancement processing result of the target text box, and the target real label of the data-enhanced image sample includes the real label of the text box position and the real label of the image tampering area, and the real label of the image tampering area includes the original real label of the image tampering area and the new real label of the image tampering area generated based on the preset processing of the target text box;

[0132] Based on the above-mentioned data-enhanced image samples, a data-enhanced sample set is determined. Then, after obtaining an initial sample dataset containing M initial image samples and a data-enhanced sample set containing P data-enhanced image samples, if P = (NM), the set of M initial image samples and P data-enhanced image samples can be directly determined as the image sample dataset, that is, each initial image sample is a target image sample and each data-enhanced image sample is a target image sample. If P > (NM), then N target image samples can be selected from the M initial image samples and P data-enhanced image samples according to the preset sample selection rules.

[0133] The training method of the image detection model in this embodiment includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks, the second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, label prediction sub-networks, and image detection tasks correspond one-to-one. Specifically, the first feature extraction network extracts detailed image features, and the image feature information output by the first feature extraction network is input to each of the n feature extraction sub-networks to extract global features of the image region through the feature extraction sub-network corresponding to each image detection task. Then, the image feature information output by the feature extraction sub-networks is input to the corresponding label prediction sub-network for label prediction. In other words, the image detection model includes a shared network (i.e., the first feature extraction network) and branch networks (i.e., feature extraction sub-networks and label prediction sub-networks) corresponding to each image detection task. The shared network extracts detailed image features. Image detail features serve as input data for each branch network. Each branch network then learns differentiated image detection knowledge based on these details and the target ground truth labels corresponding to its respective image detection task. Since the target ground truth labels differ for different image detection tasks, the parameter values ​​of each branch network after multiple rounds of parameter iteration are determined by its specific detection task. This allows a single image detection model to first extract shared image detail features across multiple detection tasks using a common network. Then, different branch networks within this model complete n image detection tasks based on these details. Thus, a single image detection model can accomplish n image detection tasks. Therefore, not only is iterative training of n image detection models unnecessary during the training phase, improving training efficiency, but it also eliminates the need for separate training of n models for each image detection task during application, further enhancing efficiency. Furthermore, only one image detection model needs to be maintained subsequently, reducing maintenance costs.

[0134] Corresponding to the above Figures 1 to 5 Based on the same technical concept, the training method for the described image detection model, as described in this application embodiment, also provides an image detection method. Figure 6 This is a schematic flowchart of the image detection method provided in the embodiments of this application. Figure 6 The method described above can be executed by an electronic device equipped with an image detection apparatus. This electronic device can be a terminal device or a designated server. The hardware device used for image detection (i.e., the electronic device equipped with the image detection apparatus) and the hardware device used for training the image detection model (i.e., the electronic device equipped with the training apparatus for the image detection model) can be the same or different. For example... Figure 6 As shown, the method includes at least the following steps:

[0135] S602, acquire the image to be detected;

[0136] S604, The above-mentioned image to be detected is input into the trained image detection model for image detection to obtain the image detection result of the image to be detected; specifically, the trained image detection model can be a neural network model obtained based on the training method of the above-mentioned image detection model;

[0137] The image detection model described above includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks. The second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, label prediction sub-networks, and image detection tasks correspond one-to-one. The specific implementation methods for each image detection are as follows:

[0138] For each image to be detected: the first feature extraction network extracts features based on the image to be detected to obtain the third image feature information; each feature extraction sub-network extracts features based on the third image feature information to obtain the fourth image feature information; each label prediction sub-network predicts labels based on the above third image feature information and fourth image feature information to obtain the image sub-detection result;

[0139] Based on the image sub-detection results output by each label prediction sub-network, the image detection result of the image to be detected is obtained.

[0140] In this embodiment, the image detection model includes a shared network (i.e., a first feature extraction network) and branch networks (i.e., feature extraction sub-networks and label prediction sub-networks) corresponding to each image detection task. The shared network extracts image detail features from the image, which serve as input data for each branch network. Each branch network then learns differentiated image detection knowledge based on these image detail features and the target ground truth labels corresponding to its respective image detection task. Since the target ground truth labels are different for different image detection tasks, the parameter values ​​of each branch network after multiple rounds of parameter iteration updates are also determined by its respective detection task. This allows an image detection model to first extract image detail features that can be shared by multiple detection tasks using a shared network, and then different branch networks in the image detection model to complete n image detection tasks based on these image detail features. In this way, n image detection tasks can be achieved with one image detection model. Therefore, not only is iterative training of n image detection models not required during the model training phase, thus improving model training efficiency, but it is also not required to use n image detection models to complete an image detection task during the model application phase, thus improving image detection efficiency. Furthermore, only one image detection model needs to be maintained subsequently, thereby reducing model maintenance costs.

[0141] In one specific embodiment, the image detection model described above includes a first feature extraction network, a second feature extraction network, and a detection label prediction network; the first feature extraction network is a shared network for n image detection tasks, the second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks, with each feature extraction sub-network, label prediction sub-network, and image detection task corresponding one-to-one; as shown... Figure 7 As shown, a schematic diagram of the first specific implementation principle of the image detection method is given, which mainly includes:

[0142] Acquire the image to be detected;

[0143] The image to be detected is input into the first feature extraction network of the trained image detection model;

[0144] The first feature extraction network extracts features based on the image to be detected, obtaining third image feature information; the feature extraction sub-network 1 extracts features based on the third image feature information, obtaining fourth image feature information 1, and the feature extraction sub-network n extracts features based on the third image feature information, obtaining fourth image feature information n; the label prediction sub-network 1 predicts labels based on the third image feature information and fourth image feature information 1, obtaining image sub-detection result 1 (corresponding to target prediction label 1), and the label prediction sub-network n predicts labels based on the third image feature information and fourth image feature information n, obtaining image sub-detection result n (corresponding to target prediction label n);

[0145] Based on the above image sub-detection result 1 and image sub-detection result n, the image detection result of the image to be detected is obtained.

[0146] The first feature extraction network mentioned above includes m convolutional layers connected in sequence; the kernel size of the subsequent convolutional layer is larger than the kernel size of the preceding convolutional layer.

[0147] Correspondingly, the first feature extraction network described above extracts features from the image to be detected to obtain the third image feature information, specifically including:

[0148] (1) The first convolutional layer in the first feature extraction network performs feature extraction based on the image to be detected to obtain initial image feature information; wherein, the initial image feature information is used as the image feature information output by the previous convolutional layer of the second convolutional layer;

[0149] (2) The non-first and non-last convolutional layers in the first feature extraction network perform feature extraction based on the image feature information output by the previous convolutional layer to obtain intermediate image feature information; wherein, the intermediate image feature information is the image feature information output by the previous convolutional layer of the p-th convolutional layer, 2 < p ≤ m;

[0150] (3) The last convolutional layer in the first feature extraction network performs feature extraction based on the image feature information output by the previous convolutional layer to obtain the third image feature information.

[0151] Specifically, the process for determining the third image feature information can refer to the process for determining the first image feature information, and will not be repeated here.

[0152] Each of the above label prediction sub-networks performs label prediction based on the third and fourth image feature information to obtain image sub-detection results, specifically including:

[0153] Step 1: Each label prediction sub-network merges the third image feature information, the preceding image feature information, and the fourth image feature information output by the corresponding feature extraction sub-network to obtain the target image feature information; wherein, the preceding image feature information includes at least one of the image feature information output by the first (m-1) convolutional layers in the first feature extraction network;

[0154] Step 2: Each label prediction sub-network performs label prediction based on the target image feature information mentioned above, and obtains the image sub-detection result; wherein, the image sub-detection result corresponds to the image detection task corresponding to the label prediction sub-network.

[0155] Specifically, the process for determining the image sub-detection results can refer to the process for determining the target prediction labels, and will not be repeated here.

[0156] In a specific embodiment, the first feature extraction network includes convolutional layers 1, 2, 3, and 4 connected in sequence; the kernel sizes from high to low are: convolutional layer 4, convolutional layer 3, convolutional layer 2, and convolutional layer 1; the image to be detected includes the projection material to be detected, and the n image detection tasks include text box detection and image tampering detection tasks; the n feature extraction sub-networks include a first extraction sub-network (i.e., convolutional layer 51) and a second extraction sub-network (i.e., convolutional layer 52), the kernel sizes of convolutional layers 51 and 52 are the same, and the kernel sizes of convolutional layers 51 and 52 are both larger than the kernel size of convolutional layer 4; the n label prediction sub-networks include a first prediction sub-network and a second prediction sub-network, the first prediction sub-network includes an image feature merging layer 1 and a detection label prediction layer 1, and the second prediction sub-network includes an image feature merging layer 2 and a detection label prediction layer 2; as shown Figure 8As shown, a schematic diagram of the second specific implementation principle of the image detection method is given, which mainly includes:

[0157] Obtain the materials to be tested and deployed;

[0158] The image to be detected is input into convolutional layer 1 of the trained image detection model;

[0159] For each target delivery material, convolutional layer 1 extracts features based on the target delivery material to obtain initial image feature information; convolutional layer 2 extracts features based on the initial image feature information to obtain intermediate image feature information c; convolutional layer 3 extracts features based on intermediate image feature information c to obtain intermediate image feature information d; convolutional layer 4 extracts features based on intermediate image feature information d to obtain third image feature information; then, convolutional layer 51 extracts features based on the third image feature information to obtain fourth image feature information 1; image feature merging layer 1 upsamples the initial image feature information, intermediate image feature information c, intermediate image feature information d, third image feature information, and fourth image feature information 1 to obtain multiple upsampled feature maps of the same size, and merges the multiple upsampled feature maps to obtain the target image related to the text box position. Feature information; the detection label prediction layer 1 predicts labels based on the target image feature information to obtain text box detection results (i.e., including prediction information representing the position of the text box); and the convolutional layer 52 extracts features based on the third image feature information to obtain fourth image feature information 2; the image feature merging layer 2 upsamples the initial image feature information, intermediate image feature information c, intermediate image feature information d, third image feature information, and fourth image feature information 2 to obtain multiple upsampled feature maps of the same size, and merges the multiple upsampled feature maps to obtain target image feature information related to the image tampering area; the detection label prediction layer 2 predicts labels based on the target image feature information to obtain image tampering detection results (i.e., including prediction information representing whether there is an image tampering area in the image, and which image tampering areas exist).

[0160] Based on the above text box detection results and image tampering detection results, the image detection results of the above-mentioned target material are generated.

[0161] In practice, after extracting the text boxes from the image, it is also necessary to identify whether the text content within the text boxes is compliant. To reduce data processing volume, if the image tampering detection results indicate that the image has been tampered with, the target material to be tested can be directly determined to be non-compliant with preset regulations (i.e., the image detection result is abnormal). Only if the image tampering detection results indicate that the image has not been tampered with, will the target material to be tested and the corresponding text box detection results be input into the text recognition network. The text recognition network extracts the text content from each text box in the target material to be tested based on the text box detection results and identifies whether the text content is compliant. If the text recognition results indicate that the text content is compliant, the image detection result of the target material to be tested is determined to be normal. In other words, only if the image is not tampered with will it continue to be input into the text recognition network to determine whether the text in the text boxes is compliant. If the image is tampered with, the image detection result of the target material to be tested is directly determined to be abnormal.

[0162] It should be noted that the above text recognition network can be part of the above image detection model, or it can be a standalone text recognition model whose image detection results are abnormal.

[0163] The image detection method in this embodiment includes an image detection model comprising a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks, the second feature extraction network comprises n feature extraction sub-networks, and the detection label prediction network comprises n label prediction sub-networks. The feature extraction sub-networks, label prediction sub-networks, and image detection tasks are corresponding one-to-one. Specifically, the first feature extraction network extracts image detail features, and the image feature information output by the first feature extraction network is input to each of the n feature extraction sub-networks to perform global feature extraction of the image region through the feature extraction sub-network corresponding to each image detection task. Then, the image feature information output by the feature extraction sub-networks is input to the corresponding label prediction sub-network for label prediction. In other words, the image detection model includes a shared network (i.e., the first feature extraction network) and branch networks (i.e., feature extraction sub-networks and label prediction sub-networks) corresponding to each image detection task. The shared network extracts image detail features from the image. The image detail features are used as input data for each branch network. Each branch network then learns differentiated image detection knowledge based on these image detail features and the target ground truth labels corresponding to its respective image detection task. Since the target ground truth labels are different for different image detection tasks, the parameter values ​​of each branch network after multiple rounds of parameter iteration are also determined by its respective detection task. This allows a single image detection model to first extract image detail features that can be shared across multiple detection tasks using a common network. Then, different branch networks within this image detection model complete n image detection tasks based on these image detail features. Thus, n image detection tasks can be achieved with a single image detection model. Therefore, not only is iterative training of n image detection models unnecessary during the model training phase, improving model training efficiency, but it is also unnecessary to use n image detection models to complete an image detection task during the model application phase, improving image detection efficiency. Furthermore, only one image detection model needs to be maintained subsequently, reducing model maintenance costs.

[0164] It should be noted that this embodiment in this application is based on the same inventive concept as the previous embodiment in this application. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned image detection model training method, and the repeated parts will not be described again.

[0165] Corresponding to the above Figures 1 to 5 The image detection model training method described herein, based on the same technical concept, also includes an image detection model training apparatus in this application. Figure 9 This is a schematic diagram of the module composition of the training device for the image detection model provided in this application embodiment. The device is used to perform... Figures 1 to 5 The training method of the image detection model is described, such as... Figure 9 As shown, the device includes:

[0166] The image sample acquisition module 902 is used to acquire an image sample dataset; the image sample dataset includes N target image samples;

[0167] The detection model training module 904 is used to perform iterative training of the image detection model to be trained based on N target image samples to obtain the trained image detection model.

[0168] The image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks. The second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, the label prediction sub-networks, and the image detection tasks correspond one-to-one. The specific implementation methods for each model training iteration are as follows:

[0169] For each target image sample: the first feature extraction network extracts features based on the target image sample to obtain first image feature information; each feature extraction sub-network extracts features based on the first image feature information to obtain second image feature information; each label prediction sub-network predicts labels based on the first image feature information and the second image feature information to obtain target predicted labels; based on the target predicted labels and the target true labels, a sub-loss set corresponding to the second feature extraction network is determined; the target true labels correspond to the image detection task corresponding to the feature extraction sub-network, and the sub-loss set includes sub-loss values ​​corresponding to n feature extraction sub-networks respectively; the parameters of the image detection model are updated based on the sub-loss values ​​corresponding to each feature extraction sub-network.

[0170] The training device for the image detection model in this embodiment includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks, the second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, label prediction sub-networks, and image detection tasks are corresponding one-to-one. Specifically, the first feature extraction network extracts detailed image features, and the image feature information output by the first feature extraction network is input to each of the n feature extraction sub-networks to perform global feature extraction of the image region through the feature extraction sub-network corresponding to each image detection task. Then, the image feature information output by the feature extraction sub-networks is input to the corresponding label prediction sub-network for label prediction. In other words, the image detection model includes a shared network (i.e., the first feature extraction network) and branch networks (i.e., feature extraction sub-networks and label prediction sub-networks) corresponding to each image detection task. The shared network extracts detailed image features. Image detail features serve as input data for each branch network. Each branch network then learns differentiated image detection knowledge based on these details and the target ground truth labels corresponding to its respective image detection task. Since the target ground truth labels differ for different image detection tasks, the parameter values ​​of each branch network after multiple rounds of parameter iteration are determined by its specific detection task. This allows a single image detection model to first extract shared image detail features across multiple detection tasks using a common network. Then, different branch networks within this model complete n image detection tasks based on these details. Thus, a single image detection model can accomplish n image detection tasks. Therefore, not only is iterative training of n image detection models unnecessary during the training phase, improving training efficiency, but it also eliminates the need for separate training of n models for each image detection task during application, further enhancing efficiency. Furthermore, only one image detection model needs to be maintained subsequently, reducing maintenance costs.

[0171] It should be noted that the embodiments of the image detection model training device in this application and the embodiments of the image detection model training method in this application are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding image detection model training method mentioned above, and the repeated parts will not be described again.

[0172] Corresponding to the above Figures 6 to 8 Based on the same technical concept, this application also provides an image detection apparatus in its embodiments, which is also described in the image detection method described. Figure 10 This is a schematic diagram of the module composition of the image detection device provided in the embodiments of this application. The device is used to perform... Figures 6 to 8 The image detection method described, such as Figure 10As shown, the device includes:

[0173] Image acquisition module 1002 is used to acquire the image to be detected;

[0174] Image detection module 1004 is used to input the image to be detected into the trained image detection model for image detection, and obtain the image detection result of the image to be detected;

[0175] The image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks. The second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, the label prediction sub-networks, and the image detection tasks correspond one-to-one. The specific implementation methods for each image detection are as follows:

[0176] For each of the images to be detected: the first feature extraction network extracts features based on the image to be detected to obtain third image feature information; each feature extraction sub-network extracts features based on the third image feature information to obtain fourth image feature information; each label prediction sub-network predicts labels based on the third image feature information and the fourth image feature information to obtain image sub-detection results; based on the image sub-detection results output by each label prediction sub-network, the image detection result of the image to be detected is obtained.

[0177] The image detection device in this embodiment includes an image detection model comprising a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks, the second feature extraction network comprises n feature extraction sub-networks, and the detection label prediction network comprises n label prediction sub-networks. The feature extraction sub-networks, label prediction sub-networks, and image detection tasks are corresponding one-to-one. Specifically, the first feature extraction network extracts image detail features, and the image feature information output by the first feature extraction network is input to each of the n feature extraction sub-networks to perform global feature extraction of the image region through the feature extraction sub-network corresponding to each image detection task. Then, the image feature information output by the feature extraction sub-networks is input to the corresponding label prediction sub-network for label prediction. In other words, the image detection model includes a shared network (i.e., the first feature extraction network) and branch networks (i.e., feature extraction sub-networks and label prediction sub-networks) corresponding to each image detection task. The shared network extracts image detail features from the image. The image detail features are used as input data for each branch network. Each branch network then learns differentiated image detection knowledge based on these image detail features and the target ground truth labels corresponding to its respective image detection task. Since the target ground truth labels are different for different image detection tasks, the parameter values ​​of each branch network after multiple rounds of parameter iteration are also determined by its respective detection task. This allows a single image detection model to first extract image detail features that can be shared across multiple detection tasks using a common network. Then, different branch networks within this image detection model complete n image detection tasks based on these image detail features. Thus, n image detection tasks can be achieved with a single image detection model. Therefore, not only is iterative training of n image detection models unnecessary during the model training phase, improving model training efficiency, but it is also unnecessary to use n image detection models to complete an image detection task during the model application phase, improving image detection efficiency. Furthermore, only one image detection model needs to be maintained subsequently, reducing model maintenance costs.

[0178] It should be noted that the embodiments of the image detection device in this application and the embodiments of the image detection method in this application are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding image detection method mentioned above, and the repeated parts will not be described again.

[0179] Furthermore, corresponding to the above Figures 1 to 8 Based on the same technical concept, this application also provides a computer device for executing the above-described image detection model training method or image detection method, such as... Figure 11 As shown.

[0180] Computer devices can vary significantly due to differences in configuration or performance. They may include one or more processors 1101 and memory 1102, with memory 1102 storing one or more application programs or data. Memory 1102 can be temporary or persistent storage. The application programs stored in memory 1102 may include one or more modules (not shown), each module including a series of computer-executable instructions for the computer device. Furthermore, processor 1101 may be configured to communicate with memory 1102 and execute the series of computer-executable instructions stored in memory 1102 on the computer device. The computer device may also include one or more power supplies 1103, one or more wired or wireless network interfaces 1104, one or more input / output interfaces 1105, one or more keyboards 1106, etc.

[0181] In one specific embodiment, the computer device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for use in the computer device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0182] Obtain an image sample dataset; the image sample dataset includes N target image samples;

[0183] Based on N target image samples, the image detection model to be trained is iteratively trained to obtain the trained image detection model;

[0184] The image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks. The second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, the label prediction sub-networks, and the image detection tasks correspond one-to-one. The specific implementation methods for each model training iteration are as follows:

[0185] For each target image sample: the first feature extraction network extracts features based on the target image sample to obtain first image feature information; each feature extraction sub-network extracts features based on the first image feature information to obtain second image feature information; each label prediction sub-network predicts labels based on the first image feature information and the second image feature information to obtain target predicted labels;

[0186] Based on the predicted target label and the true target label, a sub-loss set corresponding to the second feature extraction network is determined; the true target label corresponds to the image detection task corresponding to the feature extraction sub-network, and the sub-loss set includes n sub-loss values ​​corresponding to the feature extraction sub-network respectively.

[0187] The image detection model is updated with parameters based on the sub-loss values ​​corresponding to each feature extraction sub-network.

[0188] In another specific embodiment, the computer device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for use in the computer device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0189] Acquire the image to be detected;

[0190] The image to be detected is input into the trained image detection model to perform image detection, and the image detection result of the image to be detected is obtained.

[0191] The image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks. The second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, the label prediction sub-networks, and the image detection tasks correspond one-to-one. The specific implementation methods for each image detection are as follows:

[0192] For each of the images to be detected: the first feature extraction network extracts features based on the image to be detected to obtain third image feature information; each feature extraction sub-network extracts features based on the third image feature information to obtain fourth image feature information; each label prediction sub-network predicts labels based on the third image feature information and the fourth image feature information to obtain image sub-detection results;

[0193] Based on the image sub-detection results output by each of the label prediction sub-networks, the image detection result of the image to be detected is obtained.

[0194] The computer device in this application embodiment includes an image detection model comprising a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks, the second feature extraction network comprises n feature extraction sub-networks, and the detection label prediction network comprises n label prediction sub-networks. The feature extraction sub-networks, label prediction sub-networks, and image detection tasks are corresponding one-to-one. Specifically, the first feature extraction network extracts image detail features, and the image feature information output by the first feature extraction network is input to each of the n feature extraction sub-networks to perform global feature extraction of the image region through the feature extraction sub-network corresponding to each image detection task. Then, the image feature information output by the feature extraction sub-networks is input to the corresponding label prediction sub-network for label prediction. In other words, the image detection model includes a shared network (i.e., the first feature extraction network) and branch networks (i.e., feature extraction sub-networks and label prediction sub-networks) corresponding to each image detection task. The shared network extracts image detail features from the image. The image detail features are used as input data for each branch network. Each branch network then learns differentiated image detection knowledge based on these image detail features and the target ground truth labels corresponding to its respective image detection task. Since the target ground truth labels are different for different image detection tasks, the parameter values ​​of each branch network after multiple rounds of parameter iteration are also determined by its respective detection task. This allows a single image detection model to first extract image detail features that can be shared across multiple detection tasks using a common network. Then, different branch networks within this image detection model complete n image detection tasks based on these image detail features. Thus, n image detection tasks can be achieved with a single image detection model. Therefore, not only is iterative training of n image detection models unnecessary during the model training phase, improving model training efficiency, but it is also unnecessary to use n image detection models to complete an image detection task during the model application phase, improving image detection efficiency. Furthermore, only one image detection model needs to be maintained subsequently, reducing model maintenance costs.

[0195] It should be noted that the embodiments concerning computer devices in this application are based on the same inventive concept as the embodiments concerning the training method of image detection models or the image detection method in this application. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding image detection model training method or image detection method described above, and the repeated parts will not be described again.

[0196] Furthermore, corresponding to the above Figures 1 to 8 Based on the same technical concept, this application also provides a storage medium for storing computer-executable instructions. In one specific embodiment, the storage medium can be a USB flash drive, optical disc, hard disk, etc. When the computer-executable instructions stored in the storage medium are executed by a processor, they can achieve the following process:

[0197] Obtain an image sample dataset; the image sample dataset includes N target image samples;

[0198] Based on N target image samples, the image detection model to be trained is iteratively trained to obtain the trained image detection model;

[0199] The image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks. The second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, the label prediction sub-networks, and the image detection tasks correspond one-to-one. The specific implementation methods for each model training iteration are as follows:

[0200] For each target image sample: the first feature extraction network extracts features based on the target image sample to obtain first image feature information; each feature extraction sub-network extracts features based on the first image feature information to obtain second image feature information; each label prediction sub-network predicts labels based on the first image feature information and the second image feature information to obtain target predicted labels;

[0201] Based on the predicted target label and the true target label, a sub-loss set corresponding to the second feature extraction network is determined; the true target label corresponds to the image detection task corresponding to the feature extraction sub-network, and the sub-loss set includes n sub-loss values ​​corresponding to the feature extraction sub-network respectively.

[0202] The image detection model is updated with parameters based on the sub-loss values ​​corresponding to each feature extraction sub-network.

[0203] In another specific embodiment, the storage medium can be a USB flash drive, optical disc, hard disk, etc., and the computer-executable instructions stored on the storage medium can achieve the following process when executed by the processor:

[0204] Acquire the image to be detected;

[0205] The image to be detected is input into the trained image detection model to perform image detection, and the image detection result of the image to be detected is obtained.

[0206] The image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks. The second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, the label prediction sub-networks, and the image detection tasks correspond one-to-one. The specific implementation methods for each image detection are as follows:

[0207] For each of the images to be detected: the first feature extraction network extracts features based on the image to be detected to obtain third image feature information; each feature extraction sub-network extracts features based on the third image feature information to obtain fourth image feature information; each label prediction sub-network predicts labels based on the third image feature information and the fourth image feature information to obtain image sub-detection results;

[0208] Based on the image sub-detection results output by each of the label prediction sub-networks, the image detection result of the image to be detected is obtained.

[0209] When the computer-executable instructions stored in the storage medium in this embodiment are executed by the processor, the image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks, the second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, label prediction sub-networks, and image detection tasks correspond one-to-one. Specifically, image detail features are extracted through the first feature extraction network, and the image feature information output by the first feature extraction network is input to the n feature extraction sub-networks respectively, so that global feature extraction of the image region is performed through the feature extraction sub-network corresponding to each image detection task. Then, the image feature information output by the feature extraction sub-networks is input to the corresponding label prediction sub-network for label prediction. In other words, the image detection model includes a shared network (i.e., the first feature extraction network) and branch networks (i.e., feature extraction sub-networks and label prediction sub-networks) corresponding to each image detection task. Image detail features are extracted from the image and used as input data for each branch network. Each branch network then learns differentiated image detection knowledge based on these detail features and the target ground truth labels corresponding to its respective image detection task. Since the target ground truth labels are different for different image detection tasks, the parameter values ​​of each branch network after multiple rounds of parameter iteration are determined by its respective detection task. This allows a single image detection model to first extract image detail features that can be shared across multiple detection tasks using a common network. Then, different branch networks within this image detection model complete n image detection tasks based on these detail features. Thus, a single image detection model can achieve n image detection tasks. Therefore, not only is iterative training of n image detection models unnecessary during the model training phase, improving training efficiency, but it also eliminates the need for separate training of n image detection models for each image detection task during application, further improving efficiency. Furthermore, only one image detection model needs to be maintained subsequently, reducing model maintenance costs.

[0210] It should be noted that the embodiments concerning storage media in this application and the embodiments concerning the training method of image detection model in this application are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the corresponding image detection model training method described above, and the repeated parts will not be described again.

[0211] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0212] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0213] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0214] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0215] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0216] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0217] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0218] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0219] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the aforementioned element.

[0220] The embodiments of this application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. One or more embodiments of this application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can reside in local and remote computer storage media, including storage devices.

[0221] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0222] The above description is merely an embodiment of this document and is not intended to limit the scope of this document. Various modifications and variations can be made to this document by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this document should be included within the scope of the claims of this document.

Claims

1. A training method for an image detection model, characterized in that, The method includes: Obtain an image sample dataset; the image sample dataset includes N target image samples; each target image sample carries n target ground truth labels; Based on N target image samples, the image detection model to be trained is iteratively trained to obtain the trained image detection model; The image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks. The second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, the label prediction sub-networks, and the image detection tasks correspond one-to-one. The specific implementation methods for each model training iteration are as follows: For each target image sample: the first feature extraction network extracts features based on the target image sample to obtain first image feature information; each feature extraction sub-network extracts features based on the first image feature information to obtain second image feature information; each label prediction sub-network predicts labels based on the first image feature information and the second image feature information to obtain target predicted labels; Based on the predicted target label and the true target label, a sub-loss set corresponding to the second feature extraction network is determined; the true target label corresponds to the image detection task corresponding to the feature extraction sub-network, and the sub-loss set includes n sub-loss values ​​corresponding to the feature extraction sub-network respectively. The image detection model is updated with parameters based on the sub-loss values ​​corresponding to each of the feature extraction sub-networks; wherein each sub-loss value is used to update the parameters of the feature extraction sub-network and the label prediction sub-network corresponding to the sub-loss value; the target loss value is used to update the parameters of the first feature extraction network; the target loss value is determined based on each of the sub-loss values.

2. The method according to claim 1, characterized in that, The first feature extraction network comprises m convolutional layers connected in sequence; the kernel size of the subsequent convolutional layer is larger than the kernel size of the preceding convolutional layer. The first feature extraction network extracts features based on the target image samples to obtain first image feature information, including: The first convolutional layer extracts features based on the target image sample to obtain initial image feature information; the initial image feature information is used as the image feature information output by the previous convolutional layer of the second convolutional layer. The non-first and non-last convolutional layer extracts features based on the image feature information output by the previous convolutional layer to obtain intermediate image feature information; the intermediate image feature information is used as the image feature information output by the previous convolutional layer of the p-th convolutional layer, where 2 < p ≤ m; The last convolutional layer extracts features based on the image feature information output by the previous convolutional layer to obtain the first image feature information.

3. The method according to claim 1, characterized in that, Each of the label prediction sub-networks performs label prediction based on the first image feature information and the second image feature information to obtain the target predicted label, including: Each of the label prediction subnetworks merges the first image feature information, the preceding image feature information, and the second image feature information output by the feature extraction subnetwork to obtain the target image feature information; the preceding image feature information includes at least one of the image feature information output by the first (m-1) convolutional layers in the first feature extraction network; Each of the label prediction subnetworks performs label prediction based on the target image feature information to obtain the target predicted label; the target predicted label corresponds to the image detection task corresponding to the label prediction subnetwork.

4. The method according to claim 3, characterized in that, The target image sample includes the projected material sample; the n image detection tasks include the text box detection task and the image tampering detection task; the target ground truth label includes the text box position ground truth label and the image tampering region ground truth label; the target prediction label includes the text box position prediction label and the image tampering region prediction label. The step of determining the sub-loss set corresponding to the second feature extraction network based on the predicted target label and the true target label includes: Based on the predicted label of the text box position and the true label of the text box position, determine the first sub-loss value; Based on the predicted label of the image tampered region and the true label of the image tampered region, a second sub-loss value is determined; Based on the first sub-loss value and the second sub-loss value, the sub-loss set corresponding to the second feature extraction network is determined.

5. The method according to claim 4, characterized in that, The n feature extraction sub-networks include a first extraction sub-network and a second extraction sub-network; Each of the feature extraction sub-networks performs feature extraction based on the first image feature information to obtain second image feature information, including: The first extraction sub-network performs feature extraction based on the first image feature information to obtain text box representation information; the text box representation information is determined as the second image feature information output by the first extraction sub-network; The second extraction sub-network performs feature extraction based on the first image feature information to obtain image tampering characterization information; the image tampering characterization information is determined as the second image feature information output by the second extraction sub-network.

6. The method according to claim 4, characterized in that, The n label prediction subnetworks include a first prediction subnetwork and a second prediction subnetwork; Each of the label prediction subnetworks performs label prediction based on the target image feature information to obtain the target predicted label, including: The first prediction subnetwork performs label prediction based on the target image feature information to obtain the text box position prediction label; the text box position prediction label is determined as the target prediction label corresponding to the text box detection task. The second prediction sub-network performs label prediction based on the target image feature information to obtain the image tampering region prediction label; the image tampering region prediction label is determined as the target prediction label corresponding to the image tampering region detection task.

7. The method according to claim 4, characterized in that, The sub-loss values ​​corresponding to each feature extraction sub-network include the first sub-loss value and the second sub-loss value; The step of updating the parameters of the image detection model based on the sub-loss values ​​corresponding to each of the feature extraction sub-networks includes: Based on the first sub-loss value, the parameters of the feature extraction sub-network and the label prediction sub-network corresponding to the text box detection task are updated. Based on the second sub-loss value, the parameters of the feature extraction subnetwork and the label prediction subnetwork corresponding to the image tampering detection task are updated. Based on the first sub-loss value and the second sub-loss value, a target loss value is determined; and the parameters of the first feature extraction network are updated based on the target loss value.

8. The method according to any one of claims 1 to 6, characterized in that, The acquisition of the image sample dataset includes: Obtain the initial sample dataset; the initial sample dataset includes M initial image samples; Data augmentation processing is performed on the M initial image samples to obtain a data augmentation sample set; the data augmentation sample set includes P data augmentation image samples, where P ≥ (NM). Based on the P data-enhanced image samples and the M initial image samples, an image sample dataset containing N target image samples is generated.

9. The method according to claim 8, characterized in that, The n image detection tasks include text box detection tasks and image tampering detection tasks; The data augmentation process based on the M initial image samples yields a data augmentation sample set, including: For each initial image sample, at least one target text box is randomly selected from the initial image sample based on the text box marking information of the initial image sample; The target text box is subjected to a preset processing to obtain a data-enhanced image sample; the preset processing includes at least one of moving the text box, deleting the text box, and copying the text box; the target real label of the data-enhanced image sample includes a real label of the text box position and a real label of the image tampering area; the real label of the image tampering area includes the original real label of the image tampering area and a new real label of the image tampering area generated based on the preset processing of the target text box. Based on the data-enhanced image samples described above, a set of data-enhanced samples is determined.

10. An image detection method, characterized in that, The method includes: Acquire the image to be detected; The image to be detected is input into a trained image detection model for image detection to obtain the image detection result of the image to be detected; the image detection model is trained by the method described in any one of claims 1-9; The image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks. The second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, the label prediction sub-networks, and the image detection tasks correspond one-to-one. The specific implementation methods for each image detection are as follows: For each of the images to be detected: the first feature extraction network extracts features based on the image to be detected to obtain third image feature information; each feature extraction sub-network extracts features based on the third image feature information to obtain fourth image feature information; each label prediction sub-network predicts labels based on the third image feature information and the fourth image feature information to obtain image sub-detection results; Based on the image sub-detection results output by each of the label prediction sub-networks, the image detection result of the image to be detected is obtained.

11. The method according to claim 10, characterized in that, The first feature extraction network comprises m convolutional layers connected in sequence; the kernel size of the subsequent convolutional layer is larger than the kernel size of the preceding convolutional layer. The first feature extraction network extracts features based on the image to be detected to obtain third image feature information, including: The first convolutional layer extracts features based on the image to be detected to obtain initial image feature information; the initial image feature information is used as the image feature information output by the convolutional layer preceding the second convolutional layer. The non-first and non-last convolutional layer extracts features based on the image feature information output by the previous convolutional layer to obtain intermediate image feature information; the intermediate image feature information is used as the image feature information output by the previous convolutional layer of the p-th convolutional layer, where 2 < p ≤ m; The last convolutional layer extracts features based on the image feature information output by the previous convolutional layer to obtain the third image feature information.

12. The method according to claim 10, characterized in that, Each of the label prediction sub-networks performs label prediction based on the third image feature information and the fourth image feature information to obtain image sub-detection results, including: Each of the label prediction subnetworks merges the third image feature information, the preceding image feature information, and the fourth image feature information output by the feature extraction subnetwork to obtain the target image feature information; the preceding image feature information includes at least one of the image feature information output by the first (m-1) convolutional layers in the first feature extraction network; Each of the label prediction subnetworks performs label prediction based on the target image feature information to obtain image sub-detection results; the image sub-detection results correspond to the image detection task corresponding to the label prediction subnetwork.

13. The method according to claim 12, characterized in that, The image to be detected includes the projection material to be detected; the n image detection tasks include text box detection tasks and image tampering detection tasks; the n feature extraction subnetworks include a first extraction subnetwork and a second extraction subnetwork; the n label prediction subnetworks include a first prediction subnetwork and a second prediction subnetwork.

14. A training device for an image detection model, characterized in that, The device includes: An image sample acquisition module is used to acquire an image sample dataset; the image sample dataset includes N target image samples; each target image sample carries n target ground truth labels; The detection model training module is used to iteratively train the image detection model to be trained based on N target image samples to obtain the trained image detection model. The image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks. The second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, the label prediction sub-networks, and the image detection tasks correspond one-to-one. The specific implementation methods for each model training iteration are as follows: For each target image sample: the first feature extraction network extracts features based on the target image sample to obtain first image feature information; each feature extraction sub-network extracts features based on the first image feature information to obtain second image feature information; each label prediction sub-network predicts labels based on the first image feature information and the second image feature information to obtain target predicted labels; based on the target predicted labels and the target true labels, a sub-loss set corresponding to the second feature extraction network is determined; the target true labels correspond to the image detection task corresponding to the feature extraction sub-network, and the sub-loss set includes n sub-loss values ​​corresponding to each of the feature extraction sub-networks; the parameters of the image detection model are updated based on the sub-loss values ​​corresponding to each of the feature extraction sub-networks; wherein, each sub-loss value is used to update the parameters of the feature extraction sub-network and the label prediction sub-network corresponding to the sub-loss value; the target loss value is used to update the parameters of the first feature extraction network; the target loss value is determined based on each of the sub-loss values.

15. An image detection device, characterized in that, The device includes: The image acquisition module is used to acquire the image to be detected; An image detection module is used to input the image to be detected into a trained image detection model to perform image detection and obtain the image detection result of the image to be detected; the image detection model is trained by the method described in any one of claims 1-9; The image detection model includes a first feature extraction network, a second feature extraction network, and a detection label prediction network. The first feature extraction network is a shared network for n image detection tasks. The second feature extraction network includes n feature extraction sub-networks, and the detection label prediction network includes n label prediction sub-networks. The feature extraction sub-networks, the label prediction sub-networks, and the image detection tasks correspond one-to-one. The specific implementation methods for each image detection are as follows: For each of the images to be detected: the first feature extraction network extracts features based on the image to be detected to obtain third image feature information; each feature extraction sub-network extracts features based on the third image feature information to obtain fourth image feature information; each label prediction sub-network predicts labels based on the third image feature information and the fourth image feature information to obtain image sub-detection results; based on the image sub-detection results output by each label prediction sub-network, the image detection result of the image to be detected is obtained.

16. A computer device, characterized in that, The device includes: Processor; and A memory configured to store computer-executable instructions configured to be executed by the processor, the executable instructions including steps for performing the method as claimed in any one of claims 1 to 9 or any one of claims 10 to 13.

17. A storage medium, characterized in that, The storage medium is used to store computer-executable instructions that cause the computer to perform the method as described in any one of claims 1 to 9 or any one of claims 10 to 13.

Images