Image target detection training method and system
By generating blurred images in face detection technology and combining them with iterative training of the U-Net network and object detector, the information security risks and privacy leakage issues in the face detection training process are solved, and accurate detection of targets in blurred images is achieved.
Patent Information
- Application Number
- CN202310274368.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-20
- Publication Date
- 2025-12-16
- Estimated Expiration
- 2043-03-20
AI Technical Summary
Existing AI-based face detection technologies require a large amount of real-world facial image data during training, leading to information security risks and privacy leaks.
Blurred images are generated by extracting the boundaries of objects in images. The clear and blurred images are then input into a deep learning neural network for training. The network is combined with a U-Net network and an object detector, and iterative training is performed using supervised loss and object reconstruction loss to update the network parameters in order to detect objects in the blurred images.
While ensuring target detection performance, it mitigates information security risks and privacy leaks, and achieves accurate target detection in blurred images.
Smart Images

Figure CN116543428B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of digital image processing, in particular to an image target detection training method and system. BACKGROUND
[0002] With the development of artificial intelligence technology, the target detection and recognition technology of images has also been strongly promoted. Especially in the field of face detection, relying on the existing artificial intelligence model, accurate and stable detection effect can be achieved, and the face detection technology based on artificial intelligence has been widely applied to social production and life.
[0003] The face detection technology based on artificial intelligence needs to go through a large number of real-world face image data for deep learning to complete the model training task. As sensitive information, face data can cause information security risks and technical problems of privacy leakage. SUMMARY
[0004] (I) Invention purpose
[0005] The purpose of the present application is to provide an image target detection training method and system based on the traditional image processing technology-based face privacy protection method, which trains a deep learning neural network for fuzzy target detection, and on the basis of ensuring the performance of target detection, alleviates the technical problems of information security risks and privacy leakage caused by the disclosure of target information.
[0006] (II) Technical solutions
[0007] According to some embodiments, the first aspect of the present application provides an image target detection training method, comprising: extracting the boundary of the target in the first image to generate a second image; performing smoothing or blurring operation on the target in the first image to generate a third image; inputting the second image and the third image into a deep learning neural network to train an output fourth image; inputting the first image and the fourth image into a first target detector and a second target detector respectively to obtain a first target and a second target, the first target detector and the second target detector being used for detecting the boundary of the target in the first image or the fourth image; calculating a training loss according to the first target and the second target, the first image and the fourth image, if the training loss is lower than a set threshold, stopping the training, otherwise updating the weight parameters of the deep learning neural network and the parameters of the second target detector and continuing the training.
[0008] In one embodiment, the step of generating a third image by performing smoothing or blurring operation on the target in the first image comprises: generating a fifth image by performing smoothing or blurring operation on the first image; generating a binary mask matrix from the second image; and obtaining the third image by weighted sum of the first image, the binary mask matrix and the fifth image.
[0009] In one embodiment, the first target detector and the second target detector have the same parameters before training of the deep learning neural network.
[0010] In one embodiment, the deep learning neural network is a U-Net, and a up-sampling module is added in the U-Net before training of the U-Net.
[0011] In one embodiment, the training loss comprises a supervision loss with the first target detector and the second target detector as discriminators, and a target reconstruction loss with the deep learning neural network as a generator.
[0012] In one embodiment, the supervision loss comprises a first loss obtained by subtracting the first target from the second target.
[0013] In one embodiment, the supervision loss further comprises a second loss obtained by subtracting the first target from a third target, the third target being obtained by the first target detector from a fourth image.
[0014] In one embodiment, the supervision loss further comprises a third loss obtained by subtracting the first target from a fourth target, the fourth target being obtained by a third target detector from the first image, the third target detector being obtained by model fine-tuning of the second target detector.
[0015] In one embodiment, the target reconstruction loss comprises a truncated mean square error between the third image and the fourth image.
[0016] In one embodiment, in the training loss, the weight ratio of the target reconstruction loss to the supervision loss is 1:10.
[0017] According to some embodiments, a second aspect of the present application provides an image target detection training system, comprising: a target extraction module configured to extract a boundary of a target in a first image to generate a second image; a target blurring module configured to perform a smoothing or blurring operation on the target in the first image to generate a third image; a training module configured to input the second image and the third image into a deep learning neural network to train and output a fourth image; a detection module configured to input the first image and the fourth image into a first target detector and a second target detector respectively to obtain a first target and a second target, the first target detector and the second target detector being configured to detect the boundary of the target in the first image or the fourth image; and a loss calculation module configured to calculate a training loss according to the first target and the second target, the first image and the fourth image, and stop training if the training loss is lower than a set threshold, or continue training after updating weight parameters of the deep learning neural network and parameters of the second target detector if the training loss is not lower than the set threshold.
[0018] According to some embodiments, the third aspect of the present application provides an electronic device, comprising: a memory having a computer program stored thereon; and a processor configured to execute the computer program in the memory to implement the steps of the method of the first aspect of the present application.
[0019] (III) Beneficial Effects
[0020] The above technical solution of the present application has the following beneficial technical effects: the training loss is calculated through the first target and the second target, the first image and the fourth image, and the deep learning neural network and the second target detector are iteratively trained; the image with the blurred target output from the deep learning neural network can still accurately detect the target on the second target detector, thereby alleviating the technical problems of information security risks and privacy leakage caused by the disclosure of target information.
[0021] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS
[0022] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are one embodiment of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.
[0023] Figure 1 is a flow diagram of an image target detection training method according to an embodiment of the present application;
[0024] Figure 2 is a data flow diagram of an image target detection training method according to an embodiment of the present application;
[0025] Figure 3 is another data flow diagram of an image target detection training method according to an embodiment of the present application;
[0026] Figure 4 is a structural diagram of an image target detection training system according to an embodiment of the present application;
[0027] Figure 5 is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0028] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the embodiments and the accompanying drawings. It should be understood that the description is only exemplary but not intended to limit the scope of the present application. Moreover, in the following description, the description of the well-known structures and techniques is omitted to avoid unnecessary obscurity of the concept of the present application.
[0029] In the description of the present application, it should be noted that the terms "first", "second", "third" are only for the purpose of description and should not be understood as indicating or implying relative importance.
[0030] Figure 1 A flowchart of an image target detection training method according to an embodiment of the present application.
[0031] As shown in Figure 1 the method embodiment includes the following five steps.
[0032] Step S1: Extract the boundary of the target in the image. Specifically, extract the boundary of the target in the first image to generate the second image.
[0033] It should be noted that the target in the first image can be the face of a person or an animal, or content with sensitive information that is not suitable for direct disclosure. The boundary of the target in the first image can be extracted using edge detection or image segmentation algorithm in digital image, and the target in the first image can be extracted from the background of the first image. The boundary of the target in the first image can also be obtained from the public data set, for example, in the field of face detection, the first image and its target boundary can be obtained from the Public Figures Face Database of Columbia University.
[0034] Step S2: Perform smoothing or blurring operation on the target in the image. Specifically, perform smoothing or blurring operation on the target in the first image to generate the third image.
[0035] In one embodiment, after performing smoothing or blurring operation on the target in the first image to generate the third image, the target boundary in the first image can be read and smoothing or blurring operation can be performed in the closed area surrounded by the target boundary. After smoothing or blurring, the pixels in the target boundary range will become blurred in vision. It should be noted that the smoothing or blurring operation of the image can be achieved by using a low-pass filter. For example, Gaussian blur can be used, which is a kind of image blurring filter. The algorithm uses normal distribution to calculate the transformation of each pixel in the image, so as to achieve the technical effect of smoothing or blurring the closed area within the target boundary.
[0036] In one embodiment, the step of generating the third image after smoothing or blurring the target in the first image comprises: generating a fifth image after smoothing or blurring the first image; generating a binary mask matrix from the second image; and obtaining the third image by weighted sum of the first image, the binary mask matrix and the fifth image.
[0037] It should be noted that the binary mask matrix can be a binary image composed of 0 and 1. For example, in the second image, the pixels within the target boundary are set to 1, and the pixels outside the target boundary are set to 0. Then, the first image and the fifth image are weighted and summed to obtain the third image.
[0038] For example, when performing Gaussian blur on the target in the first image, the height and width of the Gaussian kernel can be set to 1 / 2 of the height and width of the target boundary in the first image, so that the target is blurred and difficult to clearly identify in human vision.
[0039] Step S3: sending the images processed in S1 and S2 into a deep learning neural network for training. Specifically, the second image and the third image are input into the deep learning neural network for training to output a fourth image.
[0040] It should be noted that the second image contains the target boundary of the first image, and the pixels within the target boundary of the first image in the third image have been smoothed or blurred. Both of them are sent into the deep learning neural network to extract the above-mentioned features, and in the network learning process, the above-mentioned features are learned.
[0041] Step S4: detecting the target by using a target detector. Specifically, the first image and the fourth image are sent into a first target detector and a second target detector respectively to obtain a first target and a second target. The first target detector and the second target detector are used to detect the boundary of the target in the first image or the fourth image.
[0042] In one embodiment, the parameters of the first target detector and the second target detector are the same before the deep learning neural network is trained. The first target detector and the second target detector are used to detect the target in the image, which can be an independent module or a part of the deep learning neural network. For example, in the field of face detection, the first target detector and the second target detector can be RetinaFace model.
[0043] It should be noted that the parameters of the first target detector and the second target detector are the same before the network is trained, that is, both have the same detection performance. The first target detector is used to detect the first target in the first image, and the second target detector is used to detect the second target in the fourth image, that is, the first target detector is used to detect the image in which the background and the target are clear, and the second target detector is used to detect the image in which the background is clear and the target is blurred.
[0044] Step S5: Calculate the training loss and iterate. Specifically, the training loss is calculated according to the first target and the second target, the first image and the fourth image, if the training loss is lower than the set threshold, the training is stopped, otherwise the weight parameters of the deep learning neural network and the second target detector parameters are updated and the training is continued.
[0045] It needs to be noted that, in the network training process, if the training loss is lower than the set threshold, the training expectation of the network is reached, and the network stops training. If the training loss is higher than the set threshold, the weight parameters of the deep learning neural network and the second target detector parameters are updated through the back propagation mechanism of the deep learning neural network, and the training is continued. The parameters of the second target detector are updated according to the training parameters of the deep learning neural network, so that the second target detector can detect images with clear background and blurred target.
[0046] Figure 2 A data flow diagram of an image target detection training method according to an embodiment of the present application.
[0047] Reference Figure 2 The second image and the third image are sent into the deep learning neural network for training. Exemplarily, the third image can be a three-channel RGB image, and the fourth image can be a single-channel grayscale or binary image. The first image and the fourth image are merged in the channel and then sent into the deep learning neural network for training. It needs to be noted that the function of the deep learning neural network is to extract and fuse the features of the second image and the third image, so that the output fourth image fuses the features of the two images.
[0048] In one embodiment, the deep learning neural network is a U-Net. It needs to be noted that the U-Net is adopted in the field of semantic segmentation and image segmentation, which can generate the fourth image with fused features according to the features of the second image and the third image.
[0049] In one embodiment, an up-sampling module is added to the U-Net before training. It needs to be noted that the U-Net is a deep learning network structure containing several down-sampling modules and up-sampling modules. The down-sampling module is used to gradually reveal the environmental information, and the process of the up-sampling module is to restore the detail information by combining the information of each layer of the down-sampling module and the input information of the up-sampling module, and gradually restore the image accuracy. Adding an up-sampling module to the original up-sampling module in the U-Net can restore images with higher resolution. For example, a Pixel Shuffle layer can be added to the U-Net. The function of the Pixel Shuffle layer is to convert a low-resolution input image into a high-resolution image through sub-pixel interpolation.
[0050] In one embodiment, the training loss comprises a supervised loss with the first target detector and the second target detector as discriminators, and a target reconstruction loss with the deep learning neural network as a generator.
[0051] It should be noted that in the field of deep learning technology, the task performed by the discriminator is to extract features from the input high-dimensional data to obtain low-dimensional discriminative information for classification and detection, etc. The generator is the opposite, which generates high-dimensional data from low-dimensional features, and needs to learn the overall distribution characteristics of the data, such as generating an image with a large dimension from low-dimensional random noise. In this embodiment, the first target detector and the second target detector are used to detect the target in the image, thereby serving as discriminators; the deep learning neural network fuses the features of the second image and the third image to generate the fourth image, thereby serving as a generator.
[0052] In this embodiment, the training of the generator is guided by the discriminator, and the iterative training between the deep learning neural network and the second target detector is constantly antagonized, so that the target blurred image output from the deep learning neural network can still accurately detect the target on the second target detector.
[0053] In one embodiment, the supervised loss comprises a first loss obtained by difference operation of the first target and the second target.
[0054] Exemplarily, assuming that the first image is x1, the fourth image is x4, the first target detector is mapping f, the deep learning neural network is mapping g, and the second target detector is mapping fg, the operation formula of the first loss is |f(x1)-fg(x4)|, where f(x1) represents the output result of x1 after mapping f, i.e. the first target, and fg(x4) represents the output result of x4 after mapping fg, i.e. the second target; by using the first loss to train the deep learning neural network, the detection performance of the second target detector on the fourth image and the detection performance of the first target detector on the first image are minimized, and the second target detector or the first target detector only needs the target blurred fourth image to achieve the expected training effect.
[0055] In one embodiment, the supervised loss further comprises a second loss obtained by difference operation of the first target and a third target, and the third target is obtained by the first target detector from the fourth image.
[0056] Exemplarily, the third target can be represented as f(x4), and the operation formula of the second loss is |f(x1)-f(x4)|; by using the second loss to train the deep learning neural network, the influence of the target clear first image and the target blurred fourth image on the first target detector is minimized.
[0057] In an embodiment, the supervised loss further comprises a third loss obtained by differencing the first target and a fourth target, the fourth target being obtained by the first image through a third target detector, the third target detector being obtained by model fine-tuning of the second target detector.
[0058] It should be noted that in the field of deep learning technology, model fine-tuning refers to training the model using a smaller learning rate based on the original model. Illustratively, the third target detector is a mapping f g ’ The operation formula of the third loss is |f(x1)-f g ’ (x1)|. By using the third loss to train the deep learning neural network, the detection performance of the third target detector on the first image is realized to be minimally different from the detection performance of the first target detector on the first image.
[0059] In an embodiment, the target reconstruction loss comprises a truncated mean square error of the third image and the fourth image. Through the calculation of the target loss, the deep learning neural network can be optimized to an identity mapping under the constraint of the supervised loss, and it is also controlled that the deep learning neural network ensures that there is a feasible solution for the above-mentioned supervised loss on the basis of preserving part of the target features.
[0060] In an embodiment, in the training loss, the weight ratio of the target reconstruction loss to the supervised loss is 1:10. When the weighted sum of the two losses reaches a predetermined threshold, the training is stopped.
[0061] In an embodiment, the deep learning neural network uses the back propagation mechanism to perform gradient backpropagation and update the weight parameters of the deep learning neural network. In order to prevent the optimization process from collapsing due to too large gradient, the gradient modulus is truncated, for example, the threshold is set to 500.
[0062] Figure 3 Another data flow diagram of an image target detection training method according to an embodiment of the present application.
[0063] Reference Figure 3 In this embodiment, the first image is an image of a human face, and in order to protect the privacy of humans, the face image is subjected to Gaussian blur processing, and the blurred face image in the figure is the third image. In this embodiment, the boundary of the target in the first image is obtained from a public data set to generate the second image, and the second image can be represented in the form of a binary mask of the face.
[0064] After merging the blurred face image and the binary mask of the face in the channel, the image is sent into the U-Net network for training to obtain the fourth image, i.e. Figure 3The network blurred face image.
[0065] Figure 3 The face detection technology in the method comprises a first target detector and a second target detector.
[0066] The training loss is calculated according to the first target and the second target, the first image and the fourth image.
[0067] Figure 4 FIG. 1 is a structural schematic diagram of an image target detection training system according to an embodiment of the present application.
[0068] Reference Figure 4 The image target detection training system according to an embodiment of the present application comprises the following five modules.
[0069] The target extraction module 201 is configured to extract the boundary of the target in the first image to generate the second image.
[0070] The target blurring module 202 is configured to perform smoothing or blurring operation on the target in the first image to generate the third image.
[0071] The training module 203 is configured to input the second image and the third image into the deep learning neural network to generate the fourth image.
[0072] The detection module 204 is configured to input the first image and the fourth image into the first target detector and the second target detector respectively to obtain the first target and the second target.
[0073] The loss calculation module 205 is configured to calculate the training loss according to the first target and the second target, the first image and the fourth image.
[0074] Figure 5This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention.
[0075] refer to Figure 5 The present invention also provides an electronic device. For example... Figure 4 As shown, the electronic device 300 of this embodiment includes: a processor 301, a memory 302, and a program stored in the memory 302 and executable on the processor 301. When the processor 301 executes the program, it implements the steps in the various image object detection training method embodiments described above, for example... Figure 1 Steps S1 to S5 are shown. Alternatively, when processor 301 executes the program, it implements the functions of each module in the above system embodiment, for example... Figure 2 The module in it is used to implement an image target detection training system.
[0076] Exemplarily, the program can be divided into one or more modules, which are stored in the memory 302 and executed by the processor 301 to complete the present invention. The one or more modules can be a series of program instruction segments capable of performing specific functions, which describe the execution process of the program in an electronic device. For example, the program can be divided into a target extraction module 201, a target blurring module 202, a training module 203, a detection module 204, and a loss calculation module 205.
[0077] The specific functions of each module are as follows: Target extraction module 201, which is used to extract the boundary of the target in the first image to generate the second image; Target blurring module 202, which is used to smooth or blur the target in the first image to generate the third image; Training module 203, which is used to take the second image and the third image as input, feed them into the deep learning neural network for training, and output the fourth image; Detection module 204, which is used to feed the first image and the fourth image into the first target detector and the second target detector respectively to obtain the first target and the second target. The first target detector and the second target detector are used to detect the boundary of the target in the first image or the fourth image; Loss calculation module 205, which is used to calculate the training loss based on the first target and the second target, the first image and the fourth image. If the training loss is lower than the set threshold, the training is stopped. Otherwise, the weight parameters of the deep learning neural network and the parameters of the second target detector are updated and the training continues.
[0078] The electronic device can be a single-chip system, a desktop computer, a notebook, a palm computer, a cloud server, and the like. The electronic device can include, but is not limited to, a processor, a memory. Those skilled in the art can understand that the schematic diagram is only an example and does not constitute a limitation on the electronic device, and can include more or fewer components than the diagram, or combine certain components, or different components, for example, the electronic device can also include an input / output device, and the input / output device and the processor and the memory can be connected through a bus.
[0079] The processor can be a microcontroller unit (MCU), a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor and the like. The processor is the control center of the electronic device, and connects various parts of the entire electronic device through various interfaces and lines.
[0080] The memory can be used to store programs and / or modules, and the processor realizes various functions of the image target detection training method and system by running or executing programs and / or modules stored in the memory, and calling data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function (such as a sound playing function, an image playing function, etc.), and the like; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phonebook, etc.), and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0081] It is to be understood that the above specific embodiments of the present application are merely illustrative or explanatory and are not restrictive of the present application. Thus, any modification, equivalent substitution, improvement, etc. made without departing from the spirit and scope of the present application should be included in the scope of protection of the present application. In addition, the appended claims of the present application are intended to encompass all changes and modifications falling within the scope and boundary of the appended claims, or equivalents of such scope and boundary.
[0082] Those skilled in the art will appreciate that embodiments of the present application can be devised for a method, a system, or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code.
[0083] The present application is described in reference to the flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present application. It is to be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for performing the function specified by the flowchart illustrations and / or block diagrams block or blocks.
[0084] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for performing the function specified by the flowchart illustrations and / or block diagrams block or blocks.
[0085] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 means for performing the function specified by the flowchart illustrations and / or block diagrams block or blocks.
[0086] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment methods can be completed by programs instructing relevant hardware, and the programs can be stored in a computer readable storage medium. When the programs are executed, the processes of the above-mentioned embodiments of the methods are included. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM) and the like.
[0087] The steps in the embodiment methods of the present application can be adjusted, combined and deleted in sequence according to actual needs. The modules in the embodiment system device of the present application can be combined, divided and deleted according to actual needs.
Claims
1. An image target detection training method, characterized in that, The method comprises the following steps: extracting the boundary of the target in the first image to generate a second image; performing smoothing or blurring operation on the target in the first image to generate a third image; inputting the second image and the third image into a deep learning neural network to train and output a fourth image; inputting the first image and the fourth image into a first target detector and a second target detector respectively to obtain a first target and a second target, wherein the first target detector and the second target detector are used to detect the boundary of the target in the first image and the fourth image; calculating a training loss according to the first target, the second target, the first image and the fourth image, and stopping the training if the training loss is lower than a set threshold, otherwise updating the weight parameters of the deep learning neural network and the parameters of the second target detector and continuing the training; wherein the training loss comprises: a supervision loss with the first target detector and the second target detector as discriminators, and a target reconstruction loss with the deep learning neural network as a generator; wherein the supervision loss comprises: a first loss obtained by subtracting the second target from the first target; a second loss obtained by subtracting a third target from the first target, wherein the third target is obtained by the first target detector from the fourth image; a third loss obtained by subtracting a fourth target from the first target, wherein the fourth target is obtained by a third target detector from the first image, and the third target detector is obtained by model fine-tuning of the second target detector; wherein the target reconstruction loss comprises: a truncated mean square error of the third image and the fourth image.
2. The method of claim 1, wherein, The step of performing smoothing or blurring operation on the target in the first image to generate a third image comprises: performing smoothing or blurring operation on the first image to generate a fifth image; generating a binary mask matrix from the second image; weighting and summing the first image, the binary mask matrix and the fifth image to obtain the third image.
3. The method of claim 1, wherein, The parameters of the first target detector and the second target detector are the same before the training of the deep learning neural network.
4. The method of claim 1, wherein, The deep learning neural network is a U-Net, and an up-sampling module is added in the U-Net before the training of the U-Net.
5. The method of claim 1, wherein, In the training loss, the weight ratio of the target reconstruction loss to the supervision loss is 1:
10.
6. An image object detection training system, characterized by, The method comprises the following steps: a target extraction module for extracting the boundary of the target in the first image to generate a second image; a target blurring module for performing smoothing or blurring operation on the target in the first image to generate a third image; a training module for inputting the second image and the third image into a deep learning neural network to train and output a fourth image; a detection module for inputting the first image and the fourth image into a first target detector and a second target detector respectively to obtain a first target and a second target, wherein the first target detector and the second target detector are used to detect the boundary of the target in the first image and the fourth image; a loss calculation module configured to calculate a training loss according to the first target and the second target, the first image and the fourth image, stop the training if the training loss is lower than a set threshold, and otherwise continue the training after updating the weight parameters of the deep learning neural network and the second target detector parameters; wherein the training loss comprises: a supervision loss with the first target detector and the second target detector as discriminators, and a target reconstruction loss with the deep learning neural network as a generator; wherein the supervision loss comprises: a first loss obtained by difference operation between the first target and the second target; a second loss obtained by difference operation between the first target and a third target, the third target being obtained by the first target detector from the fourth image; a third loss obtained by difference operation between the first target and a fourth target, the fourth target being obtained by the third target detector from the first image, the third target detector being obtained after model fine-tuning of the second target detector; wherein the target reconstruction loss comprises: a truncated mean square error between the third image and the fourth image.
7. An electronic device, comprising: comprising: a memory having a computer program stored thereon; a processor configured to execute the computer program in the memory to implement the steps of the method of any one of claims 1-5.
Citation Information
Patent Citations
Image processing model training method and device, image processing method and model
CN112258436A
Image processing method and system, mobile terminal and storage medium
CN112669228A