Training method and device of image decomposition model and image decomposition method

Abstract

The present disclosure provides a kind of training method, device and image decomposition method of image decomposition model, which can be applied to image processing technical field.The training method of the image decomposition model includes: obtaining training set, inputting first training image into first adversarial neural network, inputting second training image into second adversarial neural network, determining first loss function value;Third training image is input into first adversarial neural network and second adversarial neural network respectively, and second loss function value is determined;For the comparison result of the obtained fusion image and third training image, third loss function value is determined, and the fusion image is generated according to the fusion of the generated image of first adversarial neural network and second adversarial neural network;According to first loss function value, second loss function value and third loss function value, the parameters of image decomposition model are adjusted until the training termination condition is met, and first loss function, second loss function and third loss function satisfy preset weight relationship.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, and more particularly to a training method and training device of an image decomposition model, an image decomposition method, an electronic device, and a readable storage medium. BACKGROUND

[0002] With the development of global economy, goods circulation and the like become more and more, and it is necessary to check the circulating goods at customs, land port and the like to prevent illegal goods or products from entering the controlled area. In the customs and land port, the goods circulation is large, and the goods are various, and it is necessary to check the transport vehicle and goods by a special detection device, for example, an X-ray scanning system, to improve the efficiency of security check.

[0003] At present, the common security check method for trucks in the world is completed based on an X-ray scanning system, which first forms an X-ray scanning image by scanning the vehicle to be measured by an X-ray scanning device, and then obtains a conclusion whether there is a suspicious object in the vehicle by manual image analysis or machine intelligent recognition.

[0004] In the traditional manual image analysis, due to the complex structure of the vehicle and the uneven stacking of the goods, it is difficult for the manual judgment to determine the goods in the image, and the efficiency is low, which is difficult to meet the needs of security check of a large number of customs vehicles. Therefore, in recent years, the machine intelligent recognition system has developed rapidly and gradually replaced a part of manual work. The machine intelligent recognition has the advantages of fast speed and high efficiency, but due to the limitation of the current intelligent image recognition algorithm, the recognition accuracy needs to be improved in the case of complex image. For example, the penetration characteristics of X-ray cause the serious overlap of the information of the goods and the information of the vehicle structure in the image, which is shown as the mutual overlap of the goods and the vehicle in the image, the mutual interference, and the difficulty in distinguishing the characteristics of the goods, so that the machine cannot accurately recognize the goods, greatly reduces the accuracy of machine recognition, cannot realize accurate judgment, increases the risk of missed detection, and reduces the efficiency of security check.

[0005] In addition, in the process of machine intelligence recognition, training is required, and a large amount of data with real labels is required in the training process. It is difficult to collect actual X-ray images and vehicle images, and the workload is huge and time-consuming. It is difficult to collect enough true value data, which makes it difficult to obtain enough real labels for supervised learning. In addition, due to the superposition characteristics of the X-ray image, it is difficult for the annotator to distinguish the vehicle and the cargo mixed together in the same image. In addition, the vehicle image is too complex, and the intra-class difference of the same type of vehicle is large. This complexity includes two aspects. On the one hand, there are many details differences caused by the year, model, high and low configuration, etc. On the other hand, even if the vehicles are completely the same, the state of the dynamic factors such as tires, engines, cylinders, rearview mirrors, etc. that cannot be controlled will be different in the scanning process, which will cause the image difference. The above limits the accuracy of machine intelligence recognition. SUMMARY

[0006] In view of the above problems, the present disclosure provides a training method and device of an image decomposition model, and an image decomposition method, an electronic device and a readable storage medium. The ability to decompose complex images can be effectively improved, so as to obtain more accurate images of the goods, and further improve the accuracy of machine judgment.

[0007] According to a first aspect of the present disclosure, a training method of an image decomposition model is provided, the image decomposition model comprising a first generative adversarial network and a second generative adversarial network, the first generative adversarial network comprising a first generative network and a first discriminative network, the second generative adversarial network comprising a second generative network and a second discriminative network. The training method of the image decomposition model comprises but is not limited to: obtaining a training set, the training set comprising a first training image, a second training image, and a third training image, the third training image being generated by fusing the first training image and the second training image; inputting the first training image into the first generative adversarial network and inputting the second training image into the second generative adversarial network to determine a first loss function value of the first generative adversarial network and a second loss function value of the second generative adversarial network, respectively; inputting the third training image into the first generative adversarial network and the second generative adversarial network to determine a second loss function value of the first generative adversarial network and a second loss function value of the second generative adversarial network, respectively; determining a third loss function value based on a comparison result of a fusion image and the third training image, the fusion image being generated by fusing a generated image of the first generative adversarial network and a generated image of the second generative adversarial network; and adjusting parameters of the image decomposition model in a backpropagation manner based on at least one of the first loss function value, the second loss function value, and the third loss function value until a preset training termination condition is met, wherein the first loss function value is calculated based on a first loss function, the second loss function value is calculated based on a second loss function, and the third loss function value is calculated based on a third loss function, and the first loss function, the second loss function, and the third loss function satisfy a preset weight relationship.

[0008] In some exemplary embodiments of the present disclosure, inputting the first training image into the first generative adversarial network to determine the first loss function value of the first generative adversarial network comprises: inputting the first training image into the first generative network of the first generative adversarial network to obtain a first generated image; inputting the first training image and the first generated image into the first discriminative network of the first generative adversarial network to obtain a first image recognition result; and determining the first loss function value of the first generative adversarial network based on the first image recognition result.

[0009] In some example embodiments of the present disclosure, the inputting the second training image into the second generative adversarial neural network comprises: inputting the second training image into a second generative network of the second generative adversarial neural network to obtain a second generated image; inputting the second training image and the second generated image into a second discriminative network of the second generative adversarial neural network to obtain a second image recognition result; and determining the first loss function value of the second generative adversarial neural network according to the second image recognition result.

[0010] In some example embodiments of the present disclosure, the inputting the third training image into the first generative adversarial neural network and the second generative adversarial neural network respectively, and determining the second loss function value of the first generative adversarial neural network and the second generative adversarial neural network respectively comprises: inputting the third training image into a first generative network of the first generative adversarial neural network to obtain a third generated image; inputting the third generated image and a first training image associated with the third training image into a first discriminative network of the first generative adversarial neural network to obtain a third image recognition result; determining the second loss function value of the first generative adversarial neural network according to the third image recognition result; inputting the third training image into a second generative network of the second generative adversarial neural network to obtain a fourth generated image; inputting the fourth generated image and a second training image associated with the third training image into a second discriminative network of the second generative adversarial neural network to obtain a fourth image recognition result; and determining the second loss function value of the second generative adversarial neural network according to the fourth image recognition result.

[0011] In some example embodiments of the present disclosure, the determining the third loss function value according to the comparison result of the obtained fusion image and the third training image comprises: fusing the third generated image and the fourth generated image according to an image fusion algorithm to obtain a fusion image; comparing the fusion image with the third training image to obtain a comparison result; and determining the third loss function value according to the comparison result.

[0012] In some example embodiments of the present disclosure, the second loss function comprises a generator loss function and a discriminator loss function, and the second loss function value comprises a generator loss function value and a discriminator loss function value.

[0013] In some example embodiments of the present disclosure, the preset weight of the generator loss function in the second loss function is greater than the preset weight of other loss functions.

[0014] In some example embodiments of the present disclosure, the training images in the training set are X-ray images after negative logarithmic processing.

[0015] In some example embodiments of the present disclosure, adjusting the parameters of the image decomposition model in a back propagation manner according to the second loss function value includes: adjusting the parameters of the first generation network of the first adversarial neural network and adjusting the parameters of the second generation network of the second adversarial neural network according to the generator loss function value in the second loss function value; and adjusting the parameters of the first discriminator network of the first adversarial neural network and adjusting the parameters of the second discriminator network of the second adversarial neural network according to the discriminator loss function value in the second loss function value.

[0016] In a second aspect of the present disclosure, an image decomposition method is provided, which is applied to X-ray image decomposition, and the method comprises: performing a negative logarithmic transformation on an obtained X-ray image to generate a to-be-decomposed image; processing the to-be-decomposed image through a first generation network in an image decomposition model to generate a first decomposition image; processing the to-be-decomposed image through a second generation network in the image decomposition model to generate a second decomposition image; and performing a negative logarithmic inverse operation transformation on the first decomposition image and the second decomposition image to generate an image decomposition result; the image decomposition model comprises: a first adversarial neural network and a second adversarial neural network, the first adversarial neural network comprises a first generation network and a first discriminator network, and the second adversarial neural network comprises a second generation network and a second discriminator network; wherein the image decomposition model is obtained according to the training method described above.

[0017] In a third aspect of the present disclosure, a training device of an image decomposition model is provided, wherein the image decomposition model comprises: a first adversarial neural network and a second adversarial neural network, the first adversarial neural network comprises a first generation network and a first discriminator network, and the second adversarial neural network comprises a second generation network and a second discriminator network.

[0018] The training device includes but is not limited to: a first acquisition module configured to acquire a training set, the training set including a first training image, a second training image, and a third training image, the third training image being generated by fusion according to the first training image and the second training image; a first loss function value determination module configured to input the first training image into the first adversarial neural network, input the second training image into the second adversarial neural network, and determine a first loss function value of the first adversarial neural network and the second adversarial neural network, respectively; a second loss function value determination module configured to input the third training image into the first adversarial neural network and the second adversarial neural network, respectively, and determine a second loss function value of the first adversarial neural network and the second adversarial neural network, respectively; a third loss function value determination module configured to determine a third loss function value according to a comparison result of an acquired fusion image and the third training image, the fusion image being generated by fusion according to generated images of the first adversarial neural network and the second adversarial neural network; and an adjustment module configured to adjust parameters of the image decomposition model in a back propagation manner according to at least one of the first loss function value, the second loss function value, and the third loss function value until a preset training termination condition is met, wherein the first loss function value is calculated based on a first loss function, the second loss function value is calculated based on a second loss function, and the third loss function value is calculated based on a third loss function, and the first loss function, the second loss function, and the third loss function satisfy a preset weight relationship.

[0019] In some example embodiments of the present disclosure, the first loss function value determination module further includes a first subunit configured to: input the first training image into a first generation network of the first adversarial neural network to obtain a first generated image; input the first training image and the first generated image into a first discrimination network of the first adversarial neural network to obtain a first image recognition result; and determine the first loss function value of the first adversarial neural network according to the first image recognition result.

[0020] In some example embodiments of the present disclosure, the first loss function value determination module further includes a second subunit configured to: input the second training image into a second generation network of the second adversarial neural network to obtain a second generated image; input the second training image and the second generated image into a second discrimination network of the second adversarial neural network to obtain a second image recognition result; and determine the first loss function value of the second adversarial neural network according to the second image recognition result.

[0021] In some example embodiments of the present disclosure, the second loss function value determination module comprises a third subunit configured to: input the third training image into a first generation network of the first generative adversarial network to obtain a third generated image; input the third generated image and a first training image associated with the third training image into a first discriminator network of the first generative adversarial network to obtain a third image recognition result; determine a second loss function value of the first generative adversarial network according to the third image recognition result; input the third training image into a second generation network of the second generative adversarial network to obtain a fourth generated image; input the fourth generated image and a second training image associated with the third training image into a second discriminator network of the second generative adversarial network to obtain a fourth image recognition result; and determine a second loss function value of the second generative adversarial network according to the fourth image recognition result.

[0022] In some example embodiments of the present disclosure, the third loss function value determination module comprises a fourth subunit configured to: fuse the third generated image and the fourth generated image according to an image fusion algorithm to generate a fused image; compare the fused image with the third training image to generate a comparison result; and determine a third loss function value according to the comparison result.

[0023] In some example embodiments of the present disclosure, the second loss function comprises a generator loss function and a discriminator loss function, and the second loss function value comprises a generator loss function value and a discriminator loss function value.

[0024] In some example embodiments of the present disclosure, the adjustment module comprises a fifth subunit configured to: adjust parameters of the first generation network of the first generative adversarial network and adjust parameters of the second generation network of the second generative adversarial network according to the generator loss function value in the second loss function value; and adjust parameters of the first discriminator network of the first generative adversarial network and adjust parameters of the second discriminator network of the second generative adversarial network according to the discriminator loss function value in the second loss function value.

[0025] In a fourth aspect of the present disclosure, an electronic device is provided, comprising: one or more processors; a storage device for storing executable instructions, which, when executed by the processor, implement the training method according to the first aspect above, or the image decomposition method according to the second aspect above.

[0026] In a fifth aspect of the present disclosure, a computer-readable storage medium is provided, and the computer-readable storage medium has stored thereon executable instructions that, when executed by a processor, implement the training method according to the first aspect above, or the image decomposition method according to the second aspect above.

[0027] In a sixth aspect of the present disclosure, a computer program product is provided, and the computer program product comprises a computer program, and the computer program, when executed by a processor, implements the training method according to the first aspect above, or the image decomposition method according to the second aspect above. BRIEF DESCRIPTION OF DRAWINGS

[0028] The above and other objects, features and advantages of the present disclosure will become more apparent from the following description when taken in conjunction with the accompanying drawings, in which:

[0029] Figure 1 A schematic diagram of a system architecture to which the image decomposition model training method or the image decomposition method according to an embodiment of the present disclosure can be applied is schematically shown;

[0030] Figure 2A A structural schematic diagram of an image decomposition model according to an embodiment of the present disclosure is schematically shown;

[0031] Figure 2B A flowchart of a training method of an image decomposition model according to an embodiment of the present disclosure is schematically shown;

[0032] Figure 3 A flowchart of the training method of the image decomposition model according to an embodiment of the present disclosure at operation S220 is schematically shown;

[0033] Figure 4 A flowchart of the training method of the image decomposition model according to yet another embodiment of the present disclosure at operation S220 is schematically shown;

[0034] Figure 5 A flowchart of the training method of the image decomposition model according to an embodiment of the present disclosure at operation S230 is schematically shown;

[0035] Figure 6 A flowchart of the training method of the image decomposition model according to an embodiment of the present disclosure at operation S240 is schematically shown;

[0036] Figure 7 A flowchart of the training method of the image decomposition model according to an embodiment of the present disclosure at operation S250 is schematically shown;

[0037] Figure 8A A schematic diagram of negative logarithmic processing of an image by the training method of the image decomposition model according to an embodiment of the present disclosure is schematically shown;

[0038] Figure 8B a schematic diagram illustrating generation of a third training image according to the first training image and the second training image;

[0039] Figure 8C a schematic diagram illustrating generation of a fusion image according to the third generated image and the fourth generated image;

[0040] Figure 9 a flowchart illustrating a method of image decomposition according to an embodiment of the disclosure;

[0041] Figure 10 a diagram illustrating an example of decomposing an image by a method of image decomposition according to an embodiment of the disclosure;

[0042] Figure 11 a block diagram illustrating a structure of a training apparatus of an image decomposition model according to an embodiment of the disclosure; and

[0043] Figure 12 a block diagram illustrating an electronic device suitable for implementing a training method of an image decomposition model or a method of image decomposition according to an embodiment of the disclosure. DETAILED DESCRIPTION

[0044] Hereinafter, embodiments of the disclosure will be described with reference to the accompanying drawings. It should be understood, however, that the description which follows is merely exemplary and is not intended to limit the scope of the disclosure. In the following detailed description of the embodiments of the disclosure, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. However, it would be apparent to one skilled in the art that the embodiments of the disclosure can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring the concepts of the disclosure.

[0045] The terms used herein are merely used to describe specific embodiments and are not intended to limit the disclosure. The terms "include", "comprise", and the like used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0046] All terms used herein (including technical and scientific terms) have meanings that are commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings that are consistent with the context of the specification, and should not be interpreted in an idealized or excessively formal manner.

[0047] In the case of using expressions like "at least one of A, B, and C", it will be understood that the phrase is intended to mean any of the following: "A" alone, "B" alone, "C" alone, "A and B" together, "A and C" together, "B and C" together, or "A and B and C" together, unless otherwise indicated.

[0048] In the technical solutions of the present disclosure, the acquisition, storage and application of user personal information comply with relevant laws and regulations, necessary security measures are taken, and the public order and good customs are not violated.

[0049] In the technical solutions of the present disclosure, the acquisition, storage and application of user personal information comply with relevant laws and regulations, necessary security measures are taken, and the public order and good customs are not violated.

[0050] In the embodiments of the present disclosure, in order to solve the problem that the features of the overlapping image of the goods and the vehicle body cannot be accurately distinguished and judged in the related art, the present disclosure provides a training method and training device of an image decomposition model, and an image decomposition method, an electronic device, a readable storage medium and a computer program product. The X-ray image of the overlapping of the vehicle and the goods can be effectively and accurately decomposed, and the efficiency of image security inspection can be effectively improved. The image decomposition model includes but is not limited to: a first adversarial neural network and a second adversarial neural network, the first adversarial neural network includes a first generation network and a first discrimination network, and the second adversarial neural network includes a second generation network and a second discrimination network. The training method of the image decomposition model includes but is not limited to: obtaining a training set, the training set includes a first training image, a second training image and a third training image, the third training image is generated by fusing the first training image and the second training image; inputting the first training image into the first adversarial neural network and inputting the second training image into the second adversarial neural network, respectively determining a first loss function value of the first adversarial neural network and the second adversarial neural network; inputting the third training image into the first adversarial neural network and the second adversarial neural network respectively, respectively determining a second loss function value of the first adversarial neural network and the second adversarial neural network; determining a third loss function value according to the comparison result of the obtained fusion image and the third training image, the fusion image is generated by fusing the generated image of the first adversarial neural network and the second adversarial neural network; and adjusting the parameters of the image decomposition model in a back propagation manner according to at least one of the first loss function value, the second loss function value and the third loss function value, until a preset training termination condition is met, wherein the first loss function value is calculated based on the first loss function, the second loss function value is calculated based on the second loss function, and the third loss function value is calculated based on the third loss function, the first loss function, the second loss function and the third loss function satisfy a preset weight relationship.

[0051] According to the embodiments of the present disclosure, by setting multiple training images (for example, the first training image, the second training image, and the third training image) in the training set, and determining the loss function values (for example, the first loss function value, the second loss function value, and the third loss function value) of different loss functions according to the training results obtained from the input different training images, the parameters in the first image decomposition model are adjusted, so that the trained image decomposition model can effectively and accurately decompose the X-ray image in which the cargo and the vehicle body are overlapped, and generate the X-ray images of the vehicle body and the cargo respectively, which facilitates the security inspection equipment and the security inspection personnel to further detect and analyze the cargo image, and improves the efficiency of security inspection.

[0052] Figure 1 A schematic diagram of a system architecture to which the image decomposition model training method or the image decomposition method according to the embodiments of the present disclosure can be applied is shown. It should be noted that, Figure 1 The shown is only an example of the system architecture to which the embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but does not mean that the embodiments of the present disclosure cannot be applied to other architectures, devices, systems, environments or scenarios. It should be noted that the training method of the image decomposition model, the image decomposition method, the device, the electronic device and the computer readable storage medium provided by the embodiments of the present disclosure can be used in the field of artificial intelligence technology, the field of image processing technology, and can also be used in other fields, and the training method of the image decomposition model, the image decomposition method, the device, the electronic device and the computer readable storage medium provided by the embodiments of the present disclosure are not limited to the application field.

[0053] As Figure 1 shown, the system architecture 100 according to the exemplary embodiments of the present disclosure can include an X-ray image acquisition device 101, a terminal device 102, a server 103 and a network 104.

[0054] The network 104 is used as a medium to provide a communication link between the X-ray image acquisition device 101, the terminal device 102 and the server 103. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0055] The user can use the X-ray image acquisition device 101 to acquire X-ray images of the target vehicle, and send them to the terminal device 102 and the server 103 through the network 102 for storage or processing.

[0056] The terminal device 102 can be various electronic devices with a display screen and supporting data processing, such as a tablet computer, a laptop computer, a desktop computer, etc.

[0057] The server 103 can be a server that provides various services, such as a background management server that provides support (for example, storage or processing) for image data collected by a user using an X-ray image collection device (for example only). The background management server can analyze and process received user data, and feed back the processing results to the terminal device.

[0058] It should be noted that the image decomposition model training method or image decomposition method provided by the embodiments of the present disclosure can generally be executed by the server 103. Accordingly, the image decomposition model training apparatus provided by the embodiments of the present disclosure can generally be disposed in the server 103. The image decomposition model training method or image decomposition method provided by the embodiments of the present disclosure can also be executed by a server or server cluster different from the server 103 and capable of communicating with the X-ray image collection device 101, the terminal device 102, and / or the server 103. Accordingly, the image decomposition model training apparatus provided by the embodiments of the present disclosure can also be disposed in a server or server cluster different from the server 103 and capable of communicating with the X-ray image collection device 101, the terminal device 102, and / or the server 103.

[0059] It should be understood that Figure 1 The number of X-ray image collection devices, terminal devices, networks, and servers in the system 100 is merely illustrative. Any number of X-ray image collection devices, terminal devices, and servers can be provided as needed.

[0060] Figure 2A An illustrative structural diagram of an image decomposition model according to an embodiment of the present disclosure is shown. Figure 2B An illustrative flowchart of a training method of an image decomposition model according to an embodiment of the present disclosure is shown. Figure 3 An illustrative flowchart of a training method of an image decomposition model according to an embodiment of the present disclosure at operation S220 is shown. Figure 4 An illustrative flowchart of a training method of an image decomposition model according to yet another embodiment of the present disclosure at operation S220 is shown. Figure 5 An illustrative flowchart of a training method of an image decomposition model according to an embodiment of the present disclosure at operation S230 is shown. Figure 6 An illustrative flowchart of a training method of an image decomposition model according to an embodiment of the present disclosure at operation S240 is shown. Figure 7 An illustrative flowchart of a training method of an image decomposition model according to an embodiment of the present disclosure at operation S250 is shown. Figure 8A An illustrative diagram of negative logarithmic processing of an image by a training method of an image decomposition model according to an embodiment of the present disclosure is shown. Figure 8B An illustrative diagram of generating a third training image from a first training image and a second training image is shown. Figure 8CA schematic diagram illustrating generation of a fusion image according to a third generated image and a fourth generated image is shown.

[0061] The following will be described by taking Figures 2A to 8C A training method of an image decomposition model of the disclosed embodiment will be described in detail.

[0062] As Figure 2A shown, the image decomposition model 20 of the disclosed embodiment includes a first adversarial neural network 21 and a second adversarial neural network 22. The first adversarial neural network 21 includes a first generation network 211 and a first discrimination network 212. The second adversarial neural network 22 includes a second generation network 221 and a second discrimination network 222. A training set 23 for input into the image decomposition model 20 includes a first training image 231, a second training image 232, and a third training image 233.

[0063] As Figure 2B shown, the training method of the image decomposition model of the disclosed embodiment includes operations S210 to S250.

[0064] In operation S210, a training set is obtained, which includes a first training image, a second training image, and a third training image, the third training image being generated by fusion of the first training image and the second training image.

[0065] In the disclosed embodiment, by generating the third training image by fusion of the first training image and the second training image, the problem of being unable to obtain sufficient true value data is solved, facilitating effective training of the image decomposition model.

[0066] In the disclosed embodiment, the X-ray scan image is an image generated after scanning an object using an X-ray scanning device based on the principle of X-ray perspective imaging. Due to the perspective characteristics of radiation imaging, after the scanning forms an image, the cargo information and the vehicle body structure information of the cargo vehicle will overlap with each other, and the confused image is difficult to distinguish the vehicle body structure and the cargo information through manual annotation, therefore, it is necessary to synthesize training images and test images through corresponding data synthesis means. In addition, unlike visible light imaging, the multiplicative superposition principle of X-ray images requires the use of corresponding preprocessing methods to ensure the accuracy of image fusion in image reconstruction.

[0067] Before the X-ray image is used as a training image in the training set, since the X-ray image is an image generated after the signal penetrates the scanned object. Due to the different attenuation characteristics of different substances, the different structures of the object can be revealed. The gray scale of the X-ray image, i.e. the transparency in physics, is defined as the ratio of the transmission intensity of the X-ray after penetrating the scanned object to the incident intensity of the X-ray source. It can be represented as (Formula 1):

[0068]

[0069] In the formula, u(l) represents the linear mass attenuation coefficient. Since the linear attenuation of different materials exhibits an exponential characteristic along the X-ray path, the X-ray image is first preprocessed, including negative logarithmic processing, such as... Figure 8A As shown, where Figure 8A Image a in the diagram is the image before negative logarithmic processing, and image b is the image after negative logarithmic processing. The image after negative logarithmic processing is used as input to the image decomposition model, ensuring that it conforms to the superposition principle when generating the fused image.

[0070] In embodiments of this disclosure, X-ray images are processed with a negative logarithm and used as training images. The first training image may be, for example, an image of an empty vehicle formed by X-ray irradiation. The second training image may be, for example, an image of cargo formed by X-ray irradiation. The third training image is generated by fusing the first and second training images, for example, an image of a loaded vehicle generated by fusing the empty vehicle image and the cargo image using a multiplicative addition algorithm. Figure 8B As shown, image a is the first training image, image b is the second training image, and image c is the third training image. Each third training image is associated with a first training image and a second training image.

[0071] In the embodiments of this disclosure, the fusion of the first training image and the second training image to generate the third training image follows a multiplicative addition principle, and the image fusion formula is as follows:

[0072] -log(T c ) = -log(T a )-log(T b ) (Formula 2)

[0073] Among them, T c Identify the features of image c, T a T represents the features of image a. b This represents the features of image b.

[0074] In the embodiments of the present disclosure, before the third training image is generated, the first training image and the second training image can also be processed, for example, the first training image and the second training image are denoised and vehicle cropping is performed, after the third training image is synthesized, random gray scale transformation, perspective transformation and other processing operations are performed on the third training image, to simulate different thicknesses and different quality attenuation coefficients of the material, or a random transformation perspective transformation function with random X, Y and Z coordinate system positive and negative setting angles is used to process (augment) the first training image, the second training image and the third training image, to form training image data of multiple scanning angles and multiple directions. Thus, the number of training samples in the training set is increased, and the image decomposition model obtained by training has better accuracy.

[0075] In operation S220, the first training image is input into the first adversarial neural network, and the second training image is input into the second adversarial neural network, to determine the first loss function value of the first adversarial neural network and the second adversarial neural network, respectively.

[0076] In the embodiments of the present disclosure, as shown in Figure 2A The first adversarial neural network 21 and the second adversarial neural network 22 are two adversarial neural networks with the same framework and structure. Each adversarial neural network has a generator and a discriminator. The generator is used to generate a new image according to the input training image. The discriminator is used to judge the image generated by the generator.

[0077] For example, the size of the training image input into the first generation network and the second generation network can be 1024*1024, and the number of channels is 3. The first generation network 211 and the second generation network 221 can use, for example, the pix2pix framework, the generator uses the U-net structure, the number of down-sampling layers is 4, and the bn layer and the dropout layer are used. The first discriminator network 212 and the second discriminator network 222 can use, for example, the PatchGAN structure, the number of down-sampling layers is 4, and the bn layer is used.

[0078] In the first adversarial neural network 21 and the second adversarial neural network 22, the first loss function (id loss) is set according to the identity loss of the Cycle GAN idea, for example, as shown in Figure 2A The first loss function 214 of the first adversarial neural network 21 and the first loss function 224 of the second adversarial neural network 22 are shown. According to the input first training image 231 and the second training image 232, the first loss function value of the first adversarial neural network 21 and the second adversarial neural network 22 is determined, respectively.

[0079] Figure 3 The flowchart of the training method of the image decomposition model according to an embodiment of the present disclosure is schematically shown in operation S220.Figure 4 A flowchart of the method for training the image decomposition model according to another embodiment of the present disclosure is shown in operation S220.

[0080] The operation S220 of the embodiments of the present disclosure will be described in detail below in combination with Figure 2A , Figure 3 and Figure 4 As shown in Figure 3 , the operation S220 includes operations S221 to S223.

[0081] In operation S221, the first training image 231 is input into the first generation network 211 of the first adversarial neural network 21 to obtain a first generated image 213.

[0082] In the embodiments of the present disclosure, the first training image can be a real image, for example, an empty vehicle image, as an input image of the first adversarial neural network. The first generated image is a false image generated according to the first training image.

[0083] In operation S222, the first training image 231 and the first generated image 213 are input into the first discriminant network 212 of the first adversarial neural network 21 to obtain a first image recognition result.

[0084] After the first training image and the first generated image are input into the first discriminant network, the first discriminant network discriminates the input images, thereby judging the authenticity of the first generated image generated by the first generator.

[0085] In operation S223, the first loss function value of the first adversarial neural network 21 is determined according to the first image recognition result.

[0086] In the embodiments of the present disclosure, the first loss function can be selected as, for example, an L1 loss function, i.e., a least absolute deviation (LAD) or a least absolute error (LAE).

[0087] For example, according to the determination of the first loss function value of the first adversarial neural network, the parameters of the first adversarial neural network 21 can be adjusted through a back propagation algorithm.

[0088] As shown in Figure 4 , the operation S220 includes operations S224 to S226.

[0089] As shown in Figure 2A and Figure 4 , in operation S224, the second training image 232 is input into the second generation network 221 of the second adversarial neural network 22 to obtain a second generated image 223.

[0090] In the embodiments of the present disclosure, the second training image can be a real image, for example, a cargo image, as an input image of the first adversarial neural network. The second generated image is a false image generated according to the second training image.

[0091] Exemplarily, the cargo image can be air, that is, a blank image, so as to make the first adversarial neural network and the second adversarial neural network also have a good decomposition effect for the image without cargo.

[0092] In operation S225, the second training image 232 and the second generated image 223 are input into the second discriminator network 222 of the second adversarial neural network 22 to obtain a second image recognition result.

[0093] After the second training image and the second generated image are input into the second discriminator network, the second discriminator network discriminates the input image, so as to judge the true or false of the second generated image generated by the second generator.

[0094] In operation S226, the first loss function value of the second adversarial neural network 22 is determined according to the second image recognition result.

[0095] In the embodiments of the present disclosure, the first loss function can be selected as, for example, an L1 loss function, that is, a least absolute deviation (LAD) and a least absolute error (LAE). The first loss function can include, for example, a generator loss function. For example, it can be a generator loss function of the first generation network and / or a generator loss function of the second generation network.

[0096] Exemplarily, according to the determination of the first loss function value of the second adversarial neural network, the parameters of the second adversarial neural network 22 can be adjusted through a back propagation algorithm.

[0097] In the embodiments of the present disclosure, through operation S220, the first training image 231 and the second training image 232 are respectively input into the first adversarial neural network. Specifically, the first training image 231 is an empty vehicle image, and the second training image 232 can be an empty image, that is, an image without cargo, so as to determine the first loss function value based on the first loss function, and solve the problem of decomposition of the vehicle without cargo. By setting the first loss function and adjusting the parameters of the first adversarial neural network and the second adversarial neural network based on the determined first loss function value, the image decomposition model obtained by training has higher recognition accuracy and can well recognize the vehicle without cargo, thereby improving the effect of image decomposition based on the image decomposition model.

[0098] In operation S230, the third training image is input into the first adversarial neural network and the second adversarial neural network respectively, and the second loss function value of the first adversarial neural network and the second adversarial neural network is determined respectively.

[0099] Figure 5 A flowchart of the training method of the image decomposition model according to an embodiment of the present disclosure is shown in operation S230.

[0100] The following describes the training method of the image decomposition model according to an embodiment of the present disclosure in detail. Figure 2A And Figure 5 Operation S230 is described in detail. Operation S230 includes operation S231 to operation S236.

[0101] In operation S231, the third training image 233 is input into the first generation network 211 of the first adversarial neural network 21 to obtain a third generated image 213.

[0102] In an embodiment of the present disclosure, each third training image 233 is associated with a first training image 231 and a second training image 232. That is, each third training image 233 is generated by fusing the first training image 231 and the second training image 232, and when the third training image 233 is obtained, the first training image 231 and the second training image 232 associated with the third training image can be determined according to the association relationship.

[0103] In operation S232, the third generated image 213 and the first training image 231 associated with the third training image 233 are input into the first discriminant network 212 of the first adversarial neural network 21 to obtain a third image recognition result.

[0104] In an embodiment of the present disclosure, the third generated image 213 is a false image generated by the first generation network according to the input third training image, and by inputting the third generated image 213 and the first training image 231 associated with the third training image 233 into the first discriminant network, the authenticity of the third generated image 213 generated by the first generation network is determined.

[0105] Exemplarily, the fused vehicle image is input into the first generation network 211 to generate a vehicle image.

[0106] In operation S233, the second loss function value of the first adversarial neural network 21 is determined according to the third image recognition result.

[0107] In an embodiment of the present disclosure, the first adversarial neural network has a second loss function 215, and the second loss function value of the first adversarial neural network 21 is determined according to the second loss function 215 and the third image recognition result. The parameters of the first adversarial neural network are adjusted by the back propagation method according to the determined second loss function value.

[0108] In operation S234, the third training image 233 is input into the second generation network 221 of the second adversarial neural network 22 to obtain a fourth generated image 223.

[0109] In an embodiment of the present disclosure, the fourth generated image 223 is a fake image generated by the second generation network 221 according to the input third training image.

[0110] Exemplarily, the generated truck carrying vehicle image is input into the second generation network 221, and a cargo image is generated.

[0111] In operation S235, the fourth generated image 223 and the second training image 232 associated with the third training image 233 are input into the second discriminator network 222 of the second adversarial neural network, and a fourth image recognition result is obtained.

[0112] In an embodiment of the present disclosure, the authenticity of the fourth generated image 223 generated by the second generation network 221 is judged by inputting the third generated image 213 and the second training image 232 associated with the third training image 233 into the second discriminator network 222.

[0113] In operation S236, a second loss function value of the second adversarial neural network 22 is determined according to the fourth image recognition result.

[0114] In an embodiment of the present disclosure, the second adversarial neural network has a second loss function 225, and the second loss function value of the second adversarial neural network 22 is determined according to the second loss function 225 and the fourth image recognition result. The parameters of the second adversarial neural network are adjusted by the back propagation mode according to the determined second loss function value.

[0115] In an embodiment of the present disclosure, the second loss function includes a generator loss function and a discriminator loss function, and the second loss function value includes a generator loss function value and a discriminator loss function value.

[0116] Exemplarily, the generator loss function may, for example, be an L1 loss function, and the discriminator loss function may, for example, adopt a PatchGAN structure and use a BCE (Binary Cross Entropy) loss function.

[0117] For example, the first adversarial generation network adjusts the parameters of the first generation network by the back propagation mode by obtaining the generator loss function value in the second loss function, and adjusts the parameters of the first discriminator network by the back propagation mode by obtaining the discriminator loss function value in the second loss function.

[0118] For another example, the second adversarial generation network adjusts the parameters of the second generation network by the back propagation mode by obtaining the generator loss function value in the second loss function, and adjusts the parameters of the second discriminator network by the back propagation mode by obtaining the discriminator loss function value in the second loss function.

[0119] In operation S240, a third loss function value is determined according to a comparison result of the generated fusion image and the third training image. The fusion image is generated by fusing the generated image of the first generative adversarial network and the generated image of the second generative adversarial network.

[0120] Figure 6 A flowchart of the training method of the image decomposition model according to an embodiment of the present disclosure is schematically shown in operation S240.

[0121] The operation S240 is described in detail below. Figure 2A and Figure 6 The operation S240 includes operations S241 to S243.

[0122] In an embodiment of the present disclosure, the third generated image and the fourth generated image are respectively false images generated by the first generative network and the second generative network according to the input third training image.

[0123] In operation S241, the third generated image 213 and the fourth generated image 223 are fused according to an image fusion algorithm to generate a fusion image 24.

[0124] In an embodiment of the present disclosure, the image fusion algorithm may, for example, fuse the third generated image and the fourth generated image according to the formula 2 described above to generate the fusion image. When the first generative network and the second generative network are trained to achieve a relatively accurate effect, the similarity between the generated fusion image and the third training image is relatively high. If the similarity between the generated fusion image and the third training image is relatively low, it indicates that the first generative network and the second generative network have not been trained well and need to be further trained.

[0125] As shown in FIG. 2, for example, the image a1 is an empty truck image generated by the first generative network, the image b1 is a cargo image generated by the second generative network, and the image c1 is a fusion image generated by fusing the empty truck image generated by the first generative network and the cargo image generated by the second generative network through the fusion algorithm. Figure 8C

[0126] For another example, the image a2 is another empty truck image generated by the first generative network, the image b2 is another cargo image generated by the second generative network, and the image c2 is a fusion image generated by fusing the another empty truck image generated by the first generative network and the another cargo image generated by the second generative network through the fusion algorithm.

[0127] In operation S242, the fusion image is compared with the third training image to generate a comparison result.

[0128] ​Exemplarily, the fusion image 24 is compared with the third training image 233 in the comparator 25 to generate a comparison result 26. For example, a similarity between the fusion image 24 and the third training image can be compared.

[0129] In operation S243, a third loss function value is determined according to the comparison result.

[0130] Exemplarily, the size of the third loss function value is determined according to the generated comparison result 26 and the third loss function 27. The third loss function value can be used to adjust the parameters of the first and second adversarial neural networks 21 and 22 in a back propagation manner, so as to make the images of the first and second adversarial neural networks closer to the real value.

[0131] Exemplarily, the third loss function can be an L1 loss function, for example. The third loss function can include a generator loss function, for example. For example, the generator loss function of the first generator network and / or the generator loss function of the second generator network.

[0132] In operation S250, the parameters of the image decomposition model are adjusted in a back propagation manner according to at least one of the first loss function value, the second loss function value and the third loss function value, until a preset training termination condition is met. The first loss function value is calculated based on the first loss function, the second loss function value is calculated based on the second loss function, and the third loss function value is calculated based on the third loss function. The first loss function, the second loss function and the third loss function satisfy a preset weight relationship.

[0133] In an embodiment of the present disclosure, the parameters of the image decomposition model are adjusted in a back propagation manner according to at least one of the first loss function value, the second loss function value and the third loss function value, until a preset training termination condition is met.

[0134] Exemplarily, the parameters of the image decomposition model can be adjusted in a back propagation manner according to the first loss function value. For example, the parameters of the first generator network in the first adversarial neural network and / or the parameters of the second generator network in the second adversarial neural network in the image decomposition model are adjusted in a back propagation manner according to the first loss function value.

[0135] Exemplarily, the parameters of the image decomposition model can be adjusted in a back propagation manner according to the second loss function value. For example, the parameters of the first generator network and the first discriminator network in the first adversarial neural network and / or the parameters of the second generator network and the second discriminator network in the second adversarial neural network in the image decomposition model are adjusted in a back propagation manner according to the second loss function value.

[0136] Exemplarily, the parameters of the image decomposition model can be adjusted in a back propagation manner according to the third loss function value. For example, the parameters of the first generative network in the first adversarial neural network and / or the parameters of the second generative network in the second adversarial neural network in the image decomposition model are adjusted in a back propagation manner according to the third loss function value.

[0137] Exemplarily, the parameters of the image decomposition model are adjusted in a back propagation manner according to a plurality of the first loss function value, the second loss function value and the third loss function value. Specifically, the parameters of the first generative network and the second generative network are adjusted according to the weights of different loss functions.

[0138] In an embodiment of the present disclosure, the first loss function, the second loss function and the third loss function satisfy a preset weight relationship. For example, the preset weight relationship can be described by the following formula:

[0139] Total loss =G loss ×100.0+D loss ×1.0+id loss ×1.0+Recon loss ×1.0 (Formula 3)

[0140] wherein, id loss represents the first loss function, G loss represents the generator loss function in the second loss function, D loss represents the discriminator loss function in the second loss function, and Recon loss represents the third loss function.

[0141] In an embodiment of the present disclosure, the preset weight of the generator loss function G loss in the second loss function is greater than the preset weights of other loss functions (for example, the first loss function, the discriminator loss function and the third loss function), so that the generated image is closer to the true value.

[0142] In the process of training the image decomposition model, the parameters of the image decomposition model are adjusted in a back propagation manner by determining at least one of the first loss function value, the second loss function value and the third loss function value until a preset training termination condition is met. When adjusting the parameters of the image decomposition model, one of the loss function values can be used for adjustment, or the preset weight relationship can be used for adjustment, so as to realize faster training of the image decomposition model and make the trained image decomposition model have more accurate decomposition effect for X-ray images.

[0143] In the embodiments of the present disclosure, the parameters of the image decomposition model are adjusted in a back propagation manner until a preset training termination condition is met, for example, when the first discriminator network and the second discriminator network cannot determine the true or false of the third generated image generated by the first generation network and the fourth generated image generated by the second generation network, and the similarity between the fusion image fused according to the third generated image and the fourth generated image and the third training image exceeds the set threshold, it can be considered that the image decomposition model training is completed.

[0144] When adjusting the parameters of the image decomposition model in a back propagation manner, a gradient optimization method can be used, for example, methods such as batch gradient descent (BGD), stochastic gradient descent (SGD), mini-batch gradient descent (MBSGD), momentum gradient descent (Momentum), adaptive optimization (AdaGrad), etc. Preferably, the adaptive momentum estimation algorithm (Adam) is used in the present embodiment.

[0145] Exemplarily, the adaptive momentum estimation algorithm (Adam) is used, the learning rate parameter is 2e-4, and the betas parameter is 0.5-0.999.

[0146] Figure 7 The flowchart of the training method of the image decomposition model according to the embodiments of the present disclosure is schematically shown in operation S250.

[0147] Operation S250 includes operation S251 to operation S252.

[0148] In operation S251, the parameters of the first generation network of the first adversarial neural network and the parameters of the second generation network of the second adversarial neural network are adjusted according to the generator loss function value in the second loss function value.

[0149] In operation S252, the parameters of the first discriminator network of the first adversarial neural network and the parameters of the second discriminator network of the second adversarial neural network are adjusted according to the discriminator loss function value in the second loss function value.

[0150] In the embodiments of the present disclosure, after the training of the image decomposition model is completed, a test image is input to the image decomposition model to obtain a test result.

[0151] In the embodiment of the present disclosure, by acquiring X-ray irradiation on the empty vehicle to generate a vehicle image, and acquiring X-ray irradiation on the cargo to generate a cargo image, a training set, for example, including 75000 images, is obtained for training the image decomposition model by fusing the vehicle image and the cargo image to generate a fusion image. The trained image decomposition model is tested on 5000 real vehicle-cargo databases, trained on a k80 server, and the convergence speed is about 8 hours, and the iteration number is 50000. The batch size is set to 4.

[0152] Figure 9 A flowchart of an image decomposition method according to an embodiment of the present disclosure is schematically shown.

[0153] In the exemplary embodiment of the present disclosure, an image decomposition method is also provided, which is applied to X-ray image decomposition. As shown in Figure 9 The image decomposition method 300 includes operations S310 to S340.

[0154] In operation S310, for the acquired X-ray image, a negative logarithmic transformation is performed to generate a to-be-decomposed image.

[0155] In the embodiment of the present disclosure, by performing negative logarithmic transformation on the X-ray image, the to-be-decomposed image can be accurately decomposed in the image decomposition model.

[0156] In operation S320, the to-be-decomposed image is processed by a first generation network in the image decomposition model to generate a first decomposition image.

[0157] In operation S330, the to-be-decomposed image is processed by a second generation network in the image decomposition model to generate a second decomposition image.

[0158] In the embodiment of the present disclosure, the first generation network and the second generation network are obtained after training, and have good decomposition effect on the X-ray image, which can accurately obtain the vehicle image and the cargo image in the X-ray image.

[0159] In operation S340, the first decomposition image and the second decomposition image are subjected to negative logarithmic inverse operation transformation to generate an image decomposition result.

[0160] In the embodiment of the present disclosure, the first decomposition image and the second decomposition image are subjected to negative logarithmic inverse operation transformation to generate an image decomposition result, which is convenient for artificial observation or machine processing operation on the image decomposition result.

[0161] In the embodiments of the present disclosure, the image decomposition model comprises: a first adversarial neural network comprising a first generation network and a first discrimination network, and a second adversarial neural network comprising a second generation network and a second discrimination network; wherein the image decomposition model applied in the image decomposition method is obtained according to the image decomposition model training method described above.

[0162] Figure 10 A schematic diagram of image decomposition by the image decomposition method according to the embodiments of the present disclosure is schematically shown.

[0163] As Figure 10 shown, the image a, the image b, the image c and the image d can be images of goods and vehicles, and after the images of the goods and vehicles are acquired, the images are decomposed to obtain vehicle images and goods images respectively. Specifically, the image a, the image b, the image c and the image d are decomposed respectively by the image decomposition method described above, so as to obtain decomposed vehicle images and goods images.

[0164] According to the embodiments of the present disclosure, by setting multiple training images (such as the first training image, the second training image and the third training image) in the training set, and determining the loss function values (such as the first loss function value, the second loss function value and the third loss function value) of different loss functions according to the training results obtained by inputting different training images, the parameters in the first image decomposition model are adjusted, so that the trained image decomposition model can effectively and accurately decompose the X-ray images of the goods and vehicle body overlap for the X-ray images, and generate X-ray images of the vehicle body and the goods respectively, which is convenient for the security inspection equipment and the security inspection personnel to further detect and analyze the goods images, and improves the efficiency of security inspection.

[0165] Figure 11 A structural block diagram of a training device of the image decomposition model according to the embodiments of the present disclosure is schematically shown.

[0166] As Figure 11 shown, the training device 400 of the image decomposition model according to the embodiments of the present disclosure comprises a first acquisition module 410, a first loss function value determination module 420, a second loss function value determination module 430, a third loss function value determination module 440 and an adjustment module 450.

[0167] The first acquisition module 410 is configured to acquire a training set, and the training set comprises a first training image, a second training image and a third training image, and the third training image is generated by fusion of the first training image and the second training image. In an embodiment, the first acquisition module 410 can be used to perform the operation S210 described above, and details are not repeated here.

[0168] The first loss function value determination module 420 is configured to input the first training image into the first generative adversarial neural network and input the second training image into the second generative adversarial neural network, and determine the first loss function value of the first generative adversarial neural network and the second generative adversarial neural network, respectively. In an embodiment, the first loss function value determination module 420 can be configured to perform the operation S220 described above, and details are not repeated here.

[0169] The second loss function value determination module 430 is configured to input the third training image into the first generative adversarial neural network and the second generative adversarial neural network, respectively, and determine the second loss function value of the first generative adversarial neural network and the second generative adversarial neural network. The second loss function value determination module 430 can be configured to perform the operation S230 described above, and details are not repeated here.

[0170] The third loss function value determination module 440 is configured to determine the third loss function value based on the comparison result of the obtained fusion image and the third training image, the fusion image being generated by fusing the generated images of the first generative adversarial neural network and the second generative adversarial neural network. The third loss function value determination module 440 can be configured to perform the operation S240 described above, and details are not repeated here.

[0171] The adjustment module 450 is configured to adjust the parameters of the image decomposition model in a back propagation manner according to at least one of the first loss function value, the second loss function value, and the third loss function value, until a preset training termination condition is met, the first loss function value being calculated based on the first loss function, the second loss function value being calculated based on the second loss function, and the third loss function value being calculated based on the third loss function, the first loss function, the second loss function, and the third loss function satisfying a preset weight relationship. The adjustment module 450 can be configured to perform the operation S250 described above, and details are not repeated here.

[0172] In some exemplary embodiments of the present disclosure, the first loss function value determination module further includes a first subunit, the first subunit being configured to: input the first training image into the first generative network of the first generative adversarial neural network to obtain a first generated image; input the first training image and the first generated image into the first discriminant network of the first generative adversarial neural network to obtain a first image recognition result; and determine the first loss function value of the first generative adversarial neural network based on the first image recognition result.

[0173] In some example embodiments of the present disclosure, the first loss function value determination module further includes a second subunit configured to: input the second training image into a second generation network of the second generative adversarial neural network to obtain a second generated image; input the second training image and the second generated image into a second discriminator network of the second generative adversarial neural network to obtain a second image recognition result; and determine the first loss function value of the second generative adversarial neural network according to the second image recognition result.

[0174] In some example embodiments of the present disclosure, the second loss function value determination module includes a third subunit configured to: input the third training image into a first generation network of the first generative adversarial neural network to obtain a third generated image; input the third generated image and a first training image associated with the third training image into a first discriminator network of the first generative adversarial neural network to obtain a third image recognition result; determine the second loss function value of the first generative adversarial neural network according to the third image recognition result; input the third training image into a second generation network of the second generative adversarial neural network to obtain a fourth generated image; input the fourth generated image and a second training image associated with the third training image into a second discriminator network of the second generative adversarial neural network to obtain a fourth image recognition result; and determine the second loss function value of the second generative adversarial neural network according to the fourth image recognition result.

[0175] In some example embodiments of the present disclosure, the third loss function value determination module includes a fourth subunit configured to: fuse the third generated image and the fourth generated image according to an image fusion algorithm to generate a fused image; compare the fused image with the third training image to generate a comparison result; and determine the third loss function value according to the comparison result.

[0176] In some example embodiments of the present disclosure, the second loss function includes a generator loss function and a discriminator loss function, and the second loss function value includes a generator loss function value and a discriminator loss function value.

[0177] In some example embodiments of the present disclosure, the adjustment module includes a fifth subunit configured to: adjust parameters of the first generation network of the first generative adversarial neural network and adjust parameters of the second generation network of the second generative adversarial neural network according to the generator loss function value in the second loss function value; and adjust parameters of the first discriminator network of the first generative adversarial neural network and adjust parameters of the second discriminator network of the second generative adversarial neural network according to the discriminator loss function value in the second loss function value.

[0178] According to an embodiment of the present disclosure, any of the first obtaining module 410, the first loss function value determining module 420, the second loss function value determining module 430, the third loss function value determining module 440, and the adjusting module 450 can be combined in one module, or any of them can be split into multiple modules. Alternatively, at least part of the function of one or more of these modules can be combined with at least part of the function of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the first obtaining module 410, the first loss function value determining module 420, the second loss function value determining module 430, the third loss function value determining module 440, and the adjusting module 450 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of hardware or firmware that can be integrated or packaged with a circuit, or any one of software, hardware and firmware or any appropriate combination of several of them. Alternatively, at least one of the first obtaining module 410, the first loss function value determining module 420, the second loss function value determining module 430, the third loss function value determining module 440, and the adjusting module 450 can be at least partially implemented as a computer program module that can perform corresponding functions when it is run.

[0179] Figure 12 A block diagram of an electronic device suitable for implementing the training method of the image decomposition model or the image decomposition method according to an embodiment of the present disclosure is schematically shown. Figure 12 The electronic device shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0180] As shown in Figure 12 The electronic device 500 according to an embodiment of the present disclosure includes a processor 501 which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 502 or loaded from a storage portion 508 into a random access memory (RAM) 503. The processor 501 can include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor, and / or a related chipset, and / or a special-purpose microprocessor (such as an application specific integrated circuit (ASIC)), and the like. The processor 501 can also include an on-board memory for cache use. The processor 501 can include a single processing unit or multiple processing units for performing different actions of the method processes according to embodiments of the present disclosure.

[0181] In the RAM 503, various programs and data required for the operation of the electronic device 500 are stored. The processor 501, the ROM 502, and the RAM 503 are connected to each other via the bus 504. The processor 501 performs various operations of the method flow according to the embodiments of the present disclosure by executing the programs in the ROM 502 and / or the RAM 503. It should be noted that the programs can also be stored in one or more memories other than the ROM 502 and the RAM 503. The processor 501 can also perform various operations of the method flow according to the embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0182] According to an embodiment of the present disclosure, the electronic device 500 can further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 can further include one or more of the following components connected to the I / O interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 510 as necessary, so that a computer program read out therefrom is installed in the storage portion 508 as necessary.

[0183] The present disclosure also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments, or can exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, which, when executed, implement the training method of the image decomposition model or the image decomposition method according to the embodiments of the present disclosure.

[0184] According to embodiments of the present disclosure, the computer readable storage medium can be a non-transitory computer readable storage medium, for example, can include but is not limited to: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this disclosure, a computer readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, the computer readable storage medium can include one or more memories, such as the ROM 502 and / or the RAM 503 described above, and / or one or more memories other than the ROM 502 and the RAM 503.

[0185] Embodiments of the present disclosure also include a computer program product, which includes a computer program containing program codes for executing the methods shown in the flowcharts. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the training method of the image decomposition model or the image decomposition method provided by the embodiments of the present disclosure.

[0186] The above-described functions defined in the system / device / apparatus of the embodiments of the present disclosure are performed when the computer program is executed by the processor 501. According to embodiments of the present disclosure, the system, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0187] In one embodiment, the computer program can rely on tangible storage media such as optical storage media, magnetic storage media, etc. In another embodiment, the computer program can also be transmitted, distributed, and downloaded in the form of a signal on a network medium, and be downloaded and installed through the communication part 509, and / or installed from the detachable medium 511. The program codes contained in the computer program can be transmitted by any appropriate network medium, including but not limited to wireless, wired, etc., or any suitable combination of the foregoing.

[0188] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the detachable medium 511. When the computer program is executed by the processor 501, the above-described functions defined in the system of the embodiments of the present disclosure are performed. According to embodiments of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0189] According to embodiments of the present disclosure, program code of the computer program for performing the methods provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages, and can be implemented in a computer program product. Specifically, the computer program can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. The programming language includes, but is not limited to, Java, C++, python, “C” language, or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, and partly on a remote computing device, or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider (ISP).

[0190] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a part of a code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks depicted in succession can in fact be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams or flow diagrams and combinations of blocks in the block diagrams or flow diagrams can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0191] Those skilled in the art can understand that the features described in various embodiments of the present disclosure and / or claims can be combined or / and integrated, even if such combinations or integrations are not explicitly described in the present disclosure. In particular, the features described in various embodiments of the present disclosure and / or claims can be combined and / or integrated in various combinations, without departing from the spirit and teachings of the present disclosure. All these combinations and / or integrations fall within the scope of the present disclosure.

[0192] The above describes embodiments of the present disclosure. However, these embodiments are merely for illustrative purposes, and are not intended to limit the scope of the present disclosure. Although each embodiment is described above separately, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Those skilled in the art can make various substitutions and modifications without departing from the scope of the present disclosure, and these substitutions and modifications should all fall within the scope of the present disclosure.

Claims

1. A method of training an image decomposition model, wherein, The image decomposition model comprises: a first adversarial neural network comprising a first generation network and a first discriminator network, and a second adversarial neural network comprising a second generation network and a second discriminator network; The training method comprises: obtaining a training set comprising a first training image, a second training image, and a third training image generated by fusing the first training image and the second training image; inputting the first training image into the first adversarial neural network and the second training image into the second adversarial neural network to determine first loss function values of the first adversarial neural network and the second adversarial neural network, respectively; inputting the third training image into the first adversarial neural network and the second adversarial neural network to determine second loss function values of the first adversarial neural network and the second adversarial neural network, respectively; determining a third loss function value based on a comparison result of a fusion image and the third training image, the fusion image being generated by fusing generated images of the first adversarial neural network and the second adversarial neural network; and adjusting parameters of the image decomposition model in a back propagation manner based on at least one of the first loss function value, the second loss function value, and the third loss function value until a preset training termination condition is met, wherein the first loss function value is calculated based on a first loss function, the second loss function value is calculated based on a second loss function, and the third loss function value is calculated based on a third loss function, the first loss function, the second loss function, and the third loss function satisfy a preset weight relationship.

2. The training method of claim 1, wherein inputting the first training image into the first adversarial neural network to determine the first loss function value of the first adversarial neural network comprises: inputting the first training image into the first generation network of the first adversarial neural network to obtain a first generated image; inputting the first training image and the first generated image into the first discriminator network of the first adversarial neural network to obtain a first image recognition result; determining the first loss function value of the first adversarial neural network based on the first image recognition result.

3. The training method of claim 1, wherein inputting the second training image into the second adversarial neural network to determine the first loss function value of the second adversarial neural network comprises: inputting the second training image into the second generation network of the second adversarial neural network to obtain a second generated image; inputting the second training image and the second generated image into the second discriminator network of the second adversarial neural network to obtain a second image recognition result; determining the first loss function value of the second adversarial neural network based on the second image recognition result.

4. The training method of claim 1, wherein The inputting the third training image into the first generation network of the first generative adversarial network respectively, and obtaining a third generated image comprises: inputting the third training image into the first generation network of the first generative adversarial network respectively, and obtaining a third generated image; inputting the third generated image and the first training image associated with the third training image into the first discriminant network of the first generative adversarial network, and obtaining a third image recognition result; determining the second loss function value of the first generative adversarial network according to the third image recognition result; inputting the third training image into the second generation network of the second generative adversarial network, and obtaining a fourth generated image; inputting the fourth generated image and the second training image associated with the third training image into the second discriminant network of the second generative adversarial network, and obtaining a fourth image recognition result; determining the second loss function value of the second generative adversarial network according to the fourth image recognition result.

5. The training method of claim 4, wherein the comparison result of the fusion image obtained by comparing the third generated image and the fourth generated image, and determining a third loss function value, comprises: fusing the third generated image and the fourth generated image according to an image fusion algorithm to obtain a fusion image; comparing the fusion image with the third training image to obtain a comparison result; determining a third loss function value according to the comparison result.

6. The training method of any one of claims 1 to 5, wherein the second loss function comprises a generator loss function and a discriminator loss function, the second loss function value comprises a generator loss function value and a discriminator loss function value.

7. The training method of claim 6, wherein the preset weight of the generator loss function in the second loss function is greater than the preset weight of other loss functions.

8. The training method of claim 1, wherein the training image in the training set is an X-ray image after negative logarithmic processing.

9. The training method of claim 6, wherein adjusting the parameters of the image decomposition model in a back propagation manner according to the second loss function value comprises: adjusting the parameters of the first generation network of the first generative adversarial network and adjusting the parameters of the second generation network of the second generative adversarial network according to the generator loss function value in the second loss function value; and adjusting the parameters of the first discriminant network of the first generative adversarial network and adjusting the parameters of the second discriminant network of the second generative adversarial network according to the discriminator loss function value in the second loss function value.

10. An image decomposition method applied to X-ray image decomposition, comprising: performing negative logarithmic transformation on the obtained X-ray image to obtain a to-be-decomposed image; processing the to-be-decomposed image through the first generation network in the image decomposition model to obtain a first decomposed image; processing the to-be-decomposed image through the second generation network in the image decomposition model to obtain a second decomposed image; and ​ performing a negative logarithmic inverse operation transformation on the first decomposition image and the second decomposition image to generate an image decomposition result; The image decomposition model comprises: a first adversarial neural network and a second adversarial neural network, the first adversarial neural network comprising a first generation network and a first discrimination network, and the second adversarial neural network comprising a second generation network and a second discrimination network; The image decomposition model is obtained according to the training method of claims 1-9.

11. An apparatus for training an image decomposition model, wherein, The image decomposition model comprises: a first adversarial neural network and a second adversarial neural network, the first adversarial neural network comprising a first generation network and a first discrimination network, and the second adversarial neural network comprising a second generation network and a second discrimination network; The training device comprises: a first acquisition module configured to acquire a training set, the training set comprising a first training image, a second training image, and a third training image, the third training image being generated by fusion of the first training image and the second training image; a first loss function value determination module configured to input the first training image into the first adversarial neural network and input the second training image into the second adversarial neural network, and determine a first loss function value of the first adversarial neural network and the second adversarial neural network, respectively; a second loss function value determination module configured to input the third training image into the first adversarial neural network and the second adversarial neural network, respectively, and determine a second loss function value of the first adversarial neural network and the second adversarial neural network, respectively; a third loss function value determination module configured to determine a third loss function value based on a comparison result of a fusion image and the third training image, the fusion image being generated by fusion of a generated image of the first adversarial neural network and a generated image of the second adversarial neural network; an adjustment module configured to adjust parameters of the image decomposition model in a back propagation manner based on at least one of the first loss function value, the second loss function value, and the third loss function value until a preset training termination condition is met, wherein the first loss function value is calculated based on a first loss function, the second loss function value is calculated based on a second loss function, and the third loss function value is calculated based on a third loss function, the first loss function, the second loss function, and the third loss function satisfy a preset weight relationship.

12. The training device of claim 11, wherein the first loss function value determination module further comprises a first subunit configured to: input the first training image into the first generation network of the first adversarial neural network to obtain a first generated image; input the first training image and the first generated image into the first discrimination network of the first adversarial neural network to obtain a first image recognition result; determine the first loss function value of the first adversarial neural network based on the first image recognition result.

13. The training device of claim 11, wherein the first loss function value determination module further comprises a second subunit configured to: inputting the second training image into a second generation network of the second generative adversarial neural network to obtain a second generated image; inputting the second training image and the second generated image into a second discriminator network of the second generative adversarial neural network to obtain a second image recognition result; determining a first loss function value of the second generative adversarial neural network according to the second image recognition result.

14. The training apparatus of claim 11, wherein the second loss function value determination module comprises a third subunit configured to: input the third training image into a first generation network of the first generative adversarial neural network to obtain a third generated image; input the third generated image and a first training image associated with the third training image into a first discriminator network of the first generative adversarial neural network to obtain a third image recognition result; determine a second loss function value of the first generative adversarial neural network according to the third image recognition result; input the third training image into a second generation network of the second generative adversarial neural network to obtain a fourth generated image; input the fourth generated image and a second training image associated with the third training image into a second discriminator network of the second generative adversarial neural network to obtain a fourth image recognition result; determine a second loss function value of the second generative adversarial neural network according to the fourth image recognition result.

15. The training apparatus of claim 14, wherein the third loss function value determination module comprises a fourth subunit configured to: fuse the third generated image and the fourth generated image according to an image fusion algorithm to generate a fused image; compare the fused image with the third training image to generate a comparison result; determine a third loss function value according to the comparison result.

16. The training apparatus of any one of claims 11 to 15, wherein the second loss function comprises a generator loss function and a discriminator loss function, the second loss function value comprises a generator loss function value and a discriminator loss function value.

17. The training apparatus of claim 16, wherein the adjustment module comprises a fifth subunit configured to: adjust parameters of the first generation network of the first generative adversarial neural network and parameters of the second generation network of the second generative adversarial neural network according to the generator loss function value in the second loss function value; and adjust parameters of the first discriminator network of the first generative adversarial neural network and parameters of the second discriminator network of the second generative adversarial neural network according to the discriminator loss function value in the second loss function value.

18. An electronic device comprising: one or more processors; a storage device storing executable instructions that, when executed by the processor, implement the training method of any one of claims 1 to 9 or the image decomposition method of claim 10. ​ 19.A computer readable storage medium having stored thereon executable instructions that, when executed by a processor, implement the training method according to any one of claims 1 to 9, or the image decomposition method according to claim 10. 20.A computer program product comprising a computer program which, when executed by a processor, implements the training method according to any one of claims 1 to 9, or the image decomposition method according to claim 10.

Images