Smoke image generation method and device, equipment and storage medium

Abstract

The invention relates to the field of image processing, in particular to a smoke image generation method and device, equipment and a storage medium. The method comprises the following steps: obtaining a composite image through a generator; inputting the composite image and a preset real smoke image into more than two discriminators, and outputting a discrimination result of whether the image is the real smoke image through the discriminators; and updating parameters of the generator according to the accuracy rate of the judgment result, obtaining a composite image again according to the updated generator until the judgment result of the composite image and the real smoke image meets a preset requirement, and generating a smoke image according to the generator when the judgment result meets the preset requirement. More than two discriminators are used for discriminating, the probability of misjudgment can be effectively reduced, the learning efficiency of the generator is effectively improved, and the simulation degree of the synthesized smoke image is improved through a one-to-many and high-pressure game strategy, so that a smoke recognition model can recognize smoke more accurately.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a smoke image generation method, device, equipment and storage medium. Background Art

[0002] In the field of video surveillance, in order to accurately identify anomalies in the surveillance scene, it is necessary to obtain a large number of abnormal images to train the neural network model used to identify abnormal scenes. For example, when training a smoke recognition model, it is necessary to collect a large number of smoke images so that the trained smoke recognition model can effectively identify images containing smoke.

[0003] When acquiring smoke images, if the images are collected manually, the efficiency is low and it takes a long time. When smoke images are generated by a generative adversarial network including a generator and a discriminator, the image output efficiency can be effectively improved. Since the generative adversarial network is relatively simple, the generated smoke is often a pile of white blocks with low simulation. The smoke recognition model trained with low-simulation smoke images has low smoke recognition accuracy and is prone to false alarms. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a smoke image generation method, device, equipment and storage medium to solve the problem in the prior art that due to the relatively simple generative adversarial network, the generated smoke is often a pile of white blocks with low simulation degree, and the accuracy of training the smoke recognition model is not high.

[0005] A first aspect of an embodiment of the present application provides a method for generating a smoke image, the method comprising:

[0006] Obtain a synthetic image through the generator;

[0007] Inputting the synthesized image and a preset real smoke image into two or more discriminators, and outputting a discriminant result of whether it is a real smoke image through the discriminator;

[0008] The parameters of the generator are updated according to the accuracy of the discrimination result, and the synthetic image is re-obtained according to the updated generator, until the discrimination result of the synthetic image and the real smoke image meets the preset requirements, and the smoke image is generated according to the generator when the discrimination result meets the preset requirements.

[0009] In combination with the first aspect, in a first possible implementation manner of the first aspect, the generator includes at least one of a convolutional layer and a pooling layer;

[0010] Updating the parameters of the generator according to the accuracy of the discrimination result includes:

[0011] Update parameters in at least one of the convolution layer, the pooling layer, and the activation function according to the accuracy of the discrimination result.

[0012] In combination with the first aspect, in a second possible implementation manner of the first aspect, the discrimination result between the synthetic image and the real smoke image meets the preset requirements, including:

[0013] When the number of the synthetic images and the real smoke images input into the discriminator is the same, if the absolute value of the difference between the accuracy of the discrimination results of the synthetic image and the real smoke image and the preset accuracy is less than a preset difference threshold, it is determined that the synthetic image meets the preset requirements.

[0014] In combination with the second possible implementation manner of the first aspect, in a third possible implementation manner of the first aspect, when the number of the synthetic images input into the discriminator is the same as the number of the real smoke images, the method further includes:

[0015] If the absolute value of the difference between the accuracy of the discrimination results of the synthetic image and the real smoke image and the preset accuracy is greater than or equal to a preset difference threshold, the parameters of the generator are updated according to the accuracy.

[0016] In combination with the first aspect, in a fourth possible implementation manner of the first aspect, the synthetic image and the preset real smoke image are input into two or more discriminators, and the discriminator outputs a discrimination result of whether the image is a real smoke image, including:

[0017] Mixing the synthetic image and the real smoke image in a predetermined ratio to obtain an image to be identified;

[0018] Using two or more discriminators to discriminate each of the images to be discriminated, and obtaining two or more output results for each image;

[0019] Voting is performed based on more than two output results of each image to determine whether the image is a real smoke image.

[0020] In combination with the first aspect, in a fifth possible implementation of the first aspect, the discriminator includes at least two of a discriminator based on VGG16, a discriminator based on VGG19, a discriminator based on AlexNet, a discriminator based on ResNet, a discriminator based on DenseNet, a discriminator based on MobileNet, a discriminator based on GoogLeNet, a discriminator based on SqueezeNet, a discriminator based on ResNeXt, a discriminator based on Inception-ResNet, a discriminator based on Xception, and a discriminator based on U-Net.

[0021] In combination with the first aspect, in a sixth possible implementation manner of the first aspect, obtaining a synthetic image through a generator includes:

[0022] A random matrix that obeys Gaussian distribution is input into the generator, and the synthetic image is obtained through multiple rounds of convolution and pooling processing in the generator.

[0023] A second aspect of an embodiment of the present application provides a smoke image generating device, the device comprising:

[0024] A synthetic image acquisition unit, used for obtaining a synthetic image through a generator;

[0025] A discriminator, used for inputting the synthetic image and a preset real smoke image into two or more discriminators, and outputting a discriminant result of whether it is a real smoke image through the discriminator;

[0026] A generator updating unit is used to update the parameters of the generator according to the accuracy of the discrimination result, re-obtain the synthetic image according to the updated generator, until the discrimination result between the synthetic image and the real smoke image meets the preset requirements, and generate a smoke image according to the generator when the discrimination result meets the preset requirements.

[0027] A third aspect of an embodiment of the present application provides a smoke image generating device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method described in any one of the first aspects are implemented.

[0028] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of the first aspects are implemented.

[0029] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0030] Compared with the prior art, the embodiment of the present application has the following beneficial effects: when generating a smoke image, the embodiment of the present application outputs the discrimination results respectively through two or more discriminators, and updates the generator based on the accuracy of the discrimination results. Since more than two discriminators are used to distinguish between synthetic images and real smoke images, and multiple discriminators are calculated simultaneously, the probability of misjudgment can be effectively reduced, and the learning efficiency of the generator can be effectively improved. Through a one-to-many, high-pressure game strategy, the simulation degree of the synthetic smoke image is improved, which is conducive to the smoke recognition model to more accurately recognize smoke. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0032] Figure 1 This is a schematic diagram of the implementation process of a smoke image generation method provided in an embodiment of the present application;

[0033] Figure 2 It is a schematic diagram of a convolution process provided by an embodiment of the present application;

[0034] Figure 3 It is a schematic diagram of a pooling process provided in an embodiment of the present application;

[0035] Figure 4 is a schematic diagram of a generator optimization process provided by an embodiment of the present application;

[0036] Figure 5 is a schematic diagram of a smoke image generating device provided in an embodiment of the present application;

[0037] Figure 6 It is a schematic diagram of a smoke image generating device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0039] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0040] With the development of intelligent technology for workers, the application of artificial intelligence equipment is becoming more and more extensive. For example, in the field of monitoring, artificial intelligence technology can be used to automatically identify abnormal situations in monitoring scenes and automatically generate alarm information based on abnormal situations. In general, models that identify abnormal situations need to obtain a large number of samples for training. For example, when the model is used to identify whether there is smoke in the scene, a large number of smoke images need to be collected. Although smoke images can be generated through generative adversarial networks, due to the simplicity of generative adversarial networks, the smoke in the image is often a white block with a low degree of simulation, which is not conducive to improving the recognition accuracy of the model and is prone to false alarms.

[0041] Based on this, the present application embodiment proposes a smoke image generation method, such as Figure 1 As shown, the method includes:

[0042] In S101, a synthetic image is obtained by a generator.

[0043] The generator can receive a random matrix that obeys a Gaussian distribution so that the synthetic image generated by the generator can have better simulation.

[0044] The generator in the embodiment of the present application may include at least one of a convolutional layer and a pooling layer, and in a possible implementation, may also include an activation layer.

[0045] Among them, the convolution layer can be used to obtain effective local features from complex, high-dimensional images. The convolution layer includes a convolution kernel. The size of the convolution kernel and the value inside the convolution kernel can be transformed into different values ​​during the convolution calculation process, so that it can match different features. Through convolution calculation, the features in the image to be convolved are found. The generator can include multiple convolution layers. Through multi-layer convolution, the extracted features can be combined to generate images with specific features.

[0046] Figure 2 A convolution process diagram provided in an embodiment of the present application. Figure 2 As shown in the figure, the size of the convolution kernel used in the convolution process is 2*2, and the values ​​in the convolution kernel are 0, 1, 2, and 3 respectively. The values ​​in the 3*3 input matrix are 1, 2, 3, 4, 5, 6, 7, 8, and 9 respectively. In the convolution process, since the size of the convolution kernel is 2*2, the 4 elements in the upper left corner of the input matrix can be convolved first, that is, the four elements in the upper left corner are multiplied by the four elements in the convolution kernel respectively, and then the products are summed to obtain the value of the first element in the mapped 2*2 matrix, that is, the value of the first element in the mapped matrix is: 0*1+1*2+2*4+3*5=25.

[0047] In the same way, the sum of the products of the four elements in the upper right corner of the input matrix and the four elements in the convolution kernel is calculated, and the value of the element in the upper right corner of the mapping matrix is: 0*2+1*3+2*5+3*6=31. The sum of the products of the four elements in the lower left corner of the input matrix and the four elements in the convolution kernel is calculated, and the value of the element in the lower left corner of the mapping matrix is: 0*4+1*5+2*7+3*8=43. The sum of the products of the four elements in the lower right corner of the input matrix and the four elements in the convolution kernel is calculated, and the value of the element in the lower right corner of the mapping matrix is: 0*5+1*6+2*8+3*9=49.

[0048] Through convolution calculation, the main numerical features of the input matrix are retained. The size of the convolution kernel and the parameters in the convolution kernel can be iteratively updated according to iterative calculations, such as the feedback information of the discriminator, so that the generator can update the convolution parameters in the generator according to the discriminator's judgment results, including but not limited to the convolution kernel size, convolution step size, padding parameters, etc.

[0049] The pooling layer can be used to reduce redundant information in an image and output salient features. When performing pooling operations, operations such as maximum pooling, minimum pooling, and average pooling can be included. Figure 3 The figure shows a schematic diagram of the pooling process for feature extraction through maximum pooling. Figure 3 As shown in the figure, the size of the image to be pooled is 4*4, the size of the pooling kernel is 2*2, and a 2*2 mapping matrix is ​​obtained through pooling calculation. Since the pooling layer itself does not contain neurons, the number of parameters of the neural network can be greatly reduced, which reduces the amount of calculation of the neural network.

[0050] In a possible implementation, the generator may also include an activation layer to improve the generator's ability to handle nonlinear problems. This is because the convolutional layer and the pooling layer process data in a linear way. If there is no nonlinear activation layer, the network's ability to handle nonlinear problems will be very weak.

[0051] The activation layer can use the Relu function for activation calculation. The Relu function can be:

[0052] Or it can be expressed as RELU(x)=max(0,x).

[0053] like Figure 4 A schematic diagram of the network structure of a smoke image generation system provided in an embodiment of the present application. The generator may include multiple convolutional layers, or multiple convolutional layers and pooling layers. Figure 4 In the schematic diagram shown, the generator includes two convolution layers and one pooling layer. Not limited to this, the generator may also include an activation layer or a fully connected layer. The initial image is extracted through the convolution layer in the generator, such as extracting features from a randomly generated random matrix image that obeys a Gaussian distribution, and the features are combined to generate a smoke image.

[0054] In S102, the synthetic image and the preset real smoke image are input into two or more discriminators, and the discriminators output a discriminant result of whether the image is a real smoke image.

[0055] In the embodiment of the present application, a classifier can be used to determine whether an image is a real smoke image. The classifier includes more than two discriminators, and the more than two discriminators are different types of discriminators. For example, the more than two discriminators can be selected from a discriminator based on VGG16, a discriminator based on VGG19, a discriminator based on AlexNet, a discriminator based on ResNet, a discriminator based on DenseNet, a discriminator based on MobileNet, a discriminator based on GoogLeNet, a discriminator based on SqueezeNet, a discriminator based on ResNeXt, a discriminator based on Inception-ResNet, a discriminator based on Xception, and a discriminator based on U-Net.

[0056] In a possible implementation, in the same smoke image generation system, more than two discriminators may be discriminators of different types. For example, the system includes four discriminators, which may be a discriminator based on VGG16, a discriminator based on VGG19, a discriminator based on AlexNet, a discriminator based on ResNet, a discriminator based on DenseNet, a discriminator based on MobileNet, a discriminator based on GoogLeNet, a discriminator based on SqueezeNet, a discriminator based on ResNeXt, a discriminator based on Inception-ResNet, a discriminator based on Xception, a discriminator based on U-Net, etc.

[0057] In a possible implementation, the same type of neural network as a discriminator may include discriminators with different numbers of neural network nodes and different activation functions. That is, the same type of discriminator may include multiple different sub-types of discriminators. For example, the sub-discriminators included in the same type of discriminator may be determined based on the differences in network nodes and activation functions.

[0058] Different types of discriminators have different computational characteristics.

[0059] For example, the depth of VGG16 and VGG19 networks is deeper than that of AlexNet. The VGG network introduces convolution blocks, which combine multiple consecutive convolution layers together to increase the depth of the network. By using multiple consecutive small convolution kernels, the VGG network can achieve a larger receptive field in a deeper network and better understand the context of the image.

[0060] AlexNet is a relatively shallow network with 5 convolutional layers and 3 fully connected layers. ReLu can be used as the activation function to better solve the gradient diffusion problem when the network is deeper. AlexNet can prevent network overfitting and enhance the generalization ability of the model by using data enhancement, dropout and LRN layers.

[0061] ResNet can alleviate the performance degradation caused by network depth by introducing residual blocks. By learning the difference between input and output, it helps alleviate the problems of gradient vanishing and gradient exploding. Through long-distance residual connections, ResNet can effectively utilize the information between feature representations at different levels, thereby improving network performance.

[0062] The synthetic image and the real smoke image are input into multiple different types of discriminators, which can output the discrimination results of whether they are real smoke images respectively.

[0063] Whether an image is a real smoke image can be determined by voting, and the discrimination result with the largest number of votes can be selected as the discrimination result of the current image. For example, when the number of discriminators is 7, the discrimination results output by each discriminator can be obtained. If the number of votes for a discrimination result is greater than 3, the discrimination result is determined to be the discrimination result output by the classifier.

[0064] In a possible implementation, the accuracy of different types of discriminators in distinguishing smoke images can be counted, and the weight coefficients of different discriminators can be determined based on the statistical accuracy. According to the weight coefficients and the discrimination results of the discriminators, a weighted calculation is performed to obtain the discrimination result of whether the image is a real smoke image.

[0065] For example, the accuracy of the discriminator statistics can be used as a weight coefficient to calculate the weighted voting value of the image being a real smoke image and the weighted voting value of the image being a synthetic image, and the result with a higher weighted voting value is selected as the discrimination result of the classifier.

[0066] In S103, the parameters of the generator are updated according to the accuracy of the discrimination result, and the synthetic image is re-obtained according to the updated generator until the discrimination result between the synthetic image and the real smoke image meets the preset requirements, and the smoke image is generated according to the generator when the discrimination result meets the preset requirements.

[0067] Through the discrimination results output by the multiple discriminators, it can be determined whether it is a real smoke image by the number of votes or the weighted voting value. The classification results of the classifier including the multiple discriminators can be obtained by respectively judging the multiple real smoke images and the synthetic images input, that is, the discrimination results of whether it is a real smoke image.

[0068] In order to measure the performance of the classifier, the cross entropy loss function of the classifier can be used to evaluate the loss of the classifier's prediction results.

[0069] The cross loss function can be expressed as:

[0070]

[0071] Among them, x i Represents the category of the classifier output, including the category of real smoke images and synthetic images. p(x i ) indicates that the predicted category is x i The probability of q(x i ) indicates that the actual category is x i If the predicted result completely coincides with the actual result, the cross entropy loss is small. If the predicted result is highly correlated with the actual result, the cross entropy loss is large. The cross entropy loss function can effectively characterize the actual loss of the classifier.

[0072] The generator is used to output a high-simulation smoke image. The training goal of the generator is that the classifier cannot distinguish whether the image generated by the generator is a real smoke image or a synthetic image. When the ratio of the real smoke image to the synthetic image input to the classifier is a predetermined value, such as 1:1, that is, the number of real smoke images and synthetic images is the same, if the classifier cannot distinguish the real smoke image, the accuracy of the classifier is close to 50%.

[0073] An accuracy range can be set. If the accuracy of the classifier falls within the accuracy range, that is, the absolute value of the difference between the output accuracy and the preset accuracy is less than a predetermined value, it can be determined that the synthetic image meets the preset requirements, that is, the generator can generate a smoke image that meets the predetermined high-imitation requirements.

[0074] For example, the ratio of the real smoke image to the synthetic image input into the classifier is 1:1, and the preset accuracy is 50%. When the accuracy of the classifier is x, if |x-50%| is less than the preset range value, such as less than any value in the range of 3%-7%, such as less than 5%, it can be determined that the generator can generate a smoke image that meets the preset high-imitation requirements. If |x-50%| is greater than the preset range value, such as greater than or equal to any value in the range of 3%-7%, such as greater than or equal to 5%, the parameters in the generator can be reversely updated according to the accuracy of the classification result of the classifier, or according to the value of the cross entropy loss function of the classifier, including updating the parameters in at least one of the convolution layer, the pooling layer and the activation function, so that the simulation degree of the synthetic image output by the generator is gradually improved until the preset high-imitation requirements are met.

[0075] The embodiment of the present application uses multiple discriminators to form a generator, and obtains the classification result of the classifier by voting or weighted voting. Since the probability of multiple discriminators making errors at the same time is low, the recognition rate of smoke images can be effectively improved. A high-pressure game strategy is formed by multiple discriminators, which is conducive to improving the high imitation degree of the synthetic image.

[0076] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0077] Figure 5 A schematic diagram of a smoke image generating device provided in an embodiment of the present application, the device comprising:

[0078] A synthetic image acquisition unit 501, used to obtain a synthetic image through a generator;

[0079] A discriminator 502 is used to input the synthetic image and the preset real smoke image into two or more discriminators, and output a discriminator result of whether it is a real smoke image;

[0080] The generator updating unit 503 is used to update the parameters of the generator according to the accuracy of the discrimination result, re-obtain the synthetic image according to the updated generator, until the discrimination result between the synthetic image and the real smoke image meets the preset requirements, and generate a smoke image according to the generator when the discrimination result meets the preset requirements.

[0081] In a possible implementation, the generator includes at least one of a convolutional layer and a pooling layer, and the generator update unit is used to update parameters in at least one of the convolutional layer, the pooling layer and the activation function according to the accuracy of the discrimination result.

[0082] In a possible implementation, the discrimination results of the synthetic image and the real smoke image meet preset requirements, including: when the number of the synthetic images and the real smoke images input into the discriminator is the same, if the absolute value of the difference between the accuracy of the discrimination results of the synthetic image and the real smoke image and the preset accuracy is less than a preset difference threshold, it is determined that the synthetic image meets the preset requirements.

[0083] In a possible implementation, when the number of the synthetic images input into the discriminator is the same as the number of the real smoke images, the generator updating unit is used to:

[0084] If the absolute value of the difference between the accuracy of the discrimination results of the synthetic image and the real smoke image and the preset accuracy is greater than or equal to a preset difference threshold, the parameters of the generator are updated according to the accuracy.

[0085] In a possible implementation, the determination unit includes:

[0086] A mixing subunit, used for mixing the synthetic image and the real smoke image in a predetermined ratio to obtain an image to be identified;

[0087] A discriminator unit, configured to discriminate each of the images to be discriminated by using two or more discriminators to obtain two or more output results for each image;

[0088] The voting subunit is used to vote according to more than two output results of each image to determine whether the image is a real smoke image.

[0089] In a possible implementation, the discriminator includes at least two of a VGG16-based discriminator, a VGG19-based discriminator, an AlexNet-based discriminator, a ResNet-based discriminator, a DenseNet-based discriminator, a MobileNet-based discriminator, a GoogLeNet-based discriminator, a SqueezeNet-based discriminator, a ResNeXt-based discriminator, an Inception-ResNet-based discriminator, an Xception-based discriminator, and a U-Net-based discriminator.

[0090] In a possible implementation, the synthetic image acquisition unit is used to:

[0091] A random matrix that obeys Gaussian distribution is input into the generator, and the synthetic image is obtained through multiple rounds of convolution and pooling processing in the generator.

[0092] Figure 5 The smoke image generating device shown in Figure 1 The smoke image generation method shown corresponds to.

[0093] Figure 6 Schematic diagram of a smoke image generating device provided in an embodiment of the present application. Figure 6 As shown, the smoke image generating device 6 of this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60, such as a smoke image generating program. When the processor 60 executes the computer program 62, the steps in the above-mentioned smoke image generating method embodiments are implemented. Alternatively, when the processor 60 executes the computer program 62, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0094] Exemplarily, the computer program 62 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 62 in the smoke image generating device 6.

[0095] The smoke image generating device 6 can be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The smoke image generating device can include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art can understand that Figure 6 It is only an example of the smoke image generating device 6 and does not constitute a limitation of the smoke image generating device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the smoke image generating device may also include input and output devices, network access devices, buses, etc.

[0096] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0097] The memory 61 may be an internal storage unit of the smoke image generating device 6, such as a hard disk or memory of the smoke image generating device 6. The memory 61 may also be an external storage device of the smoke image generating device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the smoke image generating device 6. Further, the memory 61 may also include both an internal storage unit and an external storage device of the smoke image generating device 6. The memory 61 is used to store the computer program and other programs and data required by the smoke image generating device. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0098] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0099] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0100] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0101] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0102] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0103] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0104] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0105] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for generating a smoke image, characterized in that: The method comprises: Obtain a synthetic image through the generator; Inputting the synthesized image and a preset real smoke image into two or more discriminators, and outputting a discriminant result of whether it is a real smoke image through the discriminator; The parameters of the generator are updated according to the accuracy of the discrimination result, and the synthetic image is re-obtained according to the updated generator, until the discrimination result of the synthetic image and the real smoke image meets the preset requirements, and the smoke image is generated according to the generator when the discrimination result meets the preset requirements.

2. The method according to claim 1, characterized in that The generator includes at least one of a convolutional layer and a pooling layer; Updating the parameters of the generator according to the accuracy of the discrimination result includes: Update parameters in at least one of the convolution layer, the pooling layer, and the activation function according to the accuracy of the discrimination result.

3. The method according to claim 1, characterized in that: The discrimination results of the synthetic image and the real smoke image meet the preset requirements, including: When the number of the synthetic images and the real smoke images input into the discriminator is the same, if the absolute value of the difference between the accuracy of the discrimination results of the synthetic image and the real smoke image and the preset accuracy is less than a preset difference threshold, it is determined that the synthetic image meets the preset requirements.

4. The method according to claim 3, characterized in that When the number of the synthetic images input into the discriminator is the same as the number of the real smoke images, updating the parameters of the generator according to the accuracy of the discrimination result includes: If the absolute value of the difference between the accuracy of the discrimination results of the synthetic image and the real smoke image and the preset accuracy is greater than or equal to a preset difference threshold, the parameters of the generator are updated according to the accuracy.

5. The method according to claim 1, characterized in that The synthetic image and the preset real smoke image are input into two or more discriminators, and the discriminators output a discriminant result of whether the image is a real smoke image, including: Mixing the synthetic image and the real smoke image in a predetermined ratio to obtain an image to be identified; Using two or more discriminators to discriminate each of the images to be discriminated, and obtaining two or more output results for each image; Voting is performed based on more than two output results of each image to determine whether the image is a real smoke image.

6. The method according to claim 1, characterized in that The discriminator includes at least two of a discriminator based on VGG16, a discriminator based on VGG19, a discriminator based on AlexNet, a discriminator based on ResNet, a discriminator based on DenseNet, a discriminator based on MobileNet, a discriminator based on GoogLeNet, a discriminator based on SqueezeNet, a discriminator based on ResNeXt, a discriminator based on Inception-ResNet, a discriminator based on Xception, and a discriminator based on U-Net.

7. The method according to claim 1, characterized in that The synthetic image is obtained through the generator, including: A random matrix that obeys Gaussian distribution is input into the generator, and the synthetic image is obtained through multiple rounds of convolution and pooling processing in the generator.

8. A smoke image generating device, characterized in that: The device comprises: A synthetic image acquisition unit, used for obtaining a synthetic image through a generator; A discriminator, used for inputting the synthetic image and a preset real smoke image into two or more discriminators, and outputting a discriminant result of whether it is a real smoke image through the discriminator; A generator updating unit is used to update the parameters of the generator according to the accuracy of the discrimination result, re-obtain the synthetic image according to the updated generator, until the discrimination result between the synthetic image and the real smoke image meets the preset requirements, and generate a smoke image according to the generator when the discrimination result meets the preset requirements.

9. A smoke image generating device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images