A rotor DIC deformation measurement identification point recognition method and medium

Abstract

The present invention relates to a method and medium for identifying identification points of rotor DIC deformation measurement, and belongs to the technical field of rotor DIC deformation measurement. The method comprises: performing local enhanced attention processing on the image to be enhanced to obtain a first attention map, channel stacking and splicing the image to be enhanced and the first attention image to obtain a first feature map, and using the first feature map as the input of the image enhancement network; performing multi-layer downsampling and multi-layer upsampling on the first feature map to obtain a second feature map, performing element-by-element multiplication of the second feature map and the first attention map to obtain a third feature map, and performing element-by-element addition of the third feature map and the image to be enhanced to obtain an enhanced image; and identifying identification points based on the enhanced image. Compared with the prior art, the obtained enhanced image is easier to accurately identify the image of the deformed rotor after the identification point is enhanced.

Description

Technical Field

[0001] The invention relates to the technical field of rotor DIC deformation measurement, and in particular to a rotor DIC deformation measurement identification point recognition method and a medium. Background Art

[0002] Digital image correlation (DIC) technology is a non-contact full-field deformation measurement method and an important means to achieve three-dimensional dynamic deformation measurement of helicopter rotors. Through high-precision imaging systems and advanced image processing algorithms, DIC technology can provide detailed deformation data of the rotor in flight, reconstruct the three-dimensional surface of the blade before and after deformation, and then analyze the torsional stiffness, swing stiffness and flapping stiffness of the blade profile, which is crucial to understanding the starting performance and structural integrity of the rotor.

[0003] In DIC technology, circular marking points need to be pasted on the rotor surface, and then these marking points are extracted and matched. The extraction of these marking points relies on low-quality images such as pixel difference, low contrast, low light, and high noise. These low-quality images are difficult to distinguish due to texture details, and high noise will cause texture blur, so the difference between pixels is small, which has a great impact on the extraction of marking points and even the subsequent deformation measurement. Therefore, for the DIC deformation measurement algorithm based on feature point extraction, it is necessary to enhance the image quality collected in low-light environments.

[0004] There are two ways to improve the image quality. One is to change the physical parameters of the visual sensor during the imaging process to improve the sensor sensitivity, but it also brings greater imaging noise; the other is to enhance the image, which is also a common method. In recent decades, a large number of low-light image enhancement algorithms have emerged. Among them, the traditional methods are based on histogram equalization, including Global Histogram Equalization (GHE) and Local Histogram Equalization (LHE). In addition, linear transformation, piecewise linear transformation and nonlinear transformation methods are used to compress or stretch the image curve, which can enhance the image contrast and improve the image clarity. In addition, some people have proposed an enhancement method based on Retinex, which assumes that the image is composed of illumination and reflection, and the reflection components under different illumination conditions are consistent. The single-scale Retinex algorithm (SSR), the multi-scale Retinex algorithm (MSR), the Retinex algorithm based on total generalized variation, the Retinex model based on Poisson noise perception, the LIME algorithm and other algorithms have been proposed. These methods can solve these problems to a certain extent, but the images enhanced by traditional methods still have problems such as uneven exposure, blurred texture details, and noise amplification.

[0005] With the improvement of computer technology, the improvement of computer computing power has promoted the development of deep learning. Deep learning has also been applied in various image processing tasks and has shown its advantages. In some current technologies, deep learning is first applied to low-light image enhancement, and the LLNet algorithm is proposed. This algorithm achieves brightness enhancement and denoising by stacking sparse denoising autoencoders and training with synthetic data. In some technologies, Retinex-net is proposed on the basis of Retinex, including Decom-net for decomposition and Enhance-Net for illumination adjustment. Decom-Net decouples and splits the image to obtain an illumination map and a reflection map; Enhance-Net enhances the illumination map and multiplies the reflection map with the enhanced illumination map to obtain the enhanced image. In some technologies, additional prior information is introduced in the low-light image enhancement task. This method combines the semantic segmentation network with the Retinex method, and directly applies the semantic segmentation results to the reflection estimation network to indirectly achieve image enhancement effects. However, since these methods all avoid amplification artifacts by learning color changes and denoising, the enhanced image quality is not suitable for the extraction of marker points, making the recognition accuracy of marker points still low when used in rotor DIC deformation measurement. Summary of the invention

[0006] The technical problem to be solved by the present application is to provide a method and medium for identifying identification points of rotor DIC deformation measurement, which has the characteristic of being able to obtain identification points with higher accuracy in rotor DIC deformation measurement.

[0007] In a first aspect, an embodiment provides a method for identifying rotor DIC deformation measurement mark points, including:

[0008] Acquiring an image to be enhanced;

[0009] Perform local enhanced attention processing on the image to be enhanced to obtain a first attention map;

[0010] The image to be enhanced and the first attention image are stacked and concatenated to obtain a first feature map, and the first feature map is used as the input of the image enhancement network;

[0011] Performing multi-layer downsampling and multi-layer upsampling on the first feature map to obtain a second feature map;

[0012] Multiply the second feature map by the first attention map element by element to obtain a third feature map;

[0013] Adding the third feature map and the image to be enhanced element by element to obtain an enhanced image;

[0014] The landmarks are identified based on the enhanced image.

[0015] In one embodiment, the step of performing multi-layer downsampling and multi-layer upsampling on the first feature map to obtain the second feature map includes:

[0016] For multi-layer downsampling, the first feature map is used as the input of the first downsampling, and the first downsampling feature map is obtained after downsampling processing; except for the last layer of downsampling, for any layer of downsampling, the output of any layer of downsampling is used as the input of the next layer of downsampling;

[0017] For multi-layer upsampling, the output of the last downsampling layer is used as the input of the first upsampling layer for upsampling processing; except for the first upsampling layer, for any upsampling layer, the output of the previous upsampling layer and the output of the downsampling of the corresponding mirror layer are concatenated and used as the input of the upsampling layer; the output of the last upsampling layer is used as the second feature map;

[0018] Among them, the number of downsampling layers is equal to the number of upsampling layers.

[0019] In one embodiment, except for the first layer upsampling, for any layer upsampling, the output of the previous layer upsampling and the output of the mirrored corresponding layer downsampling are concatenated as the input of the any layer upsampling, including:

[0020] Perform local enhanced attention processing on the downsampled output of each layer to obtain the corresponding second attention map;

[0021] Except for the first layer upsampling, for any layer upsampling, the output of the previous layer upsampling and the second attention map corresponding to the mirrored layer downsampling are concatenated as the input of any layer upsampling;

[0022] For the first layer upsampling, the output of the mirrored corresponding layer downsampling is locally enhanced with the corresponding second attention map obtained as the input of the first layer upsampling.

[0023] In one embodiment, the local enhanced attention processing includes: converting the object to be locally enhanced attention processed into a grayscale image, and performing element-by-element subtraction of the grayscale image to obtain a result after the local enhanced attention processing.

[0024] In one embodiment, the step of performing element-by-element subtraction on the grayscale image includes:

[0025] Normalize the grayscale image to between 0 and 1, where 0 represents all black and 1 represents all white;

[0026] A full white image of the same size is used to perform element-by-element subtraction of the normalized grayscale image.

[0027] In one embodiment, the step of using a full white image of the same size to perform element-by-element subtraction on the normalized grayscale image includes:

[0028] ,

[0029] Among them, I represents the normalized grayscale image, 1 represents a completely white image, Represents the result of local enhanced attention processing.

[0030] In one embodiment, for any layer of downsampling, the process includes:

[0031] The input feature map is processed with two 3×3 convolutions in sequence, and then LeakyReLu activation and batch normalization are performed in sequence. The feature map after batch normalization is subjected to maximum pooling to obtain the output feature map.

[0032] In one embodiment, upsampling any one layer includes:

[0033] After bilinear upsampling of the input feature map, convolution is performed to obtain the output feature map.

[0034] In one embodiment, the learning and training of the image enhancement network includes: using a global-local discriminator, the global-local discriminator includes global interpolation and local interpolation, the local interpolation includes: randomly cropping local patches from the enhanced image and the real normal light image, and learning to distinguish the authenticity between the enhanced image and the real normal light image.

[0035] In a second aspect, an embodiment provides a computer-readable storage medium, wherein the medium stores a program, and the program can be loaded by a processor and executed by the rotor DIC deformation measurement identification point identification method as described in any of the above embodiments.

[0036] The beneficial effects of the present invention are:

[0037] On the one hand, it overcomes the problem that the current image enhancement technology cannot handle irrelevant parts or interference in the image well, and can highlight the details of the dark area in the low-light image. In the subsequent network, the detailed texture information of the key areas in the image is enhanced, so that the enhanced image is not overexposed or underexposed, which is more conducive to identifying the marking points on the deformed rotor. On the other hand, the obtained enhanced image is easier to accurately identify the image of the deformed rotor after the image enhancement of the marking point. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a schematic diagram of a method for identifying rotor DIC deformation measurement marking points according to an embodiment of the present application;

[0039] Figure 2 This application Figure 1 A method flow diagram of an embodiment of step S20;

[0040] Figure 3 This application Figure 2 A method flow diagram of an embodiment of step S202;

[0041] Figure 4 It is a schematic diagram of the structure of multi-layer downsampling and multi-layer upsampling according to an embodiment of the present application;

[0042] Figure 5 This is a flowchart of a method for obtaining input of any upsampling layer in addition to the first upsampling layer in an embodiment of the present application;

[0043] Figure 6 It is a flowchart of a method for performing local enhanced attention processing on the output of downsampling of any layer to obtain a corresponding second attention map according to an embodiment of the present application;

[0044] Figure 7 This application Figure 6 A method flow chart of an embodiment of step S1002 in FIG. DETAILED DESCRIPTION

[0045] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0046] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0047] The serial numbers assigned to the components in this article, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning.

[0048] To facilitate the description of the inventive concept of the present application, the image enhancement technology is briefly described below.

[0049] In order to further improve the image quality and identify the object to be identified more accurately, the current image enhancement technology will enhance the image based on attention enhancement. In some methods, the feature enhancement module based on residual self-attention is embedded in the bottom of the U-NET downsampling path. The high-power undersampled image is input through the U-NET downsampling path to extract the high-level feature map for feature enhancement, and then decoded through the U-NET upsampling path and merged with the feature map corresponding to the downsampling path. The features of the downsampling layer at the corresponding level during upsampling are fused to supplement the missing boundary information, accurately predict the edge information, and obtain the reconstructed image.

[0050] However, the applicant found in the research that the rotor deformation image with identification points needs to handle the irrelevant parts or interference on the image well. In the above-mentioned image enhancement method, the entire undersampled image is input into the network. Although image enhancement can be achieved, it cannot handle the irrelevant parts or interference on the image well, which is a technical difficulty. In addition, in the above-mentioned image enhancement method, feature enhancement is only performed at the bottom of the U-NET downsampling path, which cannot well enhance the important feature parts of each scale of the downsampling. For the image data of measuring the rotor DIC deformation, another technical difficulty is that what needs to be enhanced is the identification points on the deformed rotor.

[0051] In view of this, the rotor DIC deformation measurement identification point recognition method and medium of the present application adopt an end-to-end learning model. In the method, the image to be enhanced is collected, the image to be enhanced is locally enhanced attention processed to obtain a first attention map, and the image to be enhanced and the first attention image are channel stacked and spliced ​​to obtain a first feature map, and the first feature map is used as the input of the image enhancement network to provide a rich feature set for the entire image enhancement network, thereby overcoming the problem that the current image enhancement technology cannot handle irrelevant parts or interference on the image well. The first feature map is subjected to multi-layer downsampling and multi-layer upsampling to obtain a second feature map, the second feature map is multiplied element by element with the first feature map to obtain a third feature map, and then the third feature map and the image to be enhanced are added element by element to obtain an enhanced image, so that important feature information is further highlighted, thereby highlighting the texture of the original image, and obtaining an image of the deformed rotor after image enhancement that is easier to accurately identify the identification point, and the identification of the identification point is realized based on the image-enhanced deformed rotor image.

[0052] In one embodiment of the present application, a rotor DIC deformation measurement identification point recognition method is provided. Please refer to Figure 1 ,include:

[0053] Step S10: collecting the image to be enhanced.

[0054] In one implementation, the image to be enhanced is preprocessed first, and then the preprocessed image to be enhanced is used as the input of the image enhancement network. For the specific preprocessing method, those skilled in the art can set the preprocessing process based on the needs.

[0055] Step S20, performing local enhanced attention processing on the image to be enhanced to obtain a first attention map.

[0056] The applicant found in the research that we always want to enhance the dark area more than the bright area, so that the output image is neither overexposed nor underexposed, and the enhanced image can effectively extract the identification points of the dark area. Figure 2 , step S20 may include:

[0057] Step S201, converting the image to be enhanced into a grayscale image.

[0058] By processing the grayscale image, we can obtain comprehensive pixel-level information of the original image in the form of brightness.

[0059] Step S202, subtract the grayscale image element by element to obtain the first attention map.

[0060] In one embodiment, please refer to Figure 3 , step S202 may include:

[0061] Step S2021, normalize the grayscale image to between 0 and 1, where 0 represents all black and 1 represents all white.

[0062] Step S2022: Use a full white image of the same size to perform element-by-element subtraction on the normalized grayscale image.

[0063] In one embodiment, step S2022 can be expressed as:

[0064] ,

[0065] Among them, I represents the normalized grayscale image, 1 represents a completely white image, Represents the result of local enhanced attention processing.

[0066] In step S2022, the result after local enhanced attention processing is obtained That is the first attention map.

[0067] The first attention map is obtained by performing element-by-element subtraction of the grayscale image. This makes it possible to highlight the detail information of the dark area in the low-light image. Since the key area in the image is highlighted, this part of the detail texture information is enhanced in the subsequent network, so that the enhanced image is not overexposed or underexposed, which can be more conducive to identifying the marking points on the deformed rotor.

[0068] Step S30: Channel stacking and splicing of the image to be enhanced and the first attention image to obtain a first feature map, and using the first feature map as an input of the image enhancement network.

[0069] Since the first feature map obtained after splicing highlights the key areas in the image, using the first feature map as the input of the image enhancement network provides a rich feature set for the entire image enhancement network, thereby overcoming the problem that the current image enhancement technology cannot handle irrelevant parts or interference in the image well.

[0070] Step S40, performing multi-layer downsampling and multi-layer upsampling on the first feature map to obtain a second feature map.

[0071] In one embodiment, please refer to Figure 4 In step S40, for multi-layer downsampling, the first feature map is used as the input of the first downsampling, and the first downsampling feature map is obtained after downsampling processing; except for the last layer of downsampling, for any layer of downsampling, the output of the downsampling of any layer is used as the input of the next layer of downsampling. For multi-layer upsampling, the output of the last layer of downsampling is used as the input of the first layer of upsampling for upsampling processing; except for the first layer of upsampling, for any layer of upsampling, the output of the upsampling of the previous layer and the output of the downsampling of the corresponding layer of the mirror image are spliced ​​as the input of the upsampling of any layer; the output of the upsampling of the last layer is used as the second feature map. Among them, the number of downsampling layers is equal to the number of upsampling layers.

[0072] Figure 4 In the embodiment, the multi-layer downsampling includes five layers, namely, the first downsampling, the second downsampling, the third downsampling, the fourth downsampling and the fifth downsampling, and the multi-layer upsampling includes five layers, namely, the first upsampling, the second upsampling, the third upsampling, the fourth upsampling and the fifth upsampling. The first downsampling corresponds to the fifth upsampling in mirror image, the second downsampling corresponds to the fourth upsampling in mirror image, the third downsampling corresponds to the third upsampling in mirror image, the fourth downsampling corresponds to the second upsampling in mirror image, and the fifth downsampling corresponds to the first upsampling in mirror image.

[0073] The first feature map is used as the input of the first downsampling, and the output of the fifth upsampling is used as the second feature map.

[0074] The applicant also found in the research that because each downsampling process will produce features of different scales, but also lose some detail information during downsampling. In view of this, in one implementation, please refer to Figure 4 and Figure 5 , except for the first layer upsampling, for any layer upsampling, the output of the previous layer upsampling and the output of the mirrored corresponding layer downsampling are concatenated as the input of any layer upsampling, including:

[0075] Step S100, the output of each downsampled layer is locally enhanced to obtain the corresponding second attention map.

[0076] In one embodiment, the output of any downsampled layer is locally enhanced to obtain the corresponding second attention map, please refer to Figure 6 , which may include:

[0077] Step S1001, converting the down-sampled output of any layer into a grayscale image.

[0078] By processing the grayscale image, comprehensive pixel-level information in the form of brightness can be obtained for the down-sampled output of any layer.

[0079] Step S1002, subtract the grayscale image element by element to obtain the corresponding second attention map.

[0080] In one embodiment, please refer to Figure 7 , step S1002 may include:

[0081] Step S10021, normalize the grayscale image to between 0 and 1, where 0 represents all black and 1 represents all white.

[0082] In step S10022, a full white image of the same size is used to perform element-by-element subtraction on the normalized grayscale image to obtain a second attention map.

[0083] In one embodiment, step S10022 can be expressed as:

[0084] ,

[0085] Among them, I represents the normalized grayscale image, 1 represents a completely white image, Represents the result of local enhanced attention processing.

[0086] In step S10022, the result after local enhanced attention processing is obtained That is the second attention map.

[0087] For the first layer upsampling, the corresponding second attention map obtained by locally strengthening the attention process on the output of the mirrored corresponding layer downsampling is used as the input of the first layer upsampling.

[0088] The second attention map is obtained by element-by-element subtraction of the grayscale image, so that the detail information of the dark area in each downsampled result image can be highlighted. Since the key areas in the image are highlighted, it is more conducive to identifying the identification points on the deformed rotor. On the one hand, the key areas in the image are further highlighted by combining the first feature map based on the first attention map input. On the other hand, since each second attention map is calculated and fused with the feature map sampled at the corresponding level, and used as the input feature map for each upsampling, this multi-scale fusion method not only retains the benefits of feature fusion, but also uses the attention mechanism to further enhance the feature details of different scales, thereby further enhancing the detailed texture information of the identification points on the deformed rotor.

[0089] Step S200, except for the first layer upsampling, for any layer upsampling, the output of the previous layer upsampling and the second attention map corresponding to the mirrored layer downsampling are concatenated as the input of the any layer upsampling.

[0090] Through step S100 to step S200, attention is also calculated for the features generated by downsampling to obtain texture features of different scales. These features calculated accordingly are integrated in each upsampling process, so that the features in the downsampling process are not lost, and feature maps of different scales can be enhanced. Our attention map is a form of self-regularization, rather than learning under conventional supervision, so it is easy to use and the network calculation parameters are not large. Despite its simplicity, attention guidance can improve the visibility of feature points and significantly improve the feature point extraction effect of the enhanced image.

[0091] In one embodiment, in the downsampling stage, for any layer of downsampling, it includes: performing two 3×3 convolution processes on the input feature map in sequence, then performing LeakyReLu activation and batch normalization processes in sequence, and performing maximum pooling on the feature map after batch normalization to obtain the output feature map.

[0092] The applicant also found in the study that in the upsampling stage, due to the possibility that the convolution kernel cannot be compensated and divided, if the labeled deconvolution is used for upsampling, it may cause uneven overlap of the output results and produce checkerboard artifacts. In view of this, in one embodiment of the present application, for any layer of upsampling, it includes: performing bilinear upsampling processing on the input feature map, and then performing convolution processing to obtain the output feature map. In this way, a bilinear upsampling layer and a convolution layer are used to replace the standard deconvolution layer to reduce the checkerboard artifacts.

[0093] Step S50, multiply the second feature map by the first attention map element by element to obtain a third feature map.

[0094] In step S50, the first attention map is multiplied element-by-element with the second feature map, that is, the original attention map is multiplied element-by-element with the second feature map, so that the high-value area in the original attention map will enhance the corresponding features in the second feature map, while the low-value area will suppress the features, thereby further highlighting the important feature information required and obtaining the high-level features after network enhancement.

[0095] Step S60, adding the third feature map and the image to be enhanced element by element to obtain an enhanced image.

[0096] In step S60, the third feature map is added to the image to be enhanced element by element, that is, the third feature map is added to the input original low-light image element by element. In this way, the detailed information of the original low-light image and the high-level features extracted by the network can be integrated to highlight the important texture of the original image and obtain an enhanced result.

[0097] Step S70, identifying the marking points based on the enhanced image.

[0098] The rotor DIC deformation measurement mark point recognition method in any of the above embodiments, on the one hand, overcomes the problem that the current image enhancement technology cannot handle irrelevant parts or interference in the image well, and can highlight the detail information of the dark area in the low-light image, and focus on enhancing the detail texture information of the key area in the image in the subsequent network, so that the enhanced image is not overexposed or underexposed, which is more conducive to identifying the mark points on the deformed rotor. On the other hand, the obtained enhanced image is easier to accurately identify the image of the deformed rotor after the image enhancement of the mark point.

[0099] The image enhancement network in the embodiment of the present application is an end-to-end learning network. In one embodiment, in the training and learning of the image enhancement network, the image enhancement result and the real image are input into the global-local discriminator loss model together to obtain the loss of the generated network. In addition, the image enhancement result and the original input low-light image (i.e., the collected object to be enhanced) are input into the imageNet pre-trained VGG model to obtain the self-feature preservation loss. All losses are summed to correctly guide the training of the generated network guided by attention. The above process is repeated, and after a specified number of iterations, an image enhancement network for rotor image enhancement is obtained.

[0100] For the global-local discriminator, in one embodiment, an adversarial loss is used to minimize the distance between the true and output normal light distributions. However, traditional image-level enhancers often fail on spatially varying images. If the input image has local regions that need to be enhanced differently from other parts, such as a small bright region in an overall dark background, a global image alone usually cannot provide the desired adaptability.

[0101] In view of this, in one embodiment of the present application, in addition to improving the brightness globally, in order to adaptively enhance the local area, a new global-local hybrid structure is proposed, in which global interpolation is combined with PatchGAN to distinguish true / false, and local interpolation is added. The local interpolation includes: randomly cropping local patches from the image enhancement result and the real normal light image, and learning to distinguish the true and false between the image enhancement result and the real normal light image, so that the global-local structure ensures that all local patches of the enhanced image look like real normal light.

[0102] In one embodiment, for the global discriminator, a relativistic discriminator structure is used to estimate the probability that real data is more realistic than fake data, and guide the generator to synthesize fake images that are more realistic than real data.

[0103] In one implementation, the relative discriminator can be expressed as:

[0104] (1)

[0105] (2)

[0106] in, represents the global samples collected from the real image data distribution, represents the global samples collected from the fake image data distribution, represents the probability that the real image data is more real than the fake image data, represents the probability that the fake image data is more real than the real image data, Represents the sigmoid function; represents the discriminator network, which is used to evaluate the authenticity of the input sample. Indicates the authenticity of the real sample. Indicates the degree of authenticity of the fake sample; Indicates the distribution of fake image data The expectation of the evaluation value of the discriminator network for fake samples. Indicates the distribution of real image data The expected value of the discriminator network for the true sample.

[0107] Through formula (1), we first calculate the true and false evaluation value of the real global sample, and then consider the distribution of fake image data through the expected value E The average evaluation value of the false samples is obtained, and then the two are subtracted to get the evaluation difference, and finally a probability value is output through the sigmoid function.

[0108] Through formula (2), we first calculate the true and false evaluation value of the global samples collected in the fake image data distribution, and then consider the real image data distribution through the expected value E The average evaluation value of the real sample is obtained, and then the two are subtracted to get the evaluation difference, and finally a probability value is output through the sigmoid function.

[0109] In one embodiment, for the global discriminator, the least squares GAN (LSGAN) loss is used instead of the sigmoid function. The loss functions of the global discriminator and the global generator can be expressed as:

[0110] (3)

[0111] (4)

[0112] in, represents the loss function of the global discriminator, represents the loss function of the global generator, A, B, C and D are all intermediate variables; , represents the expected value of the square of the difference between the probability of the discriminator output and 1 (indicating complete truth) under real samples. This part encourages the discriminator to output real samples close to 1. , represents the expected value of the square of the probability output by the discriminator under false samples, which encourages the discriminator to output false samples close to 0. , represents the expected value of the square of the difference between the probability of the discriminator output and 1 under fake samples. This part encourages the fake samples generated by the generator to deceive the discriminator and make its output close to 1. , represents the expected value of the square of the probability output by the discriminator under real samples. This part encourages the generator to distinguish between fake samples and real samples, making the output of the discriminator for real samples close to 0.

[0113] The design of the global discriminator in the above embodiment enables the generator and the discriminator to compete with each other in global adversarial training, the generator strives to generate more realistic fake data, and the discriminator strives to improve its ability to distinguish between true and false data.

[0114] In one embodiment, for the local discriminator, 5 patches are randomly cropped from the output image and the real image each time. Here, the original LSGAN can be used as the adversarial loss. Therefore, the loss function of the local discriminator and the local generator can be expressed as:

[0115] (5)

[0116] (6)

[0117] in, represents the loss function of the local discriminator, represents the loss function of the local generator, and An intermediate amount. , which means that in the local real sample Under this condition, the expected value of the square of the difference between the probability output by the local discriminator and 1, which partially encourages the discriminator to output local true samples close to 1. represents the output probability of the local discriminator given a local true sample, and the entire probability represents the probability that the local true sample is more real than the local fake sample. Represents the local real data distribution The expected value of the loss function under . , represents the expected value of the square of the difference between the probability of the local discriminator output and 0 (indicating complete falsehood) under local false samples. This part encourages the local discriminator to output local false samples close to 0, where, Represents the local discriminator given a local fake sample The output probability when Represents local false data distribution The expected value of the loss function under . Indicates that in the local false sample Under this condition, the expected value of the square of the difference between the probability output by the local discriminator and 1, which partially encourages the local fake samples of the local generator to deceive the local discriminator and make its output close to 1.

[0118] Based on the design of the local discriminator and the local generator in the above embodiment, the local generator and the local discriminator compete with each other in local adversarial training, the local generator strives to generate more realistic local false data, and the local discriminator strives to improve its ability to distinguish between local true and false data.

[0119] In the embodiment of the present application, we use an end-to-end learned attention map to guide the generation of enhanced pictures in order to maintain the features and details of the original image. In order to constrain the perceived similarity, a pre-trained VGG can be used to simulate the feature space distance between images, which is achieved by limiting the distance between the extracted features in the output image and the original image. In the embodiment of the present application, when adjusting the image brightness, the VGG model is not very sensitive to these changes, so it is necessary to limit the VGG feature distance between the input low light and its enhanced normal light output, that is, the self-feature preservation loss, to emphasize the attention function, that is, the ability to retain the image content features to itself before and after enhancement. The purpose of this strategy is to make the enhanced image as consistent as possible with the original image at the feature level, thereby maintaining the essential characteristics of the image.

[0120] In one embodiment, the self-feature preserving loss function It can be expressed as:

[0121] (7)

[0122] in, represents the input low-light image, represents the enhanced output of the generator, Represents the feature map extracted from the VGG-16 model pre-trained on ImageNet. i represents the i-th maximum pooling, and j represents the j-th convolutional layer after the i-th maximum pooling. and The distribution represents the width and height of the extracted feature map. By default, i=5, j=1 can be selected. Used to measure the input low-light image And the enhanced output of the generator Differences at the feature level.

[0123] For local end-to-end learning, cropped local patches from the input and output images are also passed through a similarly defined local self-feature preserving loss End-to-end learning is performed to ensure that the generated local regions are similar to the local regions of the real image at the feature level. Similarly, global end-to-end learning can be achieved by defining a global self-feature preservation loss This ensures that the global features of the generated image are consistent with the real image.

[0124] From this, we can get the total loss function for training the image enhancement network, which can be expressed as:

[0125] (8)

[0126] in, Represents the total loss function.

[0127] In one embodiment of the present application, a computer-readable storage medium is provided, on which a program is stored. The stored program includes a method that can be loaded by a processor and process any one of the above embodiments.

[0128] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above-mentioned embodiments can be implemented by hardware or by computer program. When all or part of the functions in the above-mentioned embodiments are implemented by computer program, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, CD, hard disk, etc., and the program is executed by a computer to implement the above-mentioned functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above-mentioned functions can be implemented. In addition, when all or part of the functions in the above-mentioned embodiments are implemented by computer program, the program can also be stored in a storage medium such as a server, another computer, disk, CD, flash disk or mobile hard disk, and can be downloaded or copied and saved in the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0129] The above specific examples are used to illustrate the present invention, which are only used to help understand the present invention and are not used to limit the present invention. For those skilled in the art to which the present invention belongs, some simple deductions, modifications or substitutions can be made according to the idea of ​​the present invention.

Claims

1. A rotor DIC deformation measurement identification point recognition method, characterized in that: include: Acquiring an image to be enhanced; Perform local enhanced attention processing on the image to be enhanced to obtain a first attention map; The image to be enhanced and the first attention image are stacked and concatenated to obtain a first feature map, and the first feature map is used as the input of the image enhancement network; Performing multi-layer downsampling and multi-layer upsampling on the first feature map to obtain a second feature map; Multiply the second feature map by the first attention map element by element to obtain a third feature map; Adding the third feature map and the image to be enhanced element by element to obtain an enhanced image; Identify the landmarks based on the enhanced image; The local enhanced attention processing includes: converting the object to be processed with local enhanced attention into a grayscale image, and performing element-by-element subtraction on the grayscale image to obtain a result after the local enhanced attention processing; The step of performing element-by-element subtraction on the grayscale image includes: Normalize the grayscale image to between 0 and 1, where 0 represents all black and 1 represents all white; Use a full white image of the same size to perform element-by-element subtraction on the normalized grayscale image; The step of using a full white image of the same size to perform element-by-element subtraction on the normalized grayscale image includes: ； Among them, I represents the normalized grayscale image, 1 represents a completely white image, Represents the result of local enhanced attention processing.

2. The rotor DIC deformation measurement identification point identification method according to claim 1, characterized in that: The step of performing multi-layer downsampling and multi-layer upsampling on the first feature map to obtain the second feature map includes: For multi-layer downsampling, the first feature map is used as the input of the first downsampling, and the first downsampling feature map is obtained after downsampling processing; except for the last layer of downsampling, for any layer of downsampling, the output of any layer of downsampling is used as the input of the next layer of downsampling; For multi-layer upsampling, the output of the last downsampling layer is used as the input of the first upsampling layer for upsampling processing; except for the first upsampling layer, for any upsampling layer, the output of the previous upsampling layer and the output of the downsampling of the corresponding mirror layer are concatenated and used as the input of the upsampling layer; the output of the last upsampling layer is used as the second feature map; Among them, the number of downsampling layers is equal to the number of upsampling layers.

3. The rotor DIC deformation measurement identification point identification method according to claim 2, characterized in that: In addition to the first layer upsampling, for any layer upsampling, the output of the previous layer upsampling and the output of the mirrored corresponding layer downsampling are concatenated as the input of the any layer upsampling, including: Perform local enhanced attention processing on the downsampled output of each layer to obtain the corresponding second attention map; Except for the first layer upsampling, for any layer upsampling, the output of the previous layer upsampling and the second attention map corresponding to the mirrored layer downsampling are concatenated as the input of any layer upsampling; For the first layer upsampling, the output of the mirrored corresponding layer downsampling is locally enhanced with the corresponding second attention map obtained as the input of the first layer upsampling.

4. The rotor DIC deformation measurement identification point identification method according to claim 2 or 3, characterized in that: For any layer of downsampling, including: The input feature map is processed with two 3×3 convolutions in sequence, and then LeakyReLu activation and batch normalization are performed in sequence. The feature map after batch normalization is subjected to maximum pooling to obtain the output feature map.

5. The rotor DIC deformation measurement identification point identification method according to claim 2 or 3, characterized in that: For any upsampling layer, including: After bilinear upsampling of the input feature map, convolution is performed to obtain the output feature map.

6. The rotor DIC deformation measurement identification point identification method according to claim 1, characterized in that: The learning and training of the image enhancement network includes: using a global-local discriminator, the global-local discriminator includes global interpolation and local interpolation, the local interpolation includes: randomly cropping local patches from an enhanced image and a real normal light image, and learning to distinguish between the true and false between the enhanced image and the real normal light image.

7. A computer-readable storage medium, characterized in that: The medium stores a program, which can be loaded by a processor and execute the rotor DIC deformation measurement identification point recognition method according to any one of claims 1 to 6.

Images