Ultrasonic Phased Array Image Optimization and Reconstruction Method and System Based on Semi-Supervised CycleGan Network

Abstract

The present invention discloses an ultrasonic phased array image optimization and reconstruction method and system based on a semi-supervised CycleGan network, which generates training samples that correspond one by one in two domains of an ultrasonic image and a reconstructed image; superimposes an adversarial loss function, a cyclic consistency loss function, an individual loss function, and a true difference loss function to obtain a loss function; constructs a semi-supervised CycleGan network structure based on the loss function, and uses the training samples to train the semi-supervised CycleGan network structure. After the network training is completed, the ultrasonic image actually detected by the ultrasonic phased array is input into the trained semi-supervised CycleGan network structure to obtain a reconstructed image of the detected defect. The method of the present invention can achieve cross-domain reconstruction of small target images without cropping the original image; for ultrasonic images, it not only retains the position information of the defects in the image, but also eliminates the defect artifacts in the image, making the morphology of the defects more accurate.

Description

Technical Field

[0001] The present invention belongs to the technical field of industrial ultrasonic nondestructive testing, and particularly relates to an ultrasonic phased array image optimization and reconstruction method and system based on a semi-supervised CycleGan network. Background Art

[0002] Ultrasonic and ultrasonic phased array testing are important means for realizing industrial nondestructive testing technology. The test results will visually display the interior of the object to be tested in the form of a two-dimensional image, mainly involving features such as the location and geometric morphology of defects. Currently, in order to accurately quantitatively characterize the internal defects of the object to be tested, it is necessary to improve the hardware performance of the equipment, which will incur huge hardware R & D costs. Moreover, due to the scattering effect of the echo signal, there are artifacts in the defect morphology of the imaged image, specifically manifested as blurred defect edges and distorted defect characterization. With the development of the field of deep learning, relevant network models have gradually been transplanted into ultrasonic phased arrays. Researchers hope to use image processing methods in deep learning to optimize and reconstruct ultrasonic phased array images to remove artifacts.

[0003] Currently, in the field of medical ultrasonic phased array imaging, a method for ultrasonic image reconstruction with the help of deep learning has been realized. However, compared with industrial ultrasonic phased arrays, the image content is rich and highly correlated. In industrial testing, the proportion of defects in the object to be tested is relatively small. This results in the image content being mostly background (represented in black), and the characterized defects have little correlation with the image background.

[0004] The existing method is to optimize the performance of the discriminator in the CycleGan network by comparing the features of the input image and the reconstructed image. However, for ultrasonic defect images, after multiple layers of convolution, the defect information will be lost to a certain extent. Therefore, this method does not improve the performance of the discriminator much in ultrasonic image reconstruction.

[0005] How to accurately reconstruct the two-dimensional image formed by the ultrasonic phased array (hereinafter referred to as the ultrasonic image) into a two-dimensional image of the defect (hereinafter referred to as the reconstructed image) in industrial testing is an urgent problem to be solved. Through research, it is found that relevant methods cannot be directly transplanted into industrial ultrasonic phased array image reconstruction. First, in industrial testing, the distribution and morphology of internal defects of the object are often random, and supervised deep learning networks cannot meet the actual application requirements of the open set in industrial testing. Second, although increasing the proportion of defects in the image by cropping can enable the network to learn image features, this not only requires a huge cost for image preprocessing but also loses the important characterization attribute of the ultrasonic image for the defect location. None of the above methods can guarantee the accuracy of the reconstructed image information. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method and system for optimizing and reconstructing ultrasonic phased array images based on a semi-supervised CycleGan network to solve the technical problem of high-precision defect reconstruction of ultrasonic phased array detection images in view of the above-mentioned deficiencies in the prior art.

[0007] The present invention adopts the following technical solutions:

[0008] The method for optimizing and reconstructing ultrasonic phased array images based on a semi-supervised CycleGan network includes the following steps:

[0009] S1. Generate training samples that correspond one by one in two domains of ultrasonic images and reconstructed images;

[0010] S2. Superimpose the adversarial loss function, the cycle consistency loss function, the individual loss function, and the true difference loss function to obtain the loss function;

[0011] S3. Build a semi-supervised CycleGan network structure based on the loss function obtained in step S2, and use the training samples generated in step S1 to train the semi-supervised CycleGan network structure. After the network training is completed, input the ultrasonic images actually detected by the ultrasonic phased array into the trained semi-supervised CycleGan network structure to obtain the reconstructed images of the detected defects.

[0012] Specifically, in step S1, ultrasonic imaging is performed on the artificially designed defects in a two-dimensional plane, and the obtained ultrasonic images of the defects and the original defect images form the training samples; or the test blocks processed with accurate defect information are actually measured and the corresponding reconstruction calculations are performed, and the defect images of the test blocks and the corresponding ultrasonic images are respectively obtained from the processing information to form the training samples.

[0013] Specifically, in step S2, the loss function is specifically:

[0014]

[0015] where λ cyc , λ idt and λ aut are adjustable hyperparameters respectively, and are the adversarial loss functions, is the cycle consistency loss function, is the individual loss function, is the true difference loss function.

[0016] Furthermore, the adversarial loss functions and are respectively:

[0017]

[0018] Among them, GA(a) represents the image of domain B output after the sample a (a ~ P A ) of domain A passes through the generator GA GB(b) represents the image of domain A output after the sample b (b ~ P B ) of domain B passes through the generator GB DA(b) represents the class score of the discriminator DA for the image b, and DB(a) represents the class score of the discriminator DB for the image a. represents that the image a follows the probability distribution of domain A, represents that the image b follows the probability distribution of domain B.

[0019] Furthermore, the cycle consistency loss function is:

[0020]

[0021] Among them, GA(a) represents the image of domain B output after the sample a (a ~ P A ) of domain A passes through the generator GA GB(b) represents the image of domain A output after the sample b (b ~ P B ) of domain B passes through the generator GB represents that the image a follows the probability distribution of domain A, represents that the image b follows the probability distribution of domain B.

[0022] Furthermore, the individual loss function is:

[0023]

[0024] Among them, GA(a) represents the image of domain B output after the sample a (a ~ P A ) of domain A passes through the generator GA GB(b) represents the image of domain A output after the sample b (b ~ P B ) of domain B passes through the generator GB represents that the image a follows the probability distribution of domain A, represents that the image b follows the probability distribution of domain B.

[0025] Furthermore, the real difference loss function is:

[0026]

[0027] Among them, GA(a) represents the sample a (a ~ P AThe image of domain B output after passing through the generator GA GB(b) represents the sample b of domain B (b ~ P B ) The image of domain A output after passing through the generator GB Indicates that the image a follows the probability distribution of domain A, Indicates that the image b follows the probability distribution of domain B, and MSE is the mean square error loss function.

[0028] Specifically, in step S3, the training process of the semi - supervised CycleGan network is as follows:

[0029] The unpaired images of the ultrasonic image and the defect image are input into two different generators from both sides simultaneously, and the corresponding images conforming to the other domain are output simultaneously. The discriminator discriminates the images output by the generator;

[0030] Taking the ultrasonic image a as the input side, in phase, the image a distributed in domain A passes through the generator GA and outputs its corresponding defect image distributed in domain B On the one hand, the similarity between the defect image and domain B is evaluated by the adversarial loss function of the discriminator DA ; on the other hand, through the in the real difference loss function, the difference degree between the defect image generated by GA and the real defect image b corresponding to the ultrasonic image a is quantified;

[0031] Subsequently, in phase, the defect image of domain B input into the generator GB outputs its corresponding ultrasonic image of domain A and the domain similarity is compared with the input image a of the network through the of the cycle consistency loss function;

[0032] Taking the defect image b as the input side, it successively experiences and two phases, and is respectively constrained by the adversarial loss function of the discriminator DB the real difference loss function and the cycle consistency loss function .

[0033] After the two - way output of a batch of images is completed, the semi - supervised CycleGan optimizes and adjusts the network parameters according to the various loss function terms calculated above. When the training is completed, the semi - supervised CycleGan network realizes the transformation from the ultrasonic image to the defect image, and the transformation from the defect image to the ultrasonic image.

[0034] Furthermore, the semi-supervised CycleGan network includes a generator and a discriminator. The generator includes a Conv2d layer, a LeakyRelu layer, an InstanceNorm layer, a Relu layer, a TransConv2d layer, and a Tanh layer. The discriminator adopts a pixel-by-pixel scoring structure built with multiple layers of convolution, including a Conv2d layer, a LeakyRelu layer, and an InstanceNorm layer. The Conv2d layer is a two-dimensional convolutional layer, and the TransConv2d layer is a two-dimensional transposed convolutional layer.

[0035] In a second aspect, an embodiment of the present invention provides an ultrasonic phased array image optimization and reconstruction system based on a semi-supervised CycleGan network, including:

[0036] A sample module that generates training samples that correspond one-to-one in two domains of ultrasonic images and reconstructed images;

[0037] A function module that superimposes an adversarial loss function, a cycle consistency loss function, an individual loss function, and a true difference loss function to obtain a loss function;

[0038] A reconstruction module that builds a semi-supervised CycleGan network structure based on the loss function obtained by the function module, trains the semi-supervised CycleGan network structure using the training samples generated by the sample module, and uses the trained semi-supervised CycleGan network structure.

[0039] Compared with the prior art, the present invention has at least the following beneficial effects:

[0040] The ultrasonic phased array image optimization and reconstruction method based on the semi-supervised CycleGan network of the present invention first generates a one-to-one training set required for the training model. Secondly, on the basis of the original unsupervised CycleGan network model, a true difference loss function is introduced, and the difference between the foreign domain images respectively output by the generators GA and GB and the true foreign domain images is compared during training, so that the network can perform more targeted optimization on the defect morphology in the image. Finally, by inputting the ultrasonic phased array detection image into the trained network, the network can perform foreign domain reconstruction on the defect morphology without losing the defect location information, thereby eliminating the influence of artifacts in the ultrasonic image on the defect morphology.

[0041] Furthermore, in the simulation software, an objective detection environment can be built according to the parameters of the actual detection instrument and the material to be measured, and the defect features at any position and morphology can be set. By using ultrasonic phased arrays to detect the test blocks with actual designed defects, defect images and corresponding ultrasonic images can be obtained more accurately. No matter which method is adopted, the training set of the present invention does not require further cropping of the images or specific annotation of the defects, reducing the workload of making the training set. In addition, compared with the two, the cost of generating the data set by the simulation software is lower and the flexibility of the samples is better.

[0042] Furthermore, the loss function By adding the sub-loss functions acting on various parts of the network model, it shows that the accuracy of image domain reconstruction is the result of the combined action of each sub-loss function. Especially when the network model performs backpropagation, by calculating the gradient, the gradients of each sub-loss function can be solved, facilitating the implementation of the code.

[0043] Furthermore, the adversarial loss function is the discriminator's estimation of the authenticity of the image distribution output by the generator, where the authenticity of the distribution refers to whether the output image is close to the distribution of the desired domain. When the estimation is correct, the network model will improve the performance of the generator, and when the estimation is incorrect, the network model will optimize the discriminator.

[0044] Furthermore, the cycle consistency loss function compares the similarity between the one-sided input image and the corresponding output image. Since in the network model, the domain of the image undergoes multiple transformations, the cycle consistency loss can ensure that the two images only undergo style domain changes on the basis of consistent content, thus achieving accurate reconstruction of the ultrasonic images with artifact defects.

[0045] Furthermore, the individual loss function is another parameter for evaluating the performance of the generator. It calculates the distribution estimation of the image output by the generator in the same domain when the same-domain image is input. Its purpose is to ensure that the generator also outputs the same-domain image when the same-domain image is input, improving the stability of the generator from another aspect.

[0046] Furthermore, the real difference loss directly evaluates the similarity between the foreign-domain images output by each generator and the corresponding real foreign-domain images in the training set. It pays more attention to whether the target content in the images is consistent, and improves the attention to the content in the foreign-domain transformation task of small-target images in the form of a loss function.

[0047] Furthermore, a sample input strategy of synchronous different sides is adopted to train the network model. First, only two foreign images are randomly selected from the training set in each batch, reducing the volume of the training set relied on by network training and improving the training efficiency of the network model. Second, by configuring loss functions on each generator and discriminator respectively and combining the loss function for overall network evaluation, it is convenient for the network to be optimized more pertinently and can also ensure the overall performance of the network.

[0048] Furthermore, the generator and the discriminator are the basic units for the network model to achieve cross-domain image transformation. The generator can perform cross-domain reconstruction on the input image, while the discriminator estimates the cross-domain similarity of the output image of the generator. The generator adopts the Unet network framework, which can not only extract high-dimensional features of the image but also retain low-level semantic information of the image. The discriminator adopts a multi-layer convolutional module structure that only increases the dimension on the channel without changing the image size. It can score the reconstructed image pixel by pixel, thus ensuring the accuracy of image reconstruction at the pixel level. The network model optimizes the network parameters of the generator or the discriminator according to the correctness of the discriminator score to improve its corresponding performance, thereby improving the performance of the generator and the discriminator after multiple iterations.

[0049] It can be understood that the beneficial effects of the second aspect above can refer to the relevant descriptions in the first aspect above and will not be elaborated here.

[0050] In summary, the method of the present invention can achieve cross-domain reconstruction of small target images without cropping the original image; for ultrasonic images, it not only retains the position information of the defects in the image but also eliminates the defect artifacts in the image, making the morphology of the defects more accurate; using simulated cross-domain images as the training set and not requiring specific annotation of the defects in each image greatly reduces the cost required to generate the training set.

[0051] Next, through the drawings and embodiments, the technical solutions of the present invention will be further described in detail. Description of the Drawings

[0052] Figure 1 It is a schematic diagram for generating training samples. Among them, (a) is a design schematic diagram of an artificial defect with specific information, (b) is a defect image of a square hole, (c) is a simulated ultrasonic image corresponding to the square hole defect image, (d) is a defect image of a circular hole, and (e) is a simulated ultrasonic image corresponding to the circular hole defect image;

[0053] Figure 2 It is a schematic diagram of the semi-supervised CycleGan network training strategy. Among them, (a) is a schematic diagram of the training process of the CycleGan network of the present invention, (b) is a training flow chart from domain A to domain B, and (c) is a training flow chart from domain B to domain A;

[0054] Figure 3 Schematic diagram of the network structure of the generator;

[0055] Figure 4 Schematic diagram of the network structure of the discriminator;

[0056] Figure 5 Front and back comparison diagrams of bidirectional reconstruction of ultrasonic phased array simulation images using the method of the present invention. Among them, (a) is the input ultrasonic image, (b) is the reconstructed defect image, (c) is the real defect image, (d) is the input defect image, (e) is the reconstructed ultrasonic image, and (f) is the real ultrasonic image;

[0057] Figure 6 Front and back comparison diagrams of defect reconstruction of actual ultrasonic phased array images using the method of the present invention. Among them, (a) is the defect image obtained by the ultrasonic phased array, and (b) is the defect image reconstructed by the network proposed in the present invention. Detailed implementation manners

[0058] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0059] In the description of the present invention, it should be understood that the terms "include" and "comprise" indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0060] It should also be understood that the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0061] It should be further understood that the term " / and" used in the specification of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the related listed items, and includes these combinations. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the contextually related objects.

[0062] It should be understood that although terms such as first, second, and third may be used in the embodiments of the present invention to describe preset ranges and the like, these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0063] Depending on the context, the word "if" as used herein can be interpreted as "when" or "while" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "if determined" or "if detected (stated condition or event)" can be interpreted as "when determined" or "in response to determining" or "when detected (stated condition or event)" or "in response to detecting (stated condition or event)".

[0064] Schematic diagrams of various structures according to the disclosed embodiments of the present invention are shown in the drawings. These figures are not drawn to scale, where for the purpose of clear expression, some details are enlarged and some details may be omitted. The shapes of various regions and layers shown in the figures, as well as their relative sizes and positional relationships, are merely exemplary and may deviate in practice due to manufacturing tolerances or technical limitations. And those skilled in the art can design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0065] The present invention provides an ultrasonic phased array image optimization and reconstruction method based on a semi-supervised CycleGan network. Two image domains composed of a manually designed defect schematic diagram (hereinafter referred to as a defect image) and the corresponding ultrasonic image are used as a training sample set. During training, mutual reconstruction of the two-domain images is achieved through the proposed CycleGan semi-supervised neural network. During testing, only the ultrasonic image of the defect is input to output its reconstructed image. The greatest advantage is that the training sample size is extremely low. Compared with a training set of thousands of images, the network model proposed by the present invention only requires dozens of paired ultrasonic and reconstructed images to meet the training requirements, and no preprocessing of ultrasonic phased array imaging is required, that is, the neural network can learn the corresponding features of the ultrasonic image and the reconstructed image.

[0066] An ultrasonic phased array image optimization and reconstruction method based on a semi-supervised CycleGan network of the present invention includes the following steps:

[0067] S1. Generate training samples

[0068] The samples used for training the network structure proposed by the present invention need to correspond one by one in two domains of ultrasonic images and reconstructed images, and are realized through two methods:

[0069] The first method is to use simulation software to perform ultrasonic imaging on artificially designed defects in a two-dimensional plane, thereby obtaining the ultrasonic images of the defects and the original defect images.

[0070] The second method is to use an ultrasonic phased array device to perform actual measurements and corresponding reconstruction calculations on a test block processed with accurate defect information, so as to obtain the defect image of the test block and the corresponding ultrasonic image from the processing information respectively.

[0071] Please refer to Figure 1 , where square holes with a side length of 1 mm and circular holes with a diameter of 1 mm are designed at two specified positions respectively. After calculation by the simulation software, the ultrasonic images corresponding to the defect images are obtained. Using the above method, paired defect images and ultrasonic images can be designed and obtained at any position, thereby generating training samples.

[0072] S2. Design the loss function

[0073] In the network model proposed in the present invention, four loss functions, namely adversarial loss, cycle consistency loss, individual loss, and real difference loss, are respectively defined. The calculation methods of the first three of them are the same as those of the loss functions defined in the standard CycleGan network. The above four loss functions are superimposed to obtain the final model loss function. The expectations of these four loss functions are different, and they limit the network model from different angles and are all indispensable.

[0074] Specifically, the adversarial loss means that in a group of generators and discriminators in the CycleGan network, the discriminator is used to identify the authenticity of the images output by the generator. Taking Figure 2 (b) as an example, where the image a in domain A passes through the generator GA and outputs the image in domain B , the adversarial loss is expressed as follows:

[0075]

[0076] Among them, GA(a) represents the image in domain B output by the generator GA after the sample a (a ∼ P A ) in domain A passes through the generator GA

[0077] The discriminator DA discriminates between the real sample b (b ∼ P B ) in domain B and the pseudo-real sample respectively. In this process, it is expected that the discriminator DA can accurately discriminate between the real sample and the pseudo-real sample in the same domain, so a maximization scheme is adopted for the discriminator At the same time, it is expected that the pseudo-real sample of the generator is as similar as possible to the real sample in terms of domain, so a minimization scheme is adopted for the generator

[0078] Similarly, Figure 2 In (c), the domain A image That is:

[0079]

[0080] The cycle consistency loss consists of two sub - terms when converting from domain A to domain B and from domain B to domain A. During the two - way training process, the two sub - term structures simultaneously impose constraints on the network. Taking Figure 2 the example of the direction of converting domain A of b to domain B and then back to domain A, its cycle consistency loss is shown in formula (3):

[0081]

[0082] In formula (3), the domain A sample a first passes through the generator GA, and then through the generator GB. The outputs after the two conversions belong to the domain A samples That is, GB(GA(a)). Then, by calculating the similarity between the sample a and to measure the accuracy of the generator for reconstructing the image content.

[0083] Similarly, Figure 2 The cycle consistency loss in the direction of converting domain B to domain A and then back to domain B in (c) is shown in formula (4):

[0084]

[0085] Adding formula (3) and formula (4) together, the cycle consistency loss of the CycleGan network is obtained, as shown in formula (5):

[0086]

[0087] The above loss functions are all to ensure that the generator can accurately convert when inputting cross - domain images. When inputting images of the same domain, in order to ensure that the generator will not perform domain conversion on them, an identity loss is set, which is shown in formula (6):

[0088]

[0089] In formula (6), b and a are respectively input into the generators GA and GB, and their similarity with their own images is evaluated to ensure that the images do not change.

[0090] Since defects account for a small proportion of the content in ultrasound images, the discriminator uses the same-content image of the input image as the true value label on the basis of the authenticity of the generated image. The real difference loss is used to directly compare the similarity with the generated image, so as to guide the generator to enhance its performance. The real difference loss is shown in formula (7):

[0091]

[0092] Finally, the above formulas (1) to (7) are added together to obtain the overall loss function of the network model, as shown in formula (8):

[0093]

[0094] Among them, λ cyc ,λ idt and λ aut They are adjustable hyperparameters used to balance the weights of the corresponding three loss function items in the overall function.

[0095] S3. Build a semi-supervised CycleGan network and train it;

[0096] The standard CycleGan network is an unsupervised neural network. It realizes image style conversion by inputting two unpaired images from different domains. However, due to the particularity of industrial ultrasound images, it cannot effectively reconstruct ultrasound images into defect images. Therefore, the present invention proposes a semi-supervised CycleGan network structure and its corresponding training strategy, where the training strategy is as follows: Figure 2 shown.

[0097] from Figure 2 (a) As can be seen, the training of the CycleGan network is bidirectional, that is, the unpaired images of domain A and domain B are simultaneously input into two different generators from both sides, and the corresponding images that conform to the other domain are simultaneously output. In this process, the discriminator identifies the images output by the generator to improve the imaging performance of the generator. When the training is completed, the network can realize the transformation from ultrasound images (domain A) to defect images (domain B) ( Figure 2 (a) The red arrow direction from left to right) can also realize the transformation from defect image (domain B) to ultrasound image (domain A) ( Figure 2 (a) The blue arrow goes from right to left). During testing and application, only the model data of generator A needs to be used and the ultrasonic image can be input to reconstruct the defect image. Figure 2 (b) and Figure 2 (c) is a training flow chart of two specific domains, where a and b belong to domain A and domain B respectively. Figure 2(b) For example, after the input image a in domain A passes through the first generator GA, on the one hand, it is necessary to use the discriminator DA to determine whether the image generated by GA truly conforms to the distribution of domain B. On the other hand, it is necessary to evaluate the similarity between its output image in domain B and the corresponding image b in the original domain B for a to ensure the accuracy of the reconstruction of image a in domain B. Subsequently, it is input into the second generator GB, and its output is the result of reconstructing from domain B to domain A Figure 2 (c) Similarly.

[0098] In the semi-supervised CycleGan network, the generator uses the Unet network structure to implement the encoding and decoding tasks of image features, and the discriminator uses a per-pixel scoring structure built by multi-layer convolutions to distinguish the authenticity of the image. It should be noted that the implementation methods of the generator proposed in the present invention include but are not limited to using the Unet network, and also include other network structures for image feature extraction. Moreover, the specific configurations of each layer in the Unet adjusted according to specific image characteristics should also fall within the scope covered by the claims of the present invention. Taking the training process from domain A to domain B as an example, the generator is as Figure 3 shown.

[0099] In Figure 3 , the Conv2d layer is a two-dimensional convolutional layer, the TransConv2d layer is a two-dimensional transposed convolutional layer, and the other layers are all expressed normally. During the training process from domain A to domain B, the input ultrasound image not only realizes the downward extraction of features through the convolutional modules layer by layer on the left to obtain high-semantic information. At the same time, it also horizontally transfers the low-semantic information directly to the upsampling module on the right in the channel dimension to ensure the integrity of the low-semantic information.

[0100] The image output by the generator needs to be input into the discriminator to verify the reconstruction performance of the generator. The present invention uses a multi-layer convolutional module that only increases the dimension in the channel without changing the image size. Compared with other Markov discriminators (PatchGan) based on increasing the receptive field, it can score the reconstructed image pixel by pixel, thereby ensuring the accuracy of image reconstruction at the pixel level. The network structure of the discriminator is as Figure 4 shown.

[0101] In Figure 4 , the size of the per-pixel scoring matrix is the same as that of the input ultrasound image, and it only expands in the channel dimension during the convolution process without compressing the feature size, thereby ensuring the accuracy of the scoring at the pixel level.

[0102] ​In addition, the network model proposed by the present invention uses the Adam optimizer to optimize the network loss function.

[0103] It should be noted that for the convenience of discussion, the L1 loss and the MSE loss are respectively used in Formulas (1) to (7) to evaluate the similarity of features and images. However, in practice, other loss functions for image similarity evaluation, including but not limited to the L2 loss and the SSIM loss, etc., should also fall within the scope of this patent.

[0104] In another embodiment of the present invention, a semi-supervised CycleGan network-based ultrasonic phased array image optimization and reconstruction system is provided. This system can be used to implement the above-mentioned semi-supervised CycleGan network-based ultrasonic phased array image optimization and reconstruction method. Specifically, the semi-supervised CycleGan network-based ultrasonic phased array image optimization and reconstruction system includes modules, modules, modules, modules, and modules.

[0105] Among them,.

[0106] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Usually, the components described and shown in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0107] The method proposed by the present invention is used to test and verify the simulated ultrasonic images and the defective images obtained from actual detections.

[0108] The resolutions of the simulated images and the actual detection defective images are both 256x256. Among them, the reconstruction results using the simulated ultrasonic images are as Figure 5 and shown in Table 1:

[0109] Table 1 Comparison of Reconstruction Results of Ultrasonic Simulation Images

[0110] Number Image Name X - axis / pixel Y - axis / pixel Area / pixel 1 Input Ultrasonic Image 51 64 318 2 Reconstructed Defect Image 52 64 44 3 True Defect Image 51 63 32 4 Input Defect Image 102 115 39 5 Reconstructed Ultrasonic Image 102 114 484 6 True Ultrasonic Image 102 115 308

[0111] From Figure 5 and Table 1, it can be seen that Figure 5(a), (b), (c), and numbers 1 - 3 represent the performance of reconstructing defect images from the input ultrasonic images. Compared with the real defect images, the reconstructed defect images have an offset of only 1 pixel on the X-axis / Y-axis. In terms of the defect area, the defect occupies 318 pixels in the ultrasonic image, while the reconstructed defect only occupies 44 pixels in the image, and the corresponding real defect occupies 32 pixels. At the same time, Figure 5 (d), (e), (f), and numbers 4 - 6 represent the performance of reconstructing ultrasonic images from the input defect images. The defect in the reconstructed ultrasonic image has an offset of only 1 pixel on the Y-axis. The above comparison shows that the present invention can ensure the high-precision reconstruction of defects from ultrasonic images.

[0112] Finally, the method proposed in the present invention is used to verify the defect-containing images obtained from actual detection, and the results are as Figure 6 shown in Table 2

[0113] Table 2 Reconstruction Results of Ultrasonic Detection Images

[0114] Defect Number X - axis / pixel Y - axis / pixel Area / pixel 1 - Ultrasonic Image 229 182 342 1 - Reconstructed Image 228 182 217 2 - Ultrasonic Image 177 130 212 2 - Reconstructed Image 177 130 130 3 - Ultrasonic Image 124 79 232 3 - Reconstructed Image 125 79 144 4 - Ultrasonic Image 73 27 244 4 - Reconstructed Image 73 27 155 5 - Ultrasonic Image 46 27 244 5 - Reconstructed Image 46 27 154

[0115] As shown by Figure 6 and Table 2, for defects numbered 1 - 5, the reconstruction accuracy can be maintained within 1 pixel in terms of positioning after reconstruction. From the defect area in the image, all 5 defects can be effectively reduced, thereby further improving the morphological characterization of defects in the image.

[0116] In summary, the present invention, a method and system for optimizing and reconstructing ultrasonic phased array images based on a semi-supervised CycleGan network, can achieve cross-domain reconstruction of small target images without cropping the original image. For ultrasonic images, this not only retains the position information of defects in the image but also eliminates defect artifacts in the image, making the morphology of defects more accurate. In the method, simulated cross-domain images are used as the training set, and specific annotation of defects in each image is not required, greatly reducing the cost required to generate the training set.

[0117] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or a means for implementing the functions specified in multiple blocks.

[0119] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction means that implements the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or a means for implementing the functions specified in multiple blocks.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or a means for implementing the functions specified in multiple blocks.

[0121] The above content is only for explaining the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. Any modification made on the basis of the technical solution according to the technical idea proposed by the present invention falls within the protection scope of the claims of the present invention.

Claims

1. An ultrasonic phased array image optimization and reconstruction method based on a semi-supervised CycleGan network, characterized in that Including the following steps: S1. Generate training samples that are one-to-one corresponding in the two domains of ultrasonic images and reconstructed images; S2. Superimpose the adversarial loss function, cycle consistency loss function, individual loss function, and real difference loss function to obtain the loss function; S3. Build a semi-supervised CycleGan network structure based on the loss function obtained in step S2, and use the training samples generated in step S1 to train the semi-supervised CycleGan network structure. After the network training is completed, input the ultrasonic images actually detected by the ultrasonic phased array into the trained semi-supervised CycleGan network structure to obtain the reconstructed images for detecting defects. The training process of the semi-supervised CycleGan network is as follows: Unpaired images of ultrasonic images and defect images are input into two different generators from both sides at the same time, and corresponding images that conform to the other domain are output at the same time. The discriminator discriminates the images output by the generators; Ultrasound image As the input side, in stage, the image distributed in domain A After passing through the generator GA, its corresponding defective image distributed in domain B is output ; On the one hand, the similarity between the defective image and domain B is evaluated by the adversarial loss function of discriminator DA ; on the other hand, through in the real difference loss function, represents the probability distribution that the image obeys domain B, quantifying the difference degree between the defective image generated by GA and the real defective image corresponding to the ultrasonic image . Subsequently, at stage, the domain B defect image of the input generator GB outputs its corresponding domain A ultrasonic image , and through the cyclic consistency loss function, perform domain similarity comparison, indicating that the image obeys the probability distribution of domain A; Defect image As the input side, it successively experiences and two stages, and is respectively constrained by the discriminator DB adversarial loss function , the real difference loss function and the cycle consistency loss function . After the two-way output of a batch of images is completed, the semi-supervised CycleGan optimizes and adjusts the network parameters according to the various loss function terms calculated above; when the training is completed, the semi-supervised CycleGan network realizes the conversion from ultrasonic images to defect images and the conversion from defect images to ultrasonic images.

2. The ultrasonic phased array image optimization and reconstruction method based on the semi-supervised CycleGan network according to claim 1, wherein In step S1, ultrasonic imaging is performed on artificially designed defects in a two-dimensional plane, and the ultrasonic images of the defects and the original defect images are used to form training samples; or actual measurement and corresponding reconstruction calculations are performed on test blocks containing accurate defect information after processing, and the defect images of the test blocks and the corresponding ultrasonic images are obtained from the processing information respectively to form training samples.

3. The ultrasonic phased array image optimization and reconstruction method based on the semi-supervised CycleGan network according to claim 1, characterized in that In step S2, the loss function Specifically: Among them, , and are adjustable hyperparameters respectively, and is the adversarial loss function, is the cycle consistency loss function, is the individual loss function, is the true difference loss function.

4. The ultrasonic phased array image optimization and reconstruction method based on the semi-supervised CycleGan network according to claim 3, characterized in that Adversarial loss function and are respectively as follows: Among them, represents the sample of domain A The image of domain B output after passing through the generator GA , , represents the sample of domain B The image of domain A output after passing through the generator GB , , represents the class score of the discriminator DA for the image , represents the class score of the discriminator DB for the image , represents that the image obeys the probability distribution of domain A represents that the image obeys the probability distribution of domain B 5. The ultrasonic phased array image optimization and reconstruction method based on the semi-supervised CycleGan network according to claim 3, wherein Cyclic consistency loss function is as follows: Among them, represents the sample of domain A The image of domain B output after passing through the generator GA , , represents the sample of domain B The image of domain A output after passing through the generator GB , , represents the image obeys the probability distribution of domain A, represents the image obeys the probability distribution of domain B.

6. The ultrasonic phased array image optimization and reconstruction method based on the semi-supervised CycleGan network according to claim 3, wherein, Individual loss function is as follows: Among them, represents the sample of domain A The image of domain B output after passing through the generator GA , , represents the sample of domain B The image of domain A output after passing through the generator GB , .

7. The method for optimizing and reconstructing ultrasonic phased array images based on a semi-supervised CycleGan network according to claim 3, wherein True difference loss function is as follows: Among them, represents the sample of domain A The image of domain B output after passing through the generator GA , , represents the sample of domain B The image of domain A output after passing through the generator GB , , represents the image obeys the probability distribution of domain A, represents the image obeys the probability distribution of domain B, and MSE is the mean square error loss function.

8. The ultrasonic phased array image optimization and reconstruction method based on the semi-supervised CycleGan network according to claim 1, wherein The semi-supervised CycleGan network includes a generator and a discriminator. The generator includes a Conv2d layer, a LeakyRelu layer, an InstanceNorm layer, a Relu layer, a TransConv2d layer, and a Tanh layer; the discriminator adopts a pixel-by-pixel scoring structure built by multiple convolutions, including a Conv2d layer, a LeakyRelu layer, and an InstanceNorm layer; the Conv2d layer is a two-dimensional convolutional layer, and the TransConv2d layer is a two-dimensional transposed convolutional layer.

9. An ultrasonic phased array image optimization and reconstruction system based on a semi-supervised CycleGan network, characterized in that, For implementing the ultrasonic phased array image optimization and reconstruction method based on the semi-supervised CycleGan network described in claim 1, including: A sample module that generates training samples that are one-to-one corresponding in the two domains of ultrasonic images and reconstructed images; A function module that superimposes the adversarial loss function, cycle consistency loss function, individual loss function, and real difference loss function to obtain the loss function; A reconstruction module that builds a semi-supervised CycleGan network structure based on the loss function obtained by the function module, trains the semi-supervised CycleGan network structure using the training samples generated by the sample module, and uses the trained semi-supervised CycleGan network structure.

Images