Ultrasonic image processing method and device and computer equipment

Abstract

The invention relates to an ultrasonic image processing method and device and computer equipment. The method comprises the following steps: acquiring an ultrasonic echo image of a target object; according to the ultrasonic echo feature data of the ultrasonic echo image under different scales, performing multi-scale super-resolution reconstruction on the ultrasonic echo image to obtain a super-resolution ultrasonic image; performing enhancement processing on the super-resolution ultrasonic image by adopting an image enhancement adversarial network to obtain an ultrasonic enhancement echo image; according to the ultrasonic echo sparse features of the ultrasonic enhanced echo image, performing local image reconstruction on the ultrasonic enhanced echo image to obtain a sparse reconstructed echo image; and identifying a defect area from the sparse reconstruction echo image to obtain an object defect ultrasonic image. By adopting the method, the accuracy of the ultrasonic nondestructive testing technology in defect detection of the target object can be effectively improved, and a more accurate, comprehensive and efficient technical means is provided for defect evaluation in materials or structures.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular to an ultrasonic image processing method, device and computer equipment. Background Art

[0002] With the development of nondestructive testing technology, ultrasonic nondestructive testing technology has emerged. This technology uses the propagation characteristics of high-frequency sound waves in materials to detect internal defects and discontinuities in materials. By emitting ultrasonic waves to the object being tested and analyzing the waveform and time of the reflected waves, defects such as cracks, cavities, and inclusions inside the material can be found. This method does not destroy the object being tested and has the advantages of high detection accuracy, strong penetration, and a wide range of applications. It is widely used in quality control and safety testing of metals, composite materials, and welded structures.

[0003] In traditional technology, the received ultrasonic signal is converted into an electrical signal and processed by a computer, including signal amplification, filtering and time base correction; then these processed signals are used to generate A-scan, B-scan or C-scan images through echo time and amplitude information. The A-scan image shows the reflected signal on a single sound beam path, the B-scan image is a two-dimensional image of the cross section, and the C-scan image provides three-dimensional detection results. Due to signal noise interference during ultrasonic detection, internal material inhomogeneity affecting ultrasonic propagation, poor contact between the probe and the surface of the object being tested, or uneven distribution of the coupling medium, the signal distortion and image blur obtained by ultrasonic non-destructive testing technology using traditional technology leads to insufficient accuracy in defect detection of target objects by ultrasonic non-destructive testing technology. Summary of the invention

[0004] Based on this, it is necessary to provide an ultrasonic image processing method, device, computer equipment, computer-readable storage medium and computer program product that can effectively improve the accuracy of ultrasonic non-destructive testing technology in defect detection of target objects in response to the above technical problems.

[0005] In a first aspect, the present application provides an ultrasound image processing method, comprising:

[0006] Acquiring an ultrasonic echo image of a target object; the ultrasonic echo image is obtained by fusing scan images obtained by different scanning methods of the target object;

[0007] According to the ultrasonic echo feature data of the ultrasonic echo image at different scales, the ultrasonic echo image is reconstructed with multi-scale super-resolution to obtain a super-resolution ultrasonic image;

[0008] Using an image enhancement adversarial network, the super-resolution ultrasound image is enhanced to obtain an ultrasound enhanced echo image;

[0009] According to the ultrasonic echo sparse characteristics of the ultrasonic enhanced echo image, image details are reconstructed on the ultrasonic enhanced echo image to obtain a sparse reconstructed echo image;

[0010] A defect area is identified from the sparsely reconstructed echo image to obtain an object defect ultrasonic image; the object defect ultrasonic image is used to evaluate defects in a material or structure.

[0011] In a second aspect, the present application also provides an ultrasonic image processing device, comprising:

[0012] An echo image acquisition module, used for acquiring an ultrasonic echo image of a target object; the ultrasonic echo image is obtained by fusing scan images obtained by different scanning methods of the target object;

[0013] An ultrasonic image reconstruction module, used for performing multi-scale super-resolution reconstruction on the ultrasonic echo image according to ultrasonic echo feature data of the ultrasonic echo image at different scales to obtain a super-resolution ultrasonic image;

[0014] An ultrasonic image enhancement module, used to enhance the super-resolution ultrasonic image using an image enhancement adversarial network to obtain an ultrasonic enhanced echo image;

[0015] The ultrasonic image reconstruction module is further used to reconstruct the image details of the ultrasonic enhanced echo image according to the ultrasonic echo sparse characteristics of the ultrasonic enhanced echo image to obtain a sparse reconstructed echo image;

[0016] The object defect recognition module is used to identify the defect area from the sparse reconstructed echo image to obtain an object defect ultrasonic image; the object defect ultrasonic image is used to evaluate defects in materials or structures.

[0017] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0018] Acquiring an ultrasonic echo image of a target object; the ultrasonic echo image is obtained by fusing scan images obtained by different scanning methods of the target object;

[0019] According to the ultrasonic echo feature data of the ultrasonic echo image at different scales, the ultrasonic echo image is reconstructed with multi-scale super-resolution to obtain a super-resolution ultrasonic image;

[0020] Using an image enhancement adversarial network, the super-resolution ultrasound image is enhanced to obtain an ultrasound enhanced echo image;

[0021] According to the ultrasonic echo sparse characteristics of the ultrasonic enhanced echo image, image details are reconstructed on the ultrasonic enhanced echo image to obtain a sparse reconstructed echo image;

[0022] A defect area is identified from the sparsely reconstructed echo image to obtain an object defect ultrasonic image; the object defect ultrasonic image is used to evaluate defects in a material or structure.

[0023] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0024] Acquiring an ultrasonic echo image of a target object; the ultrasonic echo image is obtained by fusing scan images obtained by different scanning methods of the target object;

[0025] According to the ultrasonic echo feature data of the ultrasonic echo image at different scales, the ultrasonic echo image is reconstructed with multi-scale super-resolution to obtain a super-resolution ultrasonic image;

[0026] Using an image enhancement adversarial network, the super-resolution ultrasound image is enhanced to obtain an ultrasound enhanced echo image;

[0027] According to the ultrasonic echo sparse characteristics of the ultrasonic enhanced echo image, image details are reconstructed on the ultrasonic enhanced echo image to obtain a sparse reconstructed echo image;

[0028] A defect area is identified from the sparsely reconstructed echo image to obtain an object defect ultrasonic image; the object defect ultrasonic image is used to evaluate defects in a material or structure.

[0029] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0030] Acquiring an ultrasonic echo image of a target object; the ultrasonic echo image is obtained by fusing scan images obtained by different scanning methods of the target object;

[0031] According to the ultrasonic echo feature data of the ultrasonic echo image at different scales, the ultrasonic echo image is reconstructed with multi-scale super-resolution to obtain a super-resolution ultrasonic image;

[0032] Using an image enhancement adversarial network, the super-resolution ultrasound image is enhanced to obtain an ultrasound enhanced echo image;

[0033] According to the ultrasonic echo sparse characteristics of the ultrasonic enhanced echo image, image details are reconstructed on the ultrasonic enhanced echo image to obtain a sparse reconstructed echo image;

[0034] A defect area is identified from the sparsely reconstructed echo image to obtain an object defect ultrasonic image; the object defect ultrasonic image is used to evaluate defects in a material or structure.

[0035] The above-mentioned ultrasonic image processing method, device, computer equipment, storage medium and computer program product obtain an ultrasonic echo image of a target object; the ultrasonic echo image is obtained by fusing scanning images obtained by different scanning methods of the target object; based on the ultrasonic echo feature data of the ultrasonic echo image at different scales, the ultrasonic echo image is reconstructed with multi-scale super-resolution to obtain a super-resolution ultrasonic image; the super-resolution ultrasonic image is enhanced by using an image enhancement adversarial network to obtain an ultrasonic enhanced echo image; based on the ultrasonic echo sparse characteristics of the ultrasonic enhanced echo image, the ultrasonic enhanced echo image is reconstructed with image details to obtain a sparse reconstructed echo image; the defect area is identified from the sparse reconstructed echo image to obtain an object defect ultrasonic image; the object defect ultrasonic image is used to evaluate defects in materials or structures.

[0036] By fusing ultrasonic echo data obtained by different scanning methods, the ultrasonic echo image is obtained, and the multi-scale super-resolution reconstruction technology is used to improve the spatial resolution and detail clarity of the image, thereby overcoming the limitations of the traditional single-scale reconstruction method. Combined with the image enhancement adversarial network, the contrast and noise suppression capabilities of the ultrasonic image are effectively enhanced, so that even tiny defects can be clearly identified. Through local image reconstruction of sparse features, the positioning accuracy and significance of the defect area are further improved, and the probability of false detection and missed detection is reduced. The final object defect ultrasonic image can effectively improve the accuracy of ultrasonic non-destructive testing technology in defect detection of the target object, and provide a more accurate, comprehensive and efficient technical means for defect assessment in materials or structures, highlighting its important application value in the field of non-destructive testing, which can significantly improve the level of industrial inspection and quality control. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 is an application environment diagram of an ultrasonic image processing method in an embodiment;

[0039] Figure 2 is a schematic flow chart of an ultrasonic image processing method in an embodiment;

[0040] Figure 3A schematic diagram of a process for obtaining a super-resolution ultrasound image in one embodiment;

[0041] Figure 4 A schematic diagram of a process for obtaining a super-resolution ultrasound image in another embodiment;

[0042] Figure 5 It is a schematic diagram of the flow of a method for obtaining a cascaded reconstructed image in one embodiment;

[0043] Figure 6 It is a flowchart of a method for obtaining a sparsely reconstructed echo image in one embodiment;

[0044] Figure 7 A schematic diagram of a flow chart of a method for optimizing a sparse dictionary in one embodiment;

[0045] Figure 8 A schematic flow chart of a method for optimizing a sparse dictionary in another embodiment;

[0046] Fig. 9 It is a flowchart of a method for obtaining an ultrasonic image of an object defect in one embodiment;

[0047] Fig.10 is a structural block diagram of an ultrasonic image processing device in one embodiment;

[0048] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] The ultrasonic image processing method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 obtains the ultrasonic echo image of the target object through the terminal 102; the ultrasonic echo image is obtained by fusing the scanning images obtained by different scanning methods of the target object; according to the ultrasonic echo feature data of the ultrasonic echo image at different scales, the ultrasonic echo image is reconstructed with multi-scale super-resolution to obtain a super-resolution ultrasonic image; the super-resolution ultrasonic image is enhanced by using an image enhancement adversarial network to obtain an ultrasonic enhanced echo image; according to the ultrasonic echo sparse characteristics of the ultrasonic enhanced echo image, the ultrasonic enhanced echo image is reconstructed with image details to obtain a sparse reconstructed echo image; the defect area is identified from the sparse reconstructed echo image to obtain an object defect ultrasonic image; the object defect ultrasonic image is used to evaluate defects in materials or structures. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0051] In an exemplary embodiment, Figure 2 As shown, a method for processing ultrasonic images is provided. Figure 1 Taking the server in as an example, the method includes the following steps 202 to 210.

[0052] in:

[0053] Step 202: Acquire an ultrasonic echo image of the target object.

[0054] The ultrasonic echo image may be obtained by fusing scan images obtained by different scanning modes of the target object, wherein the different scanning modes include A-scan, B-scan, C-scan, D-scan, P-scan, S-scan, F-scan, etc.

[0055] Specifically, an ultrasonic device is used to perform ultrasonic detection on a target object using a variety of scanning methods, wherein the different scanning methods include A-scan, B-scan, C-scan, D-scan, P-scan, S-scan, F-scan, etc., to obtain ultrasonic echo images at different angles and depths; the ultrasonic echo data obtained by these various scanning methods are preprocessed to eliminate noise and inconsistency; the preprocessed ultrasonic echo data are fused through an image fusion algorithm, and the characteristic information of each data is comprehensively utilized to generate an ultrasonic echo image.

[0056] Step 204 , performing multi-scale super-resolution reconstruction on the ultrasonic echo image according to the ultrasonic echo feature data of the ultrasonic echo image at different scales to obtain a super-resolution ultrasonic image.

[0057] Among them, the ultrasonic echo feature data can be various key feature information extracted from the ultrasonic echo image, which reflects the characteristics of different structures and tissues in the image, and the feature data includes but is not limited to edges, textures, intensity changes, frequency components, etc.

[0058] Among them, multi-scale super-resolution reconstruction can be a method of gradually improving the resolution and detail clarity of the image by extracting and integrating image feature data at different scales.

[0059] The super-resolution ultrasound image may be an ultrasound image processed by super-resolution reconstruction technology, and may have higher spatial resolution and clearer details than the original resolution image.

[0060] Specifically, the ultrasonic echo image is input into a multi-scale super-resolution model, and features of different scales are extracted through the multi-layer convolutional layers of the multi-scale super-resolution model. The feature extraction effect is enhanced by using an adaptive feature extraction module and an attention mechanism to obtain ultrasonic echo feature data of the ultrasonic echo image at different scales. The ultrasonic echo feature data of different scales are weightedly fused and feature selected to generate a feature map with improved resolution relative to the ultrasonic echo image. The resolution of the image is gradually improved by using the reconstruction layer of the cascade reconstruction network and combining a multi-level loss function to restore the high-frequency details in the image and obtain a super-resolution ultrasonic image.

[0061] Step 206: Use an image enhancement adversarial network to enhance the super-resolution ultrasound image to obtain an ultrasound enhanced echo image.

[0062] Among them, the image enhancement adversarial network can be an image processing technology based on the generative adversarial network to improve the quality and details of the image. The image enhancement adversarial network consists of two neural networks, the generator and the discriminator. The generator generates high-quality images through multi-scale convolution and deconvolution modules, and the discriminator evaluates the authenticity of the generated image and the real image through a multi-scale discrimination module. The introduction of the attention mechanism and feature extraction enhancement module further improves the image details and quality.

[0063] Among them, the ultrasound enhanced echo image can be an ultrasound image processed by an image enhancement adversarial network, and has higher definition and richer details.

[0064] Specifically, the super-resolution ultrasound image is input into the generator network in the image enhancement adversarial network. The generator network enhances the super-resolution ultrasound image through multi-scale convolution and deconvolution modules to generate an enhanced image, and introduces an attention mechanism and a feature extraction enhancement module to improve the details and quality of the generated enhanced image. Then the discriminator network judges the authenticity of the generated enhanced image and the real image through a multi-scale discriminant module, and trains with adaptive adversarial loss to improve the performance of the generator and the discriminator. Finally, through multiple multi-task loss functions, optimization is performed on the adversarial loss, reconstruction loss and perceptual loss to ensure the quality and authenticity of the generated image, and finally an ultrasound enhanced echo image is obtained.

[0065] Step 208 , reconstructing the image details of the ultrasound enhanced echo image according to the ultrasound echo sparse feature of the ultrasound enhanced echo image to obtain a sparse reconstructed echo image.

[0066] Among them, ultrasound echo sparse features can be key feature information extracted from ultrasound enhanced echo images, which are presented in the form of sparse representation, meaning that only a small number of non-zero eigenvalues ​​are used to describe important structures and details in the image.

[0067] The sparsely reconstructed echo image may be an ultrasound image obtained by processing the sparse features of the ultrasound echo through sparse representation and reconstruction technology.

[0068] Specifically, the ultrasound enhanced echo image is input into the sparse representation model, and the sparse coding algorithm (such as OMP) in the sparse representation model is used for sparse coding, and the ultrasound echo sparse features are extracted, wherein, since the attention mechanism is introduced in the sparse coding process to enhance the attention to important feature areas (local images that need to be reconstructed), the ultrasound echo sparse features have different scales; at the same time, according to the enhanced echo image and the ultrasound enhanced echo image used for training, the original sparse dictionary is adaptively optimized to effectively represent the ultrasound echo sparse features; a weighted fusion algorithm is used to fuse these features, and information of different scales is integrated, and the fused features are reconstructed into a sparse reconstructed echo image using the optimized sparse dictionary.

[0069] Step 210 , identifying a defect area from the sparsely reconstructed echo image to obtain an ultrasonic image of the object defect.

[0070] The object defect ultrasonic image may be an image that identifies and displays a defect area in a material or structure by processing and analyzing an ultrasonic echo image.

[0071] Specifically, the sparse reconstructed echo image is preprocessed using an image processing algorithm to enhance the contrast and clarity of the image; then, a method based on feature extraction and pattern recognition is used to extract possible defect features, such as edges, textures, and abnormal areas, from the image; these features are classified and analyzed through machine learning or deep learning models to identify defect areas in the image; finally, the identified defect areas are marked on the sparse reconstructed echo image to generate an object defect ultrasonic image, wherein the object defect ultrasonic image provides accurate visual information by clearly displaying the location, shape, and size of defects in materials or structures; by analyzing these images, the severity of the defects, their potential impact, and the repair or preventive measures that need to be taken can be determined.

[0072] In the above-mentioned ultrasonic image processing method, an ultrasonic echo image of a target object is acquired; the ultrasonic echo image is obtained by fusing scanning images obtained by different scanning methods of the target object; based on ultrasonic echo feature data of the ultrasonic echo image at different scales, the ultrasonic echo image is reconstructed with multi-scale super-resolution to obtain a super-resolution ultrasonic image; an image enhancement adversarial network is used to enhance the super-resolution ultrasonic image to obtain an ultrasonic enhanced echo image; based on the ultrasonic echo sparse features of the ultrasonic enhanced echo image, the ultrasonic enhanced echo image is reconstructed with image details to obtain a sparse reconstructed echo image; a defect area is identified from the sparse reconstructed echo image to obtain an object defect ultrasonic image; the object defect ultrasonic image is used to evaluate defects in materials or structures.

[0073] By fusing ultrasonic echo data obtained by different scanning methods, the ultrasonic echo image is obtained, and the multi-scale super-resolution reconstruction technology is used to improve the spatial resolution and detail clarity of the image, thereby overcoming the limitations of the traditional single-scale reconstruction method. Combined with the image enhancement adversarial network, the contrast and noise suppression capabilities of the ultrasonic image are effectively enhanced, so that even tiny defects can be clearly identified. Through the reconstruction of image details with sparse features, the positioning accuracy and significance of the defect area are further improved, and the probability of false detection and missed detection is reduced. The final object defect ultrasonic image can effectively improve the accuracy of ultrasonic non-destructive testing technology in defect detection of target objects, and provide a more accurate, comprehensive and efficient technical means for defect assessment in materials or structures, highlighting its important application value in the field of non-destructive testing, which can significantly improve the level of industrial inspection and quality control.

[0074] In an exemplary embodiment, Figure 3 As shown, according to the ultrasonic echo feature data of the ultrasonic echo image at different scales, the ultrasonic echo image is reconstructed with multi-scale super-resolution to obtain a super-resolution ultrasonic image, including steps 302 to 306. Among them:

[0075] Step 302: extract the features of the ultrasonic echo image at different scales to obtain feature data of each ultrasonic echo.

[0076] Specifically, multi-scale transformation techniques, such as wavelet transform or Gaussian pyramid, are applied to decompose the ultrasound echo image into different scales, and then a feature extraction algorithm, such as local binary pattern (LBP), histogram of oriented gradients (HOG) or convolutional neural network (CNN), is used at each scale to extract feature data to obtain the feature data of each ultrasound echo.

[0077] Step 304: perform feature selection on each ultrasonic echo feature data to obtain an ultrasonic echo feature graph.

[0078] Among them, feature selection can be to screen out the most representative and influential features from a large number of features to reduce data dimensions, reduce model complexity, and improve model performance and generalization ability.

[0079] Among them, the ultrasonic echo feature map can be an image generated by new feature data after feature selection and weight adjustment of ultrasonic echo feature data, showing the feature information of different areas in the image.

[0080] Specifically, based on the application of feature selection techniques, such as principal component analysis (PCA), recursive feature elimination (RFE) or importance scoring based on a tree model, it is necessary to screen out the most representative features from the ultrasonic echo feature data at different scales; further based on the weighted average of feature importance or the weight allocation method based on a learning algorithm, the ultrasonic echo feature data at different scales that have passed feature selection are given appropriate weights and further integrated to generate an ultrasonic echo feature map.

[0081] Step 306, using the cascade reconstruction network of several stages determined by the ultrasonic echo characteristic graph, cascade reconstruction is performed several times on the ultrasonic echo characteristic graph to obtain a super-resolution ultrasonic image.

[0082] Among them, the cascade reconstruction network can be a deep learning architecture for gradually improving the quality and resolution of images. The network at each stage (usually a convolutional neural network, CNN) reconstructs and optimizes the image, and the output result is used as the input of the network at the next stage.

[0083] Specifically, the minimum number of cascade reconstructions required is determined based on the parameters of the ultrasonic echo feature map, and then a cascade reconstruction network of several stages greater than the minimum number of cascade reconstructions (i.e., the number of stages is less than the minimum number of cascade reconstructions) is selected from the cascade reconstruction network that has been trained for multiple stages, and each stage of the cascade reconstruction network of several stages includes a convolutional neural network (CNN) sub-network. The ultrasonic echo feature map is then input into the first sub-network for preliminary reconstruction to obtain a preliminary super-resolution image; the preliminary reconstructed image is then input into the next sub-network for further reconstruction, and so on, and after several cascade reconstructions, a high-resolution ultrasonic image is finally obtained.

[0084] In this embodiment, by extracting the features of the ultrasound echo image at different scales, the feature data of each ultrasound echo is obtained, and feature selection is performed to generate an ultrasound echo feature map, and then a multi-stage cascade reconstruction network is used to reconstruct it several times, and finally a super-resolution ultrasound image is obtained. This method can significantly improve the resolution and quality of the image. This can not only show the details and important features in the ultrasound image more clearly, but also enhance the accuracy and reliability of diagnosis, providing more powerful support for clinical medicine and scientific research.

[0085] In an exemplary embodiment, Figure 4 As shown, a cascade reconstruction network of several stages determined by an ultrasonic echo feature map is used to perform cascade reconstruction on the ultrasonic echo feature map several times to obtain a super-resolution ultrasonic image, including steps 402 to 408. Among them:

[0086] Step 402: Based on the preset number of cascade reconstructions determined by the ultrasonic echo characteristic graph, the ultrasonic echo characteristic graph is input into the cascade network for resolution enhancement processing to obtain a resolution enhanced image.

[0087] The preset number of cascade reconstructions may be the number of reconstruction stages set in advance in the cascade reconstruction network, which determines the number of reconstruction and optimization processes that the image undergoes in the network.

[0088] Among them, the resolution-enhanced image can be an image generated by improving its original resolution through image processing technology, so that the details of the image are clearer and sharper.

[0089] Specifically, it is necessary to determine the appropriate preset number of cascade reconstructions according to the complexity of the ultrasonic echo feature map and the required resolution improvement effect. Then, the ultrasonic echo feature map is input into the first sub-network of the designed cascade reconstruction network for preliminary reconstruction to obtain a resolution-enhanced image.

[0090] Step 404: fuse the ultrasonic echo feature map and the resolution-enhanced image to obtain a resolution-fused image.

[0091] The resolution fused image may be an image processing result obtained by fusing image data from different sources to improve image quality and details.

[0092] Specifically, the ultrasound echo feature map and the resolution-enhanced image are aligned and normalized to ensure that they are within the same scale and intensity range. Then, one or more of the methods such as weighted averaging, image pyramid fusion or deep learning fusion network are used to effectively fuse the features of the preprocessed ultrasound echo feature map and the resolution-enhanced image, combining the detailed information of the ultrasound echo feature map and the high-resolution characteristics of the resolution-enhanced image to generate a resolution-fused image.

[0093] Step 406 , using the multi-level optimization function determined by the resolution fusion image, multi-level optimization is performed on the resolution fusion image to obtain a cascaded reconstructed image and an actual number of cascaded reconstructions.

[0094] The multi-level optimization function may be a comprehensive optimization function composed of a perception optimization function, or / and a pixel optimization function, or / and a structural similarity optimization function.

[0095] The cascade reconstructed image may be a high-resolution image generated by performing an optimization reconstruction process through multiple different optimization functions.

[0096] The actual number of cascade reconstruction times may be the number of times the cascade reconstruction has actually been performed.

[0097] Specifically, according to the complexity of the resolution fusion image and the required resolution improvement effect, one or more optimization functions are selected from the perception optimization function, the pixel optimization function and the structural similarity optimization function to form or define a multi-level optimization function, wherein the weights of various optimization functions in the multi-level optimization function are different. The resolution fusion image is input into the multi-level optimization function for multi-level optimization, and the visual quality of the image is improved according to the perception optimization function, or / and the image details are finely adjusted through the pixel optimization function, or / and the consistency of the image structure is ensured by using the structural similarity optimization function. The results obtained by multiple iterative optimizations of different optimization functions are added to obtain a cascaded reconstructed image and the actual number of cascaded reconstructions is recorded.

[0098] Step 408, when the actual number of cascade reconstructions is less than the preset number of cascade reconstructions, the cascade reconstructed image is used as the ultrasonic echo feature map, and the process returns to the step of inputting the ultrasonic echo feature map into the cascade network for resolution enhancement processing to obtain a resolution enhanced image, until the actual number of cascade reconstructions is equal to the preset number of cascade reconstructions to obtain a super-resolution ultrasonic image.

[0099] Specifically, if the current actual cascade reconstruction number is less than the preset cascade reconstruction number, the current cascade reconstructed image is used as the ultrasonic echo feature map, and then the step of inputting the ultrasonic echo feature map into the cascade network for resolution enhancement processing to obtain the resolution enhanced image and the subsequent steps are re-executed. This process is repeated until the actual cascade reconstruction number is equal to the preset cascade reconstruction number, and finally a super-resolution ultrasonic image is obtained.

[0100] In this embodiment, the preset number of cascade reconstructions determined based on the ultrasonic echo feature map is input into the cascade network for resolution enhancement processing to obtain a resolution-enhanced image; then the ultrasonic echo feature map and the resolution-enhanced image are fused to obtain a resolution-fused image; a multi-level optimization function consisting of a perception optimization function, a pixel optimization function, and a structural similarity optimization function is used to optimize the resolution-fused image to obtain a cascade reconstructed image and the actual number of cascade reconstructions; if the actual number is less than the preset number, the cascade reconstructed image is used as a new ultrasonic echo feature map, and the above steps are repeated until the two are equal, and finally a super-resolution ultrasonic image is obtained. This method effectively improves the image resolution and quality through multiple iterative optimizations, ensures significant improvements in the details and overall visual effects of ultrasonic images, and improves the accuracy of diagnosis and analysis.

[0101] In an exemplary embodiment, Figure 5 As shown, the multi-level optimization function of the resolution fusion image is used to perform multi-level optimization on the resolution fusion image to obtain a cascade reconstructed image, including steps 502 to 506. Among them:

[0102] Step 502: Perform multi-level optimization on the resolution fusion image according to the perception optimization function, or / and the pixel optimization function, or / and the structure similarity optimization function to obtain a resolution optimized ultrasound image.

[0103] Among them, the resolution-optimized ultrasound image can be an image that has been improved in visual effect, detail clarity and structural consistency through steps such as perception optimization, pixel optimization and structural similarity optimization.

[0104] Specifically, according to the complexity of the resolution fusion image and the required resolution improvement effect, the specific goals and weights of the perception optimization function, or / and the pixel optimization function, or / and the structural similarity optimization function are defined. The resolution fusion image is input into each optimization function after weight adjustment for multi-level optimization. The visual quality of the image is improved by the perception optimization function, the pixel optimization function is used to refine the pixel details of the image, and the structural consistency of the image is maintained by the structural similarity optimization function. After multiple iterative optimizations and optimization synergy of different optimization functions, a resolution-optimized ultrasound image is finally obtained.

[0105] Step 504: According to the resolution difference information between the resolution-optimized ultrasound image and the resolution-fused image, the weights of the perception optimization function, or / and the pixel optimization function, or / and the structural similarity optimization function are adjusted respectively.

[0106] The resolution difference information may be a quantitative indicator of the difference in resolution and detail performance between the resolution-optimized ultrasound image and the resolution-fused image.

[0107] Specifically, the resolution of the resolution-optimized ultrasound image and the resolution of the resolution-fused image are compared, and the resolution difference information between the two is calculated; then, the direction that needs further optimization is determined based on the resolution difference information, and further, based on the optimization direction and the resolution difference information, the weights of the perception optimization function, and / or the pixel optimization function, and / or the structural similarity optimization function are adjusted respectively.

[0108] Step 506: Perform multi-level optimization on the resolution-optimized ultrasound image according to the adjusted perception optimization function, or / and the adjusted pixel optimization function, or / and the adjusted structural similarity optimization function to obtain a cascaded reconstructed image.

[0109] Specifically, the adjusted perceptual optimization function, or / and the adjusted pixel optimization function, or / and the adjusted structural similarity optimization function are applied to further refine and improve the visual quality, or / and the pixel details, or / and the structural consistency of the resolution-optimized ultrasound image, respectively, and each optimization function prioritizes a specific optimization target according to its new weight, and through the optimization steps and iterations of the adjusted perceptual optimization function, or / and the adjusted pixel optimization function, or / and the adjusted structural similarity optimization function, the image is comprehensively improved in all aspects. Finally, through a multi-level optimization process again, a cascaded reconstructed image is generated.

[0110] In this embodiment, the resolution-optimized ultrasound image is obtained by performing multi-level optimization on the resolution-fused image using the perceptual optimization function, the pixel optimization function, and the structural similarity optimization function; then, the weights of these optimization functions are adjusted respectively according to the resolution difference information between the resolution-optimized ultrasound image and the resolution-fused image; and then, the resolution-optimized ultrasound image is multi-level optimized according to the adjusted optimization function, and finally a cascaded reconstructed image is obtained. This method ensures that the optimization process fully considers the visual quality, detail clarity, and structural consistency of the image through continuous optimization and adjustment, effectively improves the resolution and quality of the ultrasound image, and ensures the accuracy and reliability of the final image.

[0111] In an exemplary embodiment, Figure 6As shown, according to the ultrasonic echo sparse characteristics of the ultrasonic enhanced echo image, the ultrasonic enhanced echo image is reconstructed for image details to obtain a sparsely reconstructed echo image, including steps 602 to 606. Wherein:

[0112] Step 602: sparsely encode the ultrasound enhanced echo image to obtain sparse features of each echo data.

[0113] The sparse features of the echo data may be feature representations obtained by representing the data of the ultrasound echo image as sparse linear combinations through sparse coding technology.

[0114] Specifically, the ultrasound enhanced echo image is divided into small blocks or key features are extracted using feature extraction methods; then sparse representation techniques, such as dictionary learning or sparse coding algorithms, are applied to represent each image block or key feature as a sparse linear combination so that most of the coefficients are zero, thereby obtaining sparse features of each echo data.

[0115] Step 604 : adaptively optimize the preset sparse dictionary according to the ultrasound enhanced echo image to obtain an optimized sparse dictionary.

[0116] Among them, adaptive optimization can be the process of continuously adjusting and improving algorithms or models to better adapt them to specific data or environments.

[0117] The optimized sparse dictionary may be a sparse dictionary that uses a dictionary learning algorithm to gradually improve dictionary elements using optimized sample data so that the dictionary can better capture the main features and patterns of the data.

[0118] Specifically, since a preset sparse dictionary has been set in the sparse representation technology of step 602, the feature data of the ultrasound enhanced echo image is used as the optimized data sample, and a dictionary learning algorithm, such as K-SVD or an online dictionary learning algorithm, is applied to iteratively update the preset sparse dictionary, wherein in each iteration, the reconstruction error is minimized and the dictionary elements are adjusted to better represent the features of the ultrasound echo image, and an optimized sparse dictionary is obtained after multiple iterations.

[0119] Step 606: According to the optimized sparse dictionary, the sparse features of each echo data are fused to obtain a sparse reconstructed echo image.

[0120] Specifically, each sparse feature is reconstructed by applying an optimized sparse dictionary and calculating its corresponding sparse coefficient. Then, these sparse coefficients are fused according to a predetermined fusion strategy (such as weighted average or direct superposition) to obtain a comprehensive sparse coefficient representation. Finally, the echo image is reconstructed using the optimized sparse dictionary and the comprehensive sparse coefficient representation, thereby generating a sparse reconstructed echo image.

[0121] In this embodiment, the sparse features of each echo data are extracted by sparse coding the ultrasound enhanced echo image; then, the preset sparse dictionary is adaptively optimized according to the ultrasound enhanced echo image to obtain an optimized sparse dictionary; finally, the sparse features of each echo data are fused according to the optimized sparse dictionary to generate a sparse reconstructed echo image. This method fully extracts and fuses image features through sparse coding and dictionary optimization, significantly improves the reconstruction quality and resolution of the echo image, retains key details and information, and enhances the clarity and diagnostic value of the image.

[0122] In an exemplary embodiment, Figure 7 As shown, according to the ultrasound enhanced echo image, the preset sparse dictionary is adaptively optimized to obtain an optimized sparse dictionary, including steps 702 to 708. Among them:

[0123] Step 702: down-sample the ultrasound enhanced echo image in different scales to obtain an echo image pyramid.

[0124] The stepwise down sampling may be to perform multiple sampling processes on the image in a manner of decreasing the resolution layer by layer, and the image size is reduced by a fixed ratio (eg, halved) each time the image is sampled.

[0125] Among them, the echo image pyramid can be a multi-scale image representation method, which generates a series of image levels with gradually decreasing resolution by down-sampling the ultrasound enhanced echo image layer by layer.

[0126] Specifically, the initial ultrasound-enhanced echo image is selected as the highest level, and then downsampling algorithms (such as Gaussian blur followed by downsampling) are applied in sequence to gradually halve the size of the ultrasound-enhanced echo image or reduce it according to a predetermined ratio to generate multiple image levels with decreasing scales. This process is repeated until the predetermined minimum scale is reached, and finally an echo image pyramid containing different resolution levels is formed. In each scale image level, the image is divided into several small blocks, called echo image blocks.

[0127] Step 704: Use a preset sparse dictionary to perform sparse coding on each echo image block to obtain each sparse coefficient matrix.

[0128] The sparse coefficient matrix may be a matrix obtained by representing the echo image block as a sparse linear combination of preset dictionary elements through sparse coding technology.

[0129] Specifically, for each echo image block, a sparse coding algorithm (such as Lasso or OMP) is applied, combined with a preset sparse dictionary to represent it as a sparse linear combination of dictionary elements, and the corresponding sparse coefficient matrix is ​​calculated. Each sparse coefficient matrix represents a sparse representation of the image block on the dictionary basis elements.

[0130] Step 706, taking any sparse coefficient matrix and a preset sparse dictionary as sparse dictionary optimization units, and based on each sparse dictionary optimization unit, performing self-optimization on the preset sparse dictionary and the sparse coefficient matrix respectively until the preset sparse dictionary and the sparse coefficient matrix converge to obtain each adjusted sparse dictionary.

[0131] The sparse dictionary may be adjusted by continuously updating the base elements in the dictionary through an iterative optimization process to better adapt the sparse dictionary to the sparse representation of a specific data set.

[0132] Specifically, any sparse coefficient matrix and a preset sparse dictionary are used as sparse dictionary optimization units. For each sparse dictionary optimization unit obtained by combining different sparse coefficient matrices, a dictionary learning algorithm (such as K-SVD or online dictionary learning) is used to iteratively update the preset sparse dictionary and sparse coefficient matrix. In each iteration, the dictionary elements and sparse coefficients are adjusted to gradually reduce the reconstruction error. This process continues until the changes in the preset sparse dictionary and the sparse coefficient matrix tend to be stable, that is, a convergence state is reached. The adjusted sparse dictionaries are obtained.

[0133] Step 708: merge the adjusted sparse dictionaries to obtain an optimized sparse dictionary.

[0134] Specifically, a fusion strategy (such as weighted averaging, consensus dictionary construction or clustering method) is adopted to comprehensively process the dictionary elements of each adjusted sparse dictionary, merge repeated or similar dictionary elements, retain the key features of each dictionary, and finally generate an optimized sparse dictionary.

[0135] In this embodiment, the ultrasound enhanced echo image is down-sampled at different scales to obtain an echo image pyramid, which includes several echo image blocks; each echo image block is sparsely encoded using a preset sparse dictionary to obtain each sparse coefficient matrix; any sparse coefficient matrix and the preset sparse dictionary are used as sparse dictionary optimization units, and the preset sparse dictionary and the sparse coefficient matrix are self-optimized based on these units until convergence to obtain each adjusted sparse dictionary; finally, these adjusted sparse dictionaries are fused to obtain an optimized sparse dictionary. This method can effectively capture the multi-level features of the image through the combination of multi-scale sampling, sparse coding and self-optimizing dictionary, improve the accuracy and efficiency of sparse representation, and finally obtain a high-quality optimized sparse dictionary, which significantly improves the reconstruction effect and detail retention of ultrasound images.

[0136] In an exemplary embodiment, Figure 8 As shown, after the step of sparse coding the ultrasound enhanced echo image to obtain the sparse features of each echo data, the process includes step 802 to step 808. Among them:

[0137] Step 802: Acquire various real-time data sparse features corresponding to the real-time ultrasound image of the target object.

[0138] The real-time data sparse features may be an efficient representation extracted from the ultrasound image by using sparse coding technology during the real-time acquisition of the ultrasound image.

[0139] Specifically, the real-time ultrasound image collected in real time is preprocessed (such as denoising and enhancement). Then the preprocessed real-time ultrasound image is divided into several small blocks, and a preset sparse dictionary is used to perform sparse coding on each image block to calculate the corresponding sparse coefficient matrix. The sparse coefficient matrices of all image blocks are collected to obtain the sparse features of each real-time data of the real-time ultrasound image of the target object.

[0140] Step 804: Identify the different data sparse features between each real-time data sparse feature and each echo data sparse feature.

[0141] The distinguishing data sparse features may be data obtained by comparing the sparse features of two data sets (such as real-time data and echo data) to identify and extract significant differences between them.

[0142] Specifically, the sparse features of real-time data and echo data are aligned and matched, and then the differences between them are calculated. By comparing the sparse coefficients of each corresponding element, the parts with significant differences are found, and these sparse coefficients with significant differences are extracted as the sparse features of distinguishing data.

[0143] Step 806: Determine dictionary extension data according to the sparse features of the distinguishing data.

[0144] The dictionary extension data may be dictionary elements generated by analyzing and extracting key features and patterns that are not fully represented by existing dictionaries and used to expand or adjust existing dictionaries.

[0145] Specifically, the sparse features of the distinguishing data are analyzed, key features and patterns in the distinguishing data sparse features that are not fully represented by the preset sparse dictionary are identified, and these key features and patterns that are not fully represented by the current dictionary are used to generate new dictionary elements or adjusted dictionary elements as dictionary extension data corresponding to the preset sparse dictionary, which are used to adjust and expand the dictionary elements of the preset sparse dictionary to better include and represent these key features.

[0146] Step 808: Expand the preset sparse dictionary according to the dictionary expansion data to obtain an optimized sparse dictionary.

[0147] Specifically, the new dictionary elements or adjusted dictionary elements generated in the dictionary extension data are added to the preset sparse dictionary, and through the iterative optimization process, the newly added dictionary elements and the original elements are adjusted and optimized to better represent the data features and patterns, thereby obtaining an optimized sparse dictionary.

[0148] In this embodiment, by acquiring each real-time data sparse feature corresponding to the real-time ultrasound image of the target object; identifying the distinguishing data sparse features between each real-time data sparse feature and the echo data sparse feature; determining the dictionary extension data according to these distinguishing data sparse features; and expanding the preset sparse dictionary according to the dictionary extension data, an optimized sparse dictionary is obtained. This method can realize personalized dictionary optimization for the target object, improve the ability to capture and express real-time ultrasound image features, thereby improving the accuracy and speed of image reconstruction, enhancing the effect of real-time imaging, and ensuring more timely and accurate diagnostic support in the application.

[0149] In an exemplary embodiment, Fig. 9 As shown, the defect area is identified from the sparsely reconstructed echo image to obtain the object defect ultrasonic image, including steps 902 to 908. Among them:

[0150] Step 902: extract features from the sparse reconstructed echo image to obtain global reconstructed echo features and local reconstructed echo features.

[0151] The global reconstruction echo feature may refer to information extracted from the sparsely reconstructed echo image that can describe the overall features of the image. These features include the overall shape, texture, contrast, and main mode of the image.

[0152] The local reconstructed echo features can be detailed information and features extracted from a specific area of ​​the sparse reconstructed echo image. By dividing the image into several local areas, feature extraction is performed on each area to capture the details, texture and local patterns in the area.

[0153] Specifically, image processing technology or deep learning methods are used to extract global features of the image, capture the overall shape, texture and important patterns, and obtain global reconstructed echo features. Then, according to the image reconstruction requirements of the sparse reconstructed echo image, the image is divided into several local areas (or several specific areas), and feature extraction is performed on each area to capture local details and local patterns, and obtain local reconstructed echo features.

[0154] Step 904: according to the global reconstructed echo feature and each local reconstructed echo feature, a local detection threshold corresponding to each local reconstructed echo feature is set.

[0155] Among them, the local detection threshold can be a numerical limit set for a specific local area in image analysis for identifying and detecting anomalies or features. This threshold is based on the feature information of the local area and is set with reference to the global image features to ensure that normal and abnormal patterns or features can be effectively distinguished in the local area.

[0156] Specifically, the global reconstructed echo features are analyzed to determine the overall characteristics and baseline level of the image, and then the local detection threshold of each local reconstructed echo feature is set based on the relative relationship between each local reconstructed echo feature and the overall characteristics and baseline level. The setting of the local detection threshold needs to consider the overall impact of the global reconstructed echo features and the detailed differences of the local features, ensuring that each local detection threshold can effectively capture local anomalies.

[0157] Step 906: Determine the initial point of the defect area corresponding to each local reconstructed echo feature according to the difference between the object defect ultrasonic image and the ultrasonic echo image.

[0158] The initial point of the defect area may be a point with the largest difference in an area significantly different from a normal image, identified by comparing the difference between the object defect ultrasound image and the standard ultrasound echo image during image analysis.

[0159] Specifically, the object defect ultrasound image and the ultrasound echo image are registered to ensure that the object defect ultrasound image and the ultrasound echo image are aligned in the same coordinate system; the difference between the object defect ultrasound image and the ultrasound echo image is calculated by the image difference method, and significantly different areas are identified; these difference areas are combined with each local reconstructed echo feature to perform traversal calculations, determine the most significant difference as the initial point of the defect area, and mark the specific location where the defect may exist.

[0160] Step 908, using the initial points of each defect area as the growth points of the region growing algorithm, using the region growing algorithm, according to each local detection threshold and the corresponding initial point of the defect area, identifying the defect area in the sparse reconstructed echo image, and obtaining the object defect ultrasonic image.

[0161] The region growing algorithm may be an algorithm that starts with a (seed point) and gradually adds neighboring pixels to the region according to a similarity criterion (such as grayscale, color or texture).

[0162] Specifically, the initial points of each defect area are used as the growth points of the region growing algorithm. The region growing algorithm is used. According to each local detection threshold and the corresponding initial point of the defect area, in the sparsely reconstructed echo image, each initial point of the defect area is used as the starting condition, and the neighboring pixels are compared with the initial point of the defect area. If the neighboring pixel value meets the condition of the local detection threshold, it is included in the defect area, and its neighboring pixels are continued to be checked. This process is repeated until no more pixels meet the conditions, and finally the complete defect area is identified; each defect area is combined and marked in the sparsely reconstructed echo image to obtain the object defect ultrasonic image.

[0163] In this embodiment, the global and local reconstructed echo features are obtained by extracting features from the sparse reconstructed echo image; the local detection threshold is set according to these features; the initial point of the defect area is determined according to the difference between the ultrasonic image of the object defect and the ultrasonic echo image; the initial point is used as the growth point of the regional growing algorithm, and the defect area is identified by using the regional growing algorithm and based on the local detection threshold, and finally the ultrasonic image of the object defect is obtained. This method can accurately locate and identify the defect area in the image, ensure high sensitivity and accuracy of detection, help to implement timely intervention, and improve the reliability and efficiency of practical applications.

[0164] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0165] Based on the same inventive concept, the embodiment of the present application also provides an ultrasonic image processing device for implementing the ultrasonic image processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more ultrasonic image processing device embodiments provided below can refer to the above limitations on an ultrasonic image processing method, and will not be repeated here.

[0166] In an exemplary embodiment, Fig.10As shown, an ultrasonic image processing device is provided, comprising: an echo image acquisition module 1002, an ultrasonic image reconstruction module 1004, an ultrasonic image enhancement module 1006 and an object defect recognition module 1008, wherein:

[0167] The echo image acquisition module 1002 is used to acquire an ultrasonic echo image of the target object; the ultrasonic echo image is obtained by fusing scan images obtained by different scanning methods of the target object;

[0168] The ultrasonic image reconstruction module 1004 is used to perform multi-scale super-resolution reconstruction on the ultrasonic echo image according to the ultrasonic echo feature data at different scales of the ultrasonic echo image to obtain a super-resolution ultrasonic image;

[0169] The ultrasonic image enhancement module 1006 is used to enhance the super-resolution ultrasonic image by using an image enhancement adversarial network to obtain an ultrasonic enhanced echo image;

[0170] The ultrasonic image reconstruction module 1004 is further used to reconstruct the image details of the ultrasonic enhanced echo image according to the ultrasonic echo sparse characteristics of the ultrasonic enhanced echo image to obtain a sparse reconstructed echo image;

[0171] The object defect recognition module 1008 is used to recognize defect areas from the sparse reconstructed echo image to obtain an object defect ultrasonic image; the object defect ultrasonic image is used to evaluate defects in materials or structures.

[0172] In one embodiment, the ultrasonic image reconstruction module 1004 is also used to extract the features of the ultrasonic echo image at different scales to obtain characteristic data of each ultrasonic echo; perform feature selection on each ultrasonic echo characteristic data to obtain an ultrasonic echo characteristic graph; use a cascade reconstruction network of several stages determined by the ultrasonic echo characteristic graph to perform several cascade reconstructions on the ultrasonic echo characteristic graph to obtain a super-resolution ultrasonic image.

[0173] In one embodiment, the ultrasonic image reconstruction module 1004 is also used to input the ultrasonic echo feature map into the cascade network for resolution improvement processing based on the preset cascade reconstruction number determined by the ultrasonic echo feature map to obtain a resolution-enhanced image; fuse the ultrasonic echo feature map and the resolution-enhanced image to obtain a resolution-fused image; use the multi-level optimization function determined by the resolution-fused image to perform multi-level optimization on the resolution-fused image to obtain a cascaded reconstructed image and an actual cascade reconstruction number; the multi-level optimization function is composed of a perception optimization function, and / or a pixel optimization function, and / or a structural similarity optimization function; when the actual cascade reconstruction number is less than the preset cascade reconstruction number, use the cascaded reconstructed image as the ultrasonic echo feature map, return to execute the step of inputting the ultrasonic echo feature map into the cascade network for resolution improvement processing to obtain a resolution-enhanced image, until the actual cascade reconstruction number is equal to the preset cascade reconstruction number, and obtain a super-resolution ultrasonic image.

[0174] In one embodiment, the ultrasound image reconstruction module 1004 is also used to perform multi-level optimization on the resolution fused image according to the perception optimization function, or / and the pixel optimization function, or / and the structural similarity optimization function to obtain a resolution-optimized ultrasound image; adjust the weights of the perception optimization function, or / and the pixel optimization function, or / and the structural similarity optimization function according to the resolution difference information between the resolution-optimized ultrasound image and the resolution-fused image; perform multi-level optimization on the resolution-optimized ultrasound image according to the adjusted perception optimization function, or / and the adjusted pixel optimization function, or / and the adjusted structural similarity optimization function to obtain a cascaded reconstructed image.

[0175] In one embodiment, the ultrasound image reconstruction module 1004 is also used to perform sparse encoding on the ultrasound enhanced echo image to obtain sparse features of each echo data; adaptively optimize the preset sparse dictionary according to the ultrasound enhanced echo image to obtain an optimized sparse dictionary; and fuse the sparse features of each echo data according to the optimized sparse dictionary to obtain a sparse reconstructed echo image.

[0176] In one embodiment, the ultrasound image reconstruction module 1004 is also used to perform step-down sampling of ultrasound enhanced echo images at different scales to obtain an echo image pyramid; the echo image pyramid includes a plurality of echo image blocks; each echo image block is sparsely encoded using a preset sparse dictionary to obtain each sparse coefficient matrix; any sparse coefficient matrix and a preset sparse dictionary are used as sparse dictionary optimization units, and based on each sparse dictionary optimization unit, the preset sparse dictionary and the sparse coefficient matrix are self-optimized respectively until the preset sparse dictionary and the sparse coefficient matrix converge to obtain each adjusted sparse dictionary; and each adjusted sparse dictionary is merged to obtain an optimized sparse dictionary.

[0177] In one embodiment, the ultrasound image reconstruction module 1004 is also used to obtain each real-time data sparse feature corresponding to the real-time ultrasound image of the target object; identify the distinguishing data sparse features between each real-time data sparse feature and each echo data sparse feature; determine the dictionary extension data according to the distinguishing data sparse features; and expand the preset sparse dictionary according to the dictionary extension data to obtain an optimized sparse dictionary.

[0178] In one embodiment, the object defect recognition module 1008 is also used to extract features from the sparse reconstructed echo image to obtain global reconstructed echo features and local reconstructed echo features; set local detection thresholds corresponding to each local reconstructed echo feature according to the global reconstructed echo features and each local reconstructed echo feature; determine the initial points of the defect areas corresponding to each local reconstructed echo feature according to the difference between the object defect ultrasound image and the ultrasound echo image; use the initial points of each defect area as the growth points of the regional growing algorithm, adopt the regional growing algorithm, and identify the defect areas in the sparse reconstructed echo image according to the local detection thresholds and the corresponding initial points of the defect area to obtain the object defect ultrasound image.

[0179] Each module in the above-mentioned ultrasonic image processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0180] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.11 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store server data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an ultrasonic image processing method is implemented.

[0181] Those skilled in the art will understand that Fig.11The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0182] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0183] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0184] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.

[0185] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0186] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0187] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0188] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An ultrasonic image processing method, characterized in that: The method comprises: Acquiring an ultrasonic echo image of a target object; the ultrasonic echo image is obtained by fusing scan images obtained by different scanning methods of the target object; According to the ultrasonic echo feature data of the ultrasonic echo image at different scales, the ultrasonic echo image is reconstructed with multi-scale super-resolution to obtain a super-resolution ultrasonic image; Using an image enhancement adversarial network, the super-resolution ultrasound image is enhanced to obtain an ultrasound enhanced echo image; According to the ultrasonic echo sparse characteristics of the ultrasonic enhanced echo image, image details are reconstructed on the ultrasonic enhanced echo image to obtain a sparse reconstructed echo image; A defect area is identified from the sparsely reconstructed echo image to obtain an object defect ultrasonic image; the object defect ultrasonic image is used to evaluate defects in a material or structure.

2. The method according to claim 1, characterized in that The method of performing multi-scale super-resolution reconstruction on the ultrasonic echo image according to the ultrasonic echo feature data of the ultrasonic echo image at different scales to obtain a super-resolution ultrasonic image includes: Extracting features of the ultrasonic echo image at different scales to obtain feature data of each ultrasonic echo; Performing feature selection on each of the ultrasonic echo feature data to obtain an ultrasonic echo feature graph; The super-resolution ultrasound image is obtained by using a cascade reconstruction network of several stages determined by the ultrasound echo characteristic map and performing cascade reconstruction on the ultrasound echo characteristic map for several times.

3. The method according to claim 2, characterized in that The method of using the cascade reconstruction network of several stages determined by the ultrasonic echo characteristic graph to perform cascade reconstruction on the ultrasonic echo characteristic graph several times to obtain the super-resolution ultrasonic image comprises: Based on a preset number of cascade reconstructions determined by the ultrasonic echo characteristic graph, the ultrasonic echo characteristic graph is input into a cascade network for resolution enhancement processing to obtain a resolution enhanced image; Fusing the ultrasonic echo characteristic image with the resolution-enhanced image to obtain a resolution-fused image; Using the multi-level optimization function determined by the resolution fusion image, multi-level optimization is performed on the resolution fusion image to obtain a cascaded reconstructed image and an actual cascaded reconstruction number; the multi-level optimization function is composed of a perception optimization function, and / or a pixel optimization function, and / or a structural similarity optimization function; When the actual number of cascade reconstructions is less than the preset number of cascade reconstructions, the cascade reconstructed image is used as the ultrasonic echo feature map, and the step of inputting the ultrasonic echo feature map into the cascade network for resolution improvement processing to obtain a resolution-enhanced image is returned to be executed until the actual number of cascade reconstructions is equal to the preset number of cascade reconstructions to obtain the super-resolution ultrasonic image.

4. The method according to claim 3, characterized in that The multi-level optimization function determined by the resolution fusion image is used to perform multi-level optimization on the resolution fusion image to obtain a cascaded reconstructed image, including: According to the perception optimization function, or / and the pixel optimization function, or / and the structural similarity optimization function, the resolution fusion image is optimized at multiple levels to obtain a resolution optimized ultrasound image; According to the resolution difference information between the resolution-optimized ultrasound image and the resolution-fused image, respectively adjusting the weights of the perception optimization function, or / and the pixel optimization function, or / and the structural similarity optimization function; According to the adjusted perception optimization function, or / and the adjusted pixel optimization function, or / and the adjusted structural similarity optimization function, the resolution-optimized ultrasound image is optimized at multiple levels to obtain the cascade reconstructed image.

5. The method according to claim 1, characterized in that The ultrasonic echo sparse characteristics of the ultrasonic enhanced echo image, Reconstructing the image details of the ultrasonic enhanced echo image to obtain a sparse reconstructed echo image includes: Performing sparse coding on the ultrasonic enhanced echo image to obtain sparse features of each echo data; According to the ultrasonic enhanced echo image, adaptively optimizing a preset sparse dictionary to obtain an optimized sparse dictionary; According to the optimized sparse dictionary, the sparse features of each echo data are fused to obtain the sparse reconstructed echo image.

6. The method according to claim 5, characterized in that The method of adaptively optimizing a preset sparse dictionary according to the ultrasound enhanced echo image to obtain an optimized sparse dictionary includes: The ultrasound enhanced echo image is downsampled in different scales to obtain an echo image pyramid; the echo image pyramid includes a plurality of echo image blocks; Perform sparse coding on each of the echo image blocks using the preset sparse dictionary to obtain each sparse coefficient matrix; Using any of the sparse coefficient matrices and the preset sparse dictionary as sparse dictionary optimization units, and based on each of the sparse dictionary optimization units, performing self-optimization on the preset sparse dictionary and the sparse coefficient matrix respectively until the preset sparse dictionary and the sparse coefficient matrix converge to obtain each adjusted sparse dictionary; The adjusted sparse dictionaries are merged to obtain the optimized sparse dictionary.

7. The method according to claim 5, characterized in that After the step of sparsely encoding the ultrasound enhanced echo image to obtain sparse features of each echo data, the method further includes: Acquire sparse features of real-time data corresponding to the real-time ultrasound image of the target object; Identifying a different data sparse feature between each of the real-time data sparse features and each of the echo data sparse features; Determining dictionary extension data according to the sparse characteristics of the distinguishing data; The preset sparse dictionary is expanded according to the dictionary expansion data to obtain an optimized sparse dictionary.

8. The method according to claim 1, characterized in that The step of identifying a defect area from the sparsely reconstructed echo image to obtain an ultrasonic image of the object defect includes: Extracting features from the sparsely reconstructed echo image to obtain global reconstructed echo features and local reconstructed echo features; According to the global reconstruction echo feature and each of the local reconstruction echo features, setting a local detection threshold corresponding to each of the local reconstruction echo features; Determining the initial point of the defect area corresponding to each of the local reconstructed echo features according to the difference between the object defect ultrasonic image and the ultrasonic echo image; Taking the initial points of each defect area as the growth points of the region growing algorithm, the region growing algorithm is used to identify the defect area in the sparse reconstructed echo image according to each local detection threshold and the corresponding initial point of the defect area, so as to obtain the ultrasonic image of the object defect.

9. An ultrasonic image processing device, characterized in that: The device comprises: An echo image acquisition module, used for acquiring an ultrasonic echo image of a target object; the ultrasonic echo image is obtained by fusing scan images obtained by different scanning methods of the target object; An ultrasonic image reconstruction module, used for performing multi-scale super-resolution reconstruction on the ultrasonic echo image according to ultrasonic echo feature data of the ultrasonic echo image at different scales to obtain a super-resolution ultrasonic image; An ultrasonic image enhancement module, used to enhance the super-resolution ultrasonic image using an image enhancement adversarial network to obtain an ultrasonic enhanced echo image; The ultrasonic image reconstruction module is further used to reconstruct the image details of the ultrasonic enhanced echo image according to the ultrasonic echo sparse characteristics of the ultrasonic enhanced echo image to obtain a sparse reconstructed echo image; The object defect recognition module is used to identify the defect area from the sparse reconstructed echo image to obtain an object defect ultrasonic image; the object defect ultrasonic image is used to evaluate defects in materials or structures.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images