Automatic repairing method for ultrasonic image

By identifying and classifying defective factors in ultrasound images, using mask plates for repair, the problem of noise interference in ultrasound examination is solved, and image quality and recognition capabilities of artificial intelligence models are improved.

CN118986399APending Publication Date: 2024-11-22SHANGHAI SOUNDWISE TECHNOLOGY CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411043773.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

During ultrasound examination, due to factors such as the deviation of the scanning process, external vibration interference and the patient's own movement, large areas of noise appear in areas with larger areas such as the abdominal cavity, which affects the quality of ultrasound images, and thus affects the training of artificial intelligence models and Identification process.

Method used

By identifying the area to be repaired, the interval echo sequence is extracted, defect factors are classified and determined, such as scanning offset, vibration and medium changes, the corresponding mask plate is retrieved for repair, eliminate interference components, and automatic image repair is achieved.

Benefits of technology

It effectively repairs local defects in ultrasound images, improves image quality, and enhances the training and recognition capabilities of artificial intelligence models for ultrasound images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118986399A_ABST
    Figure CN118986399A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of ultrasonic image processing, in particular to an automatic repairing method of an ultrasonic image, which comprises the following steps: identifying a to-be-repaired image to obtain a to-be-repaired area, and intercepting an interval echo sequence from a complete ultrasonic echo sequence for the to-be-repaired area; classifying the interval echo sequence and the to-be-repaired area to determine defect factors; the defect factors comprise at least one of scanning offset, vibration and medium change. And calling a corresponding mask plate according to the defect factors, and repairing the to-be-repaired area according to the mask plate. In order to solve the problem that in the prior art, when ultrasonic inspection equipment synthesizes an ultrasonic image of a large area, local defects exist, and artificial intelligence model training is affected, a process of extracting a to-be-repaired area of the local defects and analyzing defect factors of the to-be-repaired area according to an echo sequence is introduced, the defect factors causing the imaging defects are determined, and the defect repairing efficiency is improved. And the corresponding mask plate is called for repairing, so that a relatively good image repairing process is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of ultrasonic image processing, and in particular to an automatic repair method for ultrasonic images. Background Art

[0002] Palmtop ultrasound equipment, also known as handheld ultrasound, is a palm-sized, easy-to-control medical ultrasound device that combines a host and a probe into one. It is usually composed of a probe, a processor, a display, and a battery. It has the characteristics of small size, light weight, and low power consumption.

[0003] For example, Chinese patent CN201710560949.0 discloses a handheld three-dimensional ultrasound imaging system and method, including a handheld ultrasound probe for scanning and acquiring ultrasound images; a display, control and processing terminal connected to the handheld ultrasound probe by wired or wireless connection. It also includes: a handheld three-dimensional spatial positioning system, which is connected to the handheld ultrasound probe and moves with the movement of the handheld ultrasound probe, and is connected to the display, control and processing terminal by wired or wireless connection, and is used to independently locate the three-dimensional position of the handheld ultrasound probe. By applying the handheld three-dimensional ultrasound imaging system and method, the huge spatial positioning system in the existing three-dimensional ultrasound imaging system is transformed into a portable and ready-to-use spatial positioning system, so that handheld three-dimensional ultrasound imaging can be widely used.

[0004] However, in the actual implementation process, the inventors found that during the ultrasonic examination, due to the influence of factors such as scanning process offset, external vibration interference, and the patient's own movement, during the ultrasonic examination of large areas such as the abdominal cavity, large areas of noise are likely to appear in some areas, resulting in the inability to be used in the training and recognition process of the artificial intelligence model. Summary of the invention

[0005] In view of the above problems existing in the prior art, a method for automatically repairing ultrasonic images is now provided.

[0006] The specific technical solutions are as follows:

[0007] An automatic repair method for ultrasonic images, comprising:

[0008] Step S1: Identify the area to be repaired based on the image to be repaired, and extract interval echo sequences from the complete ultrasonic echo sequence for the area to be repaired;

[0009] Step S2: classifying the interval echo sequence and the area to be repaired to determine defect factors;

[0010] The defect factors include at least one of scanning deviation, vibration, and medium change.

[0011] Step S3: Retrieving a corresponding mask plate according to the defect factor, and repairing the area to be repaired according to the mask plate.

[0012] On the other hand, the step S1 comprises:

[0013] Step S11: performing sliding window segmentation on the image to be restored according to a preset step length to obtain a plurality of segmented images;

[0014] Step S12: using a pre-recognition model to respectively recognize the segmented images, and marking the segmented images that cannot be correctly recognized as the areas to be repaired;

[0015] Step S13: extracting the interval echo sequence from the complete ultrasonic echo sequence according to the position of the to-be-repaired area in the to-be-repaired image.

[0016] On the other hand, the step S13 comprises:

[0017] Step S131: obtaining a pixel set corresponding to the to-be-repaired area in the to-be-repaired image;

[0018] Step S132: determining a reconstruction timing according to a corresponding position of each pixel point in the pixel set in the image to be repaired;

[0019] Step S133: respectively extracting the interval echo sequence from the complete ultrasound echo sequence according to the reconstruction timing.

[0020] On the other hand, the step S2 comprises:

[0021] Step S21: extracting signal features from the interval echo sequence, and extracting image features from the area to be repaired;

[0022] Step S22: Assembling the signal feature and the image feature to obtain a feature vector, and inputting the feature vector into a vector machine for classification to obtain the defect factor.

[0023] On the other hand, in step S21, the signal feature extraction method includes:

[0024] Step A211: performing fast Fourier processing on the interval echo sequence to obtain a plurality of frequency domain signals;

[0025] Step A212: acquiring an adjacent sequence of the interval echo sequence in the complete ultrasonic echo sequence, and determining an ultrasonic frequency domain signal according to the adjacent sequence;

[0026] Step A213: subtract the frequency domain signal from the ultrasonic frequency domain signal to separate and obtain the signal feature.

[0027] On the other hand, in step S21, the method for extracting image features includes:

[0028] Step B211: extracting a neighboring image of the area to be repaired, and searching for a pre-constructed sample image template according to the neighboring image;

[0029] Step B212: extracting a regional sample image corresponding to the area to be repaired from the sample image template;

[0030] Step B213: subtract the area to be repaired from the area sample image to separate and obtain the image features.

[0031] On the other hand, the step S3 comprises:

[0032] Step S31: calling the mask plate according to the defect factor, and determining the defect intensity of the defect factor;

[0033] Step S32: correcting the mask plate according to the defect strength to obtain a corrected mask plate;

[0034] Step S33: repairing the area to be repaired according to the correction mask.

[0035] On the other hand, the step S33 comprises:

[0036] Step S331: eliminating interference components in the area to be repaired according to the correction mask to obtain a filtered image;

[0037] Step S332: stacking and synthesizing the area to be repaired according to the sample image corresponding to the area to be repaired to obtain a fitting area image;

[0038] Step S333: synthesizing the fitting area image and the image to be restored.

[0039] The above technical solution has the following advantages or beneficial effects:

[0040] In view of the problem that ultrasonic inspection equipment in the prior art has local defects when synthesizing ultrasonic images of a larger area, which affects the training of artificial intelligence models, this solution introduces a process of extracting the area to be repaired with local defects and analyzing its defect factors according to the echo sequence. The defect factors that cause the imaging defects are determined, and the corresponding mask plate is called for repair, thereby achieving a better image restoration process. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The embodiments of the present invention will be described more fully with reference to the attached drawings, which are provided for illustration and description only and are not intended to limit the scope of the present invention.

[0042] Figure 1 It is an overall schematic diagram of an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of step S1 in an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of step S13 in an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of step S2 in an embodiment of the present invention;

[0046] Figure 5 A schematic diagram of signal feature extraction in an embodiment of the present invention;

[0047] Figure 6 Schematic diagram of image feature extraction in an embodiment of the present invention;

[0048] Figure 7 This is a schematic diagram of step S3 in an embodiment of the present invention;

[0049] Figure 8 Schematic diagram of step S33 in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0052] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0053] The present invention comprises:

[0054] An automatic repair method for ultrasound images, such as Figure 1 As shown, including:

[0055] Step S1: Identify the area to be repaired based on the image to be repaired, and extract interval echo sequences from the complete ultrasonic echo sequence for the area to be repaired;

[0056] Step S2: Classify the interval echo sequence and the area to be repaired to determine the defect factors; the defect factors include: at least one of scanning deviation, vibration, and medium change

[0057] Step S3: retrieve the corresponding mask plate according to the defect factor, and repair the area to be repaired according to the mask plate.

[0058] Specifically, in order to address the problem that ultrasonic inspection equipment in the prior art has local defects when synthesizing ultrasonic images of a larger area, which affects the training of the artificial intelligence model, this solution introduces the extraction of the area to be repaired for local defects, and performs signal analysis according to the echo sequence, so as to determine the defect factors that cause the imaging defects, and call the corresponding mask plate for repair, thereby achieving a better image restoration process.

[0059] In one embodiment, Figure 2 As shown, step S1 includes:

[0060] Step S11: performing sliding window segmentation according to a preset step length on the image to be restored to obtain a plurality of segmented images;

[0061] Step S12: using the pre-recognition model to recognize the segmented images respectively, and marking the segmented images that cannot be correctly recognized as areas to be repaired;

[0062] Step S13: extracting an interval echo sequence from the complete ultrasonic echo sequence according to the position of the to-be-repaired area in the to-be-repaired image.

[0063] Specifically, in order to realize the extraction of the area to be repaired, in this embodiment, the image to be repaired is first randomly segmented using sliding windows of different sizes according to a preset step size to form a plurality of segmented images of different sizes and positions. Then, the segmented images are respectively recognized using a pre-recognition model, which is an image recognition model used to extract image features through a series of convolutional networks and predict which part of the body the local ultrasound image corresponds to. Subsequently, the segmented images that cannot be correctly identified are marked as the area to be repaired, which is usually reflected as a rectangle corresponding to the position of the sliding window. Subsequently, according to the position of the area to be repaired in the image to be repaired, an interval echo sequence is intercepted from the complete ultrasound echo sequence.

[0064] In one embodiment, Figure 3 As shown, step S13 includes:

[0065] Step S131: obtaining a pixel set corresponding to the to-be-repaired area in the to-be-repaired image;

[0066] Step S132: determining the reconstruction timing according to the corresponding position of each pixel point in the pixel set in the image to be repaired;

[0067] Step S133: extracting interval echo sequences from the complete ultrasound echo sequence according to the reconstruction timing.

[0068] Specifically, in order to achieve a better extraction effect, in this embodiment, for the area to be repaired, the pixels included in the sliding window corresponding to the area to be repaired are extracted as a pixel set. Since a large ultrasonic image is usually scanned and reconstructed in rows / columns according to a specific trajectory during scanning, the pixels can be traversed in the direction of the trajectory, and the signal content of the pixels involved and their sources can be determined during the traversal process, and multiple groups of interval echo sequences can be intercepted in the complete ultrasonic echo sequence for signal analysis.

[0069] In one embodiment, Figure 4 As shown, step S2 includes:

[0070] Step S21: extracting signal features from the interval echo sequence, and extracting image features from the area to be repaired;

[0071] Step S22: Assemble the signal features and the image features to obtain a feature vector, and input the feature vector into a vector machine for classification to obtain defect factors.

[0072] Specifically, in order to achieve a better discrimination effect, in this embodiment, Fourier transform processing is first performed on the interval echo sequence to extract signal features from the interval echo sequence; and image features are extracted from the area to be repaired through a processing algorithm. Then, the signal features and image features can be assembled together to obtain a two-dimensional vector, which is input into a pre-trained vector machine model and classified by the vector machine model to determine the type of defect factor that may cause the defect.

[0073] In one embodiment, Figure 5 As shown, in step S21, the signal feature extraction method includes:

[0074] Step A211: performing fast Fourier processing on the interval echo sequence to obtain multiple frequency domain signals;

[0075] Step A212: acquiring an adjacent sequence of the interval echo sequence in the complete ultrasonic echo sequence, and determining an ultrasonic frequency domain signal according to the adjacent sequence;

[0076] Step A213: Subtract the frequency domain signal from the ultrasonic frequency domain signal to separate and obtain signal features.

[0077] Specifically, in order to achieve a better signal feature extraction process, in this embodiment, fast Fourier processing is first performed on the interval echo sequence to extract frequency domain signals in different frequency domains. Since the energy of ultrasonic scanning is usually concentrated in a narrow frequency band, the ultrasonic echo component in the echo can be easily removed through time-frequency domain transformation, leaving only the signal noise part.

[0078] In order to accurately remove the ultrasonic echo components in the echo, in this embodiment, for each group of interval echo sequences, adjacent sequences of a certain length are determined in the complete ultrasonic echo sequence according to the time sequence, and then the adjacent sequences are averaged to determine the frequency band and amplitude where the ultrasonic echoes are mainly concentrated as the ultrasonic frequency domain signal.

[0079] Finally, the frequency domain signal is subtracted from the ultrasonic frequency domain signal to separate and obtain the signal features.

[0080] In one embodiment, Figure 6 As shown, in step S21, the method for extracting image features includes:

[0081] Step B211: extracting neighboring images from the area to be repaired, and searching for a pre-built sample image template according to the neighboring images;

[0082] Step B212: extracting a regional sample image corresponding to the area to be repaired from the sample image template;

[0083] Step B213: subtract the area to be repaired from the area sample image to separate and obtain image features.

[0084] Specifically, in order to achieve a better separation effect of image features, in this embodiment, for the area to be repaired with noise components, eight surrounding images are extracted as neighboring area images in a nine-square grid manner.

[0085] At the same time, before starting the processing, a large number of medical data images have been collected for the panoramic images that need to be collected, and the mean value of the medical data images has been calculated to construct a sample image template. The sample image template can be regarded as the morphology of a standard tissue under normal circumstances.

[0086] Subsequently, the sample image template is aligned with the image to be repaired and mapped to the same coordinate system. The regional sample image corresponding to the area to be repaired is cut out from the sample image template. Finally, the area to be repaired and the regional sample image are subtracted to separate the image features corresponding to the noise components.

[0087] In one embodiment, Figure 7 As shown, step S3 includes:

[0088] Step S31: calling a mask plate according to a defect factor and determining the defect intensity of the defect factor;

[0089] Step S32: correcting the mask plate according to the defect intensity to obtain a corrected mask plate;

[0090] Step S33: repairing the area to be repaired according to the correction mask.

[0091] Specifically, in order to achieve a better image restoration process, in this embodiment, different mask plates are made for different defect factors. Among them, the defect factors mainly include:

[0092] Scanning deviation: During the scanning process, the direction and angle of the handheld probe rotate when it moves in line with the patient's body surface, resulting in incomplete acquisition of echoes;

[0093] Vibration: During the scanning process, sudden external vibrations may cause echo superposition and blurred imaging.

[0094] The medium changes, during the scanning process, the echo parameters change due to factors such as uneven application of coupling agent.

[0095] The above factors will cause certain artifacts and noise in the subsequent imaging process. Therefore, around the above factors, the inventors obtained the noise components generated by different defect factors through experiments in advance and constructed a mask. In the actual processing process, the mask is adjusted according to the defect intensity.

[0096] Taking scanning offset as an example, this process will be reflected in the characteristics of the signal as a significant extension of the echo time, which will cause the imaging part to shift in position. In this regard, by performing time domain analysis on the signal sequence, the ratio of its phase offset to the standard offset of the mask can be determined, and then the mask can be adjusted to make its phase offset consistent with the actual scanning situation;

[0097] Taking vibration as an example, this part is mainly manifested as a group of signals with wider bandwidth at the signal sequence level. By performing frequency domain analysis on it, it is easy to obtain the ratio and frequency band of its amplitude relative to the signal amplitude of the mask plate. According to the two, the mask plate is adjusted to make the noise it produces consistent with the defect factors.

[0098] Changes in the medium usually cause the frequency of the echo sequence to shift. By performing frequency domain analysis, it is easy to obtain the ratio and frequency band of its amplitude relative to the signal amplitude of the mask plate. Based on the two, the mask plate is adjusted to make the noise it produces consistent with the defect factors.

[0099] The noise components in the area to be repaired can be removed according to the correction mask, thereby realizing the repair process.

[0100] In one embodiment, Figure 8 As shown, step S33 includes:

[0101] Step S331: eliminating interference components in the area to be repaired according to the correction mask to obtain a filtered image;

[0102] Step S332: stacking and synthesizing the area to be repaired according to the sample images corresponding to the area to be repaired to obtain a fitting area image;

[0103] Step S333: synthesize the fitting area image and the image to be restored.

[0104] Specifically, in order to achieve a better restoration effect, in this embodiment, after eliminating the interference components in the area to be restored according to the correction mask, the sample image previously used for comparison is intercepted, and the image corresponding to the area to be restored is obtained to obtain a local image for restoration, and the local image and the image of the area to be restored are stacked to obtain a fitting area image. Among them, the sample image has been classified according to the disease process and the patient's personal attributes so that the local part is close to the area to be restored. Finally, the fitting area image is synthesized with the image to be restored to obtain a close image.

[0105] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for automatically repairing an ultrasonic image, characterized in that: include: Step S1: Identify the area to be repaired based on the image to be repaired, and extract interval echo sequences from the complete ultrasonic echo sequence for the area to be repaired; Step S2: classifying the interval echo sequence and the area to be repaired to determine defect factors; The defect factors include at least one of scanning deviation, vibration, and medium change. Step S3: Retrieving a corresponding mask plate according to the defect factor, and repairing the area to be repaired according to the mask plate.

2. The automatic repair method according to claim 1, characterized in that: The step S1 comprises: Step S11: performing sliding window segmentation on the image to be restored according to a preset step length to obtain a plurality of segmented images; Step S12: using a pre-recognition model to respectively recognize the segmented images, and marking the segmented images that cannot be correctly recognized as the areas to be repaired; Step S13: extracting the interval echo sequence from the complete ultrasonic echo sequence according to the position of the to-be-repaired area in the to-be-repaired image.

3. The automatic repair method according to claim 2, characterized in that: The step S13 comprises: Step S131: obtaining a pixel set corresponding to the to-be-repaired area in the to-be-repaired image; Step S132: determining a reconstruction timing according to a corresponding position of each pixel point in the pixel set in the image to be repaired; Step S133: respectively extracting the interval echo sequence from the complete ultrasound echo sequence according to the reconstruction timing.

4. The automatic repair method according to claim 1, characterized in that: The step S2 comprises: Step S21: extracting signal features from the interval echo sequence, and extracting image features from the area to be repaired; Step S22: Assembling the signal feature and the image feature to obtain a feature vector, and inputting the feature vector into a vector machine for classification to obtain the defect factor.

5. The automatic repair method according to claim 4, characterized in that: In step S21, the signal feature extraction method includes: Step A211: performing fast Fourier processing on the interval echo sequence to obtain a plurality of frequency domain signals; Step A212: acquiring an adjacent sequence of the interval echo sequence in the complete ultrasonic echo sequence, and determining an ultrasonic frequency domain signal according to the adjacent sequence; Step A213: subtract the frequency domain signal from the ultrasonic frequency domain signal to separate and obtain the signal feature.

6. The automatic repair method according to claim 4, characterized in that: In step S21, the method for extracting image features includes: Step B211: extracting a neighboring image of the area to be repaired, and searching for a pre-constructed sample image template according to the neighboring image; Step B212: extracting a regional sample image corresponding to the area to be repaired from the sample image template; Step B213: subtract the area to be repaired from the area sample image to separate and obtain the image features.

7. The automatic repair method according to claim 1, characterized in that: The step S3 comprises: Step S31: retrieving the mask according to the defect factor, and determining the defect intensity of the defect factor; Step S32: correcting the mask plate according to the defect strength to obtain a corrected mask plate; Step S33: repairing the area to be repaired according to the correction mask.

8. The automatic repair method according to claim 7, characterized in that: The step S33 comprises: Step S331: eliminating interference components in the area to be repaired according to the correction mask to obtain a filtered image; Step S332: stacking and synthesizing the area to be repaired according to the sample image corresponding to the area to be repaired to obtain a fitting area image; Step S333: synthesizing the fitting area image and the image to be restored.

Citation Information

Cited By

  • Ultrasound image artifact restoration method, program product, electronic equipment and storage medium

    CN120219243A