Image processing method, device, electronic device and computer readable medium

Abstract

The present invention discloses an image processing method, including: obtaining a first image and a second image generated when an ultrasound probe is located at different positions, wherein the generation time interval of the first image and the second image is less than a preset time threshold; determining an initial superposition area of ​​the first image and the second image according to a preset mark of the first image and a preset mark of the second image; determining a target image according to a non-artifact area of ​​the first image, a non-artifact area of ​​the second image, and the initial superposition area. By comprehensively utilizing the clear parts of the two images, i.e., the non-artifact areas, a target image without artifacts is obtained through superposition processing, and high-quality monitoring images can be obtained safely and efficiently without raising and lowering the ultrasound probe over a long distance, and without tightly pressing the skin, and it helps to improve the treatment effect. The present invention also discloses an image processing device, an electronic device, and a computer-readable medium.

Description

Technical Field

[0001] The present invention relates to the technical field of minimally invasive treatment image monitoring, and in particular to an image processing method, an image processing device, an electronic device and a computer-readable medium. Background Art

[0002] During minimally invasive surgical treatment, B-scan ultrasonography is often used as a real-time image monitoring method. However, the monitoring images currently obtained often have artifacts, that is, there are one or more arcs on the image, covering and floating on the real monitoring object, resulting in the obstruction of part of the real monitoring object, affecting the quality of the monitoring image. Summary of the invention

[0003] To this end, the present invention provides an image processing method, an image processing device, an electronic device and a computer-readable medium to solve the problem of poor image quality caused by artifacts in monitoring images in the prior art.

[0004] A first aspect of the present invention provides an image processing method, wherein the method comprises:

[0005] Acquire a first image and a second image generated when the ultrasound probe is located at different positions, wherein a time interval between the generation of the first image and the second image is less than a preset time threshold;

[0006] determining an initial overlapping area of ​​the first image and the second image according to a preset mark of the first image and a preset mark of the second image;

[0007] A target image is determined according to the non-artifact area of ​​the first image, the non-artifact area of ​​the second image, and the initial superposition area.

[0008] In some embodiments, the step of determining an initial overlapping area of ​​the first image and the second image according to a preset mark of the first image and a preset mark of the second image includes:

[0009] identifying a preset mark of the first image and a preset mark of the second image;

[0010] Moving the first image or the second image so that a preset mark of the first image overlaps with a preset mark of the second image;

[0011] A portion formed by superimposing the first image and the second image is determined as the initial superimposed area.

[0012] In some embodiments, before the step of determining the portion formed by superimposing the first image and the second image as the initial superimposed area, the method further includes:

[0013] The first image and the second image are preprocessed so that the portions where the first image and the second image can be overlapped have the same brightness and contrast.

[0014] In some embodiments, the non-artifact area of ​​the first image and the non-artifact area of ​​the second image each include an independent non-artifact area and a non-artifact area corresponding to the initial superposition area;

[0015] The step of determining the target image according to the non-artifact area of ​​the first image, the non-artifact area of ​​the second image, and the initial superposition area comprises:

[0016] Performing a replacement process on the initial superimposed area according to the non-artifact area corresponding to the initial superimposed area to obtain a final superimposed area;

[0017] The independent non-artifact area of ​​the first image, the independent non-artifact area of ​​the second image, and the final superimposed area are spliced ​​to obtain the target image.

[0018] In some embodiments, the step of replacing the initial superimposed area according to the non-artifact area corresponding to the initial superimposed area includes:

[0019] Replacing a portion of the initial superposition area corresponding to the artifact area in the first image with a non-artifact area of ​​a corresponding position range in the second image;

[0020] Replacing a portion of the initial superposition area corresponding to the artifact area in the second image with a non-artifact area of ​​a corresponding position range in the first image;

[0021] A target area is determined from the non-artifact area corresponding to the initial superimposed area according to the pixel values ​​of the first image and the pixel values ​​of the second image, and the non-artifact area in the initial superimposed area is replaced by the target area.

[0022] In some embodiments, the first image is generated when the ultrasound probe is located closer to the target monitoring position than when the second image is generated, and the step of determining the target area from the non-artifact area corresponding to the initial superposition area according to the pixel values ​​of the first image and the pixel values ​​of the second image includes:

[0023] When the difference between the pixel value of the first image and the pixel value of the second image is less than a preset pixel threshold, the target area is determined based on the pixel information of the first image, or the target image is determined based on the average value of the pixel information of the first image and the pixel information of the second image.

[0024] In some embodiments, the step of acquiring the first image and the second image generated when the ultrasound probe is located at different positions includes:

[0025] According to the vibration signal, the ultrasonic probe is controlled to vibrate in a direction perpendicular to the target position at a preset vibration frequency and emit a detection ultrasonic wave;

[0026] generating a plurality of images according to the ultrasonic echoes received by the ultrasonic probe;

[0027] The first image and the second image are determined from the plurality of images according to the vibration signal.

[0028] In some embodiments, the step of controlling the ultrasonic probe to vibrate in a direction perpendicular to the target position at a preset vibration frequency and emitting a detection ultrasonic wave according to the vibration signal comprises:

[0029] The vibration signal is sent to the vibration mechanism, so that the vibration mechanism drives the ultrasonic probe to vibrate in a direction perpendicular to the target position and emit detection ultrasonic waves according to the preset vibration frequency.

[0030] In some embodiments, the vibration mechanism is a cam mechanism, and the period angle of the cam mechanism when it is in the short diameter and the period angle when it is in the long diameter are both in the interval [90°, 140°], and the period angle of the cam mechanism when it is between the short diameter and the long diameter is in the interval [80°, 180°].

[0031] A second aspect of the present invention provides an image processing device, wherein the image processing device comprises:

[0032] An acquisition module, used for acquiring a first image and a second image generated when the ultrasound probe is located at different positions and the generation time interval is less than a preset time threshold;

[0033] A first processing module, configured to determine an initial overlapping area of ​​the first image and the second image according to a preset mark of the first image and a preset mark of the second image;

[0034] The second processing module is used to determine a target image according to the non-artifact area of ​​the first image, the non-artifact area of ​​the second image, and the initial superposition area.

[0035] A third aspect of the present invention provides an electronic device, characterized in that it includes:

[0036] one or more processors;

[0037] A storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described above;

[0038] One or more I / O interfaces are connected between the processor and the memory and are configured to implement information interaction between the processor and the memory.

[0039] A fourth aspect of the present invention provides a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the image processing method as described above.

[0040] The present invention has the following advantages:

[0041] The image processing method provided by the embodiment of the present invention is adopted, and the first image and the second image generated when the ultrasound probe is located at different positions and the generation time interval is less than the preset time threshold are used. The initial superposition area of ​​the first image and the second image is determined according to the preset mark of the first image and the preset mark of the second image, and the target image is determined according to the non-artifact area of ​​the first image, the non-artifact area of ​​the second image, and the initial superposition area. The clear part of the two images, that is, the non-artifact area, is comprehensively utilized, and the target image without artifacts is obtained through superposition processing. There is no need to raise and lower the ultrasound probe over a long distance, and there is no need to press the skin tightly, so that high-quality monitoring images can be obtained safely and efficiently. Moreover, since the generation time interval of the first image and the second image is less than the preset time threshold, the state of the human body hardly changes, and the obtained target image can truly and accurately reflect the condition of the body's organs and tissues, which helps to improve the treatment effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present invention, but do not constitute a limitation of the present invention.

[0043] Figure 1a A schematic diagram of monitoring images when the vibration of the ultrasonic probe provided by the embodiment of the present invention is at a high level and a low level;

[0044] Figure 1b A schematic diagram of image superposition provided by an embodiment of the present invention;

[0045] Figure 2 A flowchart of an image processing method provided by an embodiment of the present invention is shown in FIG1 ;

[0046] Figure 3 Schematic diagram of the image processing method provided by the embodiment of the present invention Figure 2 ;

[0047] Figure 4 Schematic diagram of the image processing method provided by the embodiment of the present invention Figure 3 ;

[0048] Figure 5 Schematic diagram of the image processing method provided by the embodiment of the present invention Figure 4 ;

[0049] Figure 6 Schematic diagram of the image processing method provided by the embodiment of the present invention Figure 5 ;

[0050] Figure 7 A schematic diagram of the structure of an image processing device provided by an embodiment of the present invention;

[0051] Figure 8 A schematic diagram of a module of an image processing device provided by an embodiment of the present invention;

[0052] Fig. 9 A schematic diagram of a module of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0054] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0055] The terms used in the present invention are only used to describe specific embodiments and are not intended to limit the present invention. As used in the present invention, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0056] When the terms “comprising” and / or “made of…” are used in the present invention, it specifies the existence of the stated features, integers, steps, operations, elements and / or components, but does not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0057] The embodiments of the present invention may be described with reference to plan views and / or cross-sectional views by way of ideal schematic views of the present invention. Therefore, the exemplary illustrations may be modified according to manufacturing techniques and / or tolerances.

[0058] Unless otherwise defined, all terms (including technical and scientific terms) used in the present invention have the same meaning as those commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present invention, and will not be interpreted as having an idealized or overly formal meaning unless the present invention clearly defines it as such.

[0059] During minimally invasive surgeries such as FUS (focused ultrasound surgery) treatment, B-ultrasound is often used as a means of real-time image monitoring. However, the ultrasound probe is often set on a mechanical device. In addition, coupling agents cannot be used and only water and other media can be used. As a result, the ultrasound probe cannot contact the skin well. There are other media between the ultrasound probe and the skin, which leads to multiple reflections of ultrasound and produces artifacts on the image. That is, there are one or more reflected images of the surface of the FUS (focused ultrasound surgery) transducer on the image, which appear as one or more arcs, covering and floating on the real monitored object, resulting in the obstruction of part of the real monitored object, affecting the quality of the monitored image, which is not only lower than the anatomical expression of MRI (nuclear magnetic resonance imaging) / CT, but even lower than the ultrasound effect used for diagnosis.

[0060] When the ultrasound probe is in full contact with the skin, the artifacts will be significantly reduced or even disappear completely. However, at this time, the ultrasound probe will block the high-intensity focused ultrasound of FUS and press the skin, which will cause the energy of FUS to decay, the focus shape to be poor, and even cause skin burns and damage to the ultrasound probe. Therefore, people have proposed a safe and effective means to solve the artifact problem, that is, to manually raise and lower the ultrasound probe continuously, so that the body tissue and target area can be clearly seen when not irradiated by FUS, and the safety of the skin and ultrasound probe can be guaranteed during FUS irradiation.

[0061] This method can solve the artifact problem to a certain extent. However, since the ultrasound probe needs to be controlled to move away from or close to the skin, the doctor's operation intensity is relatively high. In some cases, in order to see the monitored object clearly, the doctor needs to press the ultrasound probe tightly against the skin, which may cause skin abrasions. When the ultrasound probe is further moved horizontally when it is already pressed very tightly in the direction perpendicular to the target position, the risk of skin abrasions is even greater. The mechanical mechanism is also easily damaged by high-frequency and long-distance movement. Therefore, people have proposed to use automatic control to control the continuous lifting and lowering of the ultrasound probe. Although this reduces the doctor's operation, the high-frequency and long-distance movement of the ultrasound probe is still inefficient, and it is impossible to keep the image in a stable and clear state, which still has an adverse effect on treatment.

[0062] In view of this, the present inventors have found through careful research that when the ultrasound probe approaches or moves away from the skin by moving up and down, the body tissue will move the same distance in the image. For example, for every 1 cm reduction in the distance between the ultrasound probe and the skin, the position of the body tissue in the image will also move down 1 cm, closer to the bottom of the image (i.e., the position of the ultrasound probe). However, the position of the artifact in the image does not move according to this rule. The artifact is generated by reflection, and its movement speed in the image is 2 times (after one reflection) or 4 times (after two reflections) the movement speed of the body tissue in the image. Figure 1 is a schematic diagram of a monitoring image when the vibration of the ultrasound probe is at a high position and a low position, as shown in Figure 1. Figure 1a As shown, Figure A on the left is the image when the vibration of the ultrasound probe is at a high position, and Figure B on the right is the image when the vibration of the ultrasound probe is at a low position. The dotted line in Figure B maps the artifact area and the skin in Figure A to Figure B at the same position. Compared with the ultrasound probe being in a low position, the ultrasound probe is closer to the skin when it is in a high position. Therefore, the skin in Figure A is closer to the bottom of the image than the skin in Figure B. It can be seen that when the ultrasound probe moves quickly and the distance from the skin is different, the artifact will obscure different body tissues on the image.

[0063] Therefore, we can make full use of this phenomenon. According to the characteristic that when the ultrasound probe moves quickly, the movement speed of body tissue in the image is lower than the movement speed of artifacts in the image, we can obtain two images generated when the ultrasound probe is located at different positions, and make comprehensive use of the clear parts of the two images, that is, the non-artifact areas, to superimpose them to obtain a target image without artifacts. This can achieve the purpose of eliminating artifacts in the image safely and efficiently without raising or lowering the ultrasound probe over a long distance or pressing the skin tightly.

[0064] Accordingly, in a first aspect, an embodiment of the present invention provides an image processing method, such as Figure 2 As shown, the method may include the following steps:

[0065] In step S1 , a first image and a second image generated when the ultrasound probe is located at different positions are acquired, wherein a time interval between the generation of the first image and the second image is less than a preset time threshold.

[0066] In step S2, an initial overlapping area of ​​the first image and the second image is determined according to a preset mark of the first image and a preset mark of the second image.

[0067] In step S3, a target image is determined according to the non-artifact area of ​​the first image, the non-artifact area of ​​the second image, and the initial superposition area.

[0068] The ultrasound probe being located at different positions refers to the ultrasound probe being at different heights from the target detection position, and the preset time threshold may be a shorter duration, such as 0.5 seconds, 1 second, 1.5 seconds, etc. The preset mark may include an object whose moving speed is less than the moving speed of the artifact in the image when the ultrasound probe moves quickly, such as a specific body organ tissue. The non-artifact area is an area in the image where no artifact exists.

[0069] It can be seen from the above steps S1-S3 that the image processing method provided by the embodiment of the present invention is used, and the first image and the second image generated when the ultrasound probe is located at different positions and the generation time interval is less than the preset time threshold are used, and the initial superposition area of ​​the first image and the second image is determined according to the preset mark of the first image and the preset mark of the second image, and the target image is determined according to the non-artifact area of ​​the first image, the non-artifact area of ​​the second image, and the initial superposition area. The clear part of the two images, that is, the non-artifact area, is comprehensively utilized, and the target image without artifacts is obtained through superposition processing. There is no need to raise and lower the ultrasound probe over a long distance, and there is no need to press the skin tightly, so that high-quality monitoring images can be obtained safely and efficiently. Moreover, since the generation time interval of the first image and the second image is less than the preset time threshold, the state of the human body hardly changes, and the obtained target image can truly and accurately reflect the condition of the body's organs and tissues, which helps to improve the treatment effect.

[0070] At present, an existing method for eliminating image artifacts is to pre-save a clear image with fewer artifacts when the ultrasound probe is close to the skin. When the real-time image is currently received, the pre-saved clear image is superimposed with the current real-time image according to the position information, that is, the corresponding part of the pre-saved clear image is filled into the current real-time image using a software algorithm. In this way, the image obtained is also clear and artifact-free. However, since the clear image is pre-saved, and the position of the organ tissue may have changed slightly due to physiological movement and other reasons, the pre-saved clear image cannot accurately reflect the current organ tissue condition of the body. In contrast, in the method of the embodiment of the present invention, the time interval between the generation of the two images used to eliminate artifacts is less than the preset time threshold, and both are collected in real time, which can truly and accurately reflect the condition of the body's organ tissue, and help improve the treatment effect.

[0071] Specifically, the preset mark of the first image and the preset mark of the second image can be used as a reference, and the preset mark of the first image and the preset mark of the second image can be overlapped by moving the first image or the second image to obtain the superimposed part of the first image and the second image. Accordingly, in some embodiments, Figure 3 As shown, the step of determining the initial overlapping area of ​​the first image and the second image according to the preset mark of the first image and the preset mark of the second image (ie, step S2) may further include the following steps:

[0072] In step S21, a preset mark of the first image and a preset mark of the second image are identified.

[0073] In step S22, the first image or the second image is moved so that the preset mark of the first image overlaps with the preset mark of the second image.

[0074] In step S23, the portion formed by superimposing the first image and the second image is determined as an initial superimposed area.

[0075] In order to further improve the image quality, the first image and the second image may be preprocessed so that the portions where the first image and the second image can be superimposed have the same brightness and contrast. Accordingly, in some embodiments, before the step of determining the portion where the first image and the second image are superimposed as the initial superposition area (i.e., step S23), the method may further include the following steps: preprocessing the first image and the second image so that the portions where the first image and the second image can be superimposed have the same brightness and contrast.

[0076] The initial superposition area is formed by superimposing the preset mark of the first image and the preset mark of the second image as a reference. Since the first image and the second image are generated when the ultrasound probe is located at different positions, the position of the preset mark in the first image is different from the position of the preset mark in the second image. The first image and the second image cannot be completely superimposed, and usually there will be no artifacts in the part that cannot be superimposed on the first image and the second image due to distance reasons. Therefore, the first image and the second image both include both independent non-artifact areas that do not correspond to the initial superposition area and non-artifact areas corresponding to the initial superposition area. The artifact area in the initial superposition area can be replaced according to the non-artifact areas of the first image and the second image corresponding to the initial superposition area to obtain a final superposition area without artifacts.

[0077] Accordingly, in some embodiments, Figure 4 As shown, the non-artifact area of ​​the first image and the non-artifact area of ​​the second image both include an independent non-artifact area and a non-artifact area corresponding to the initial superimposed area; the step of determining the target image according to the non-artifact area of ​​the first image, the non-artifact area of ​​the second image and the initial superimposed area (i.e., step S3) may further include the following steps:

[0078] In step S31, the initial superimposed area is replaced according to the non-artifact area corresponding to the initial superimposed area to obtain a final superimposed area.

[0079] In step S32, the independent non-artifact area of ​​the first image, the independent non-artifact area of ​​the second image, and the final superimposed area are spliced ​​to obtain a target image.

[0080] First, according to the non-artifact area of ​​the first image corresponding to the initial superposition area and the non-artifact area of ​​the second image corresponding to the initial superposition area, the artifact area and the non-artifact area in the initial superposition area are replaced, so that the final superposition area obtained after the processing does not have artifacts. Then, the independent non-artifact area in the first image that cannot be superimposed with the second image, the independent non-artifact area in the second image that cannot be superimposed with the first image, and the final superposition area without artifacts are spliced ​​to obtain a high-quality target image without artifacts.

[0081] Specifically, the portion of the initial superposition region corresponding to the artifact region in the first image may also be referred to as the artifact region of the initial superposition region, in which the artifact is obtained by superimposing the first image, and the portion of the initial superposition region corresponding to the artifact region in the second image may also be referred to as the artifact region of the initial superposition region, in which the artifact is obtained by superimposing the second image. For the portion of the initial superposition region corresponding to the artifact region in the first image, the portion of the corresponding position range in the second image is usually a non-artifact region, and this portion may be directly replaced by the non-artifact region of the corresponding position range in the second image. For the portion of the initial superposition region corresponding to the artifact region in the second image, the portion of the corresponding position range in the first image is usually a non-artifact region, and this portion may be directly replaced by the non-artifact region of the corresponding position range in the first image. As for the non-artifact area in the initial superimposed area, no artifacts are obtained by superimposing from the first image or the second image. In theory, this part can be directly replaced by the non-artifact area in the corresponding position range in the first image, or by the non-artifact area in the corresponding position range in the second image. However, in order to further improve the image quality, the target area can be determined from the first image and the second image according to the pixel values ​​of the first image and the pixel values ​​of the second image, and the non-artifact area in the initial superimposed area can be replaced by the target area.

[0082] Accordingly, in some embodiments, Figure 5 As shown, the step of replacing the initial superimposed area according to the non-artifact area corresponding to the initial superimposed area (ie, step S31) may further include the following steps:

[0083] In step S311, the portion of the initial superimposed area corresponding to the artifact area in the first image is replaced by a non-artifact area in a corresponding position range in the second image.

[0084] In step S312, the portion of the initial superimposed area corresponding to the artifact area in the second image is replaced by a non-artifact area in a corresponding position range in the first image.

[0085] In step S313, a target area is determined from the non-artifact area corresponding to the initial superimposed area according to the pixel values ​​of the first image and the pixel values ​​of the second image, and the non-artifact area in the initial superimposed area is replaced by the target area.

[0086] Figure 1b The image superposition diagram provided by the embodiment of the present invention is combined with Figure 1a and Figure 1b As shown, it is assumed that image A is the first image, image B is the second image, and the preset mark is the skin. Since image A and image B are generated when the ultrasound probe is located at different positions, the position of the skin in image A is different from the position of the skin in image B. Image A is fixed and image B is moved so that the skin in image B overlaps with the skin in image A. After the skin in image B overlaps with the skin in image A, the artifact area in image B does not overlap with the artifact area in image A. Therefore, the skin in image A shown in the figure is also the skin in image B after superposition, and there is a certain distance between the artifact area in image A and the artifact area in image B after superposition. For the part of the artifact area in image A, it can be replaced by the non-artifact area in the corresponding position range in image B; for the part of the artifact area in image B after superposition, it can be replaced by the non-artifact area in the corresponding position range in image A. It should be noted that at this time, the "corresponding position range" is determined based on the skin in image A or the skin in image B. For example, when the part of the artifact area in image A is 3 cm away from the skin in image A, the part of the artifact area in image A is replaced by the part of the non-artifact area in the same position range in image B that is 3 cm away from the skin in image B.

[0087] In some embodiments, the first image is generated when the ultrasound probe is at a position closer to the target monitoring position than when the second image is generated, and the step of determining the target area from the non-artifact area corresponding to the initial superposition area based on the pixel value of the first image and the pixel value of the second image (i.e., step S313) may further include the following steps: when the difference between the pixel value of the first image and the pixel value of the second image is less than a preset pixel threshold, determining the target area based on the pixel information of the first image, or determining the target image based on the average value of the pixel information of the first image and the pixel information of the second image.

[0088] For the non-artifact area in the initial superimposed area, no artifacts are obtained by superimposing from the first image or the second image, there is a corresponding non-artifact area in the first image, and there is also a corresponding non-artifact area in the second image. When the pixel value of the first image is not much different from the pixel value of the second image, the non-artifact area in the initial superimposed area can be directly replaced by the pixel information of the corresponding non-artifact area in the first image, or by the average value of the pixel information of the corresponding non-artifact area in the first image and the corresponding non-artifact area in the second image.

[0089] In some embodiments, when the pixel value of the first image is significantly different from the pixel value of the second image and the non-artifact area in the initial superimposed area is close to the skin, the pixel information of the non-artifact area in the second image corresponding to the non-artifact area in the initial superimposed area can be used to replace the non-artifact area in the initial superimposed area; otherwise, the pixel information of the non-artifact area in the first image corresponding to the non-artifact area in the initial superimposed area can be used to replace the non-artifact area in the initial superimposed area.

[0090] The first image and the second image generated when the ultrasound probe is located at different positions can be obtained by controlling the ultrasound probe to vibrate up and down. Accordingly, in some embodiments, Figure 6 As shown, the step of acquiring the first image and the second image generated when the ultrasound probe is located at different positions (ie, step S1) may further include the following steps:

[0091] In step S11, the ultrasonic probe is controlled to vibrate in a direction perpendicular to the target position at a preset vibration frequency according to the vibration signal and emit detection ultrasonic waves.

[0092] In step S12, a plurality of images are generated according to the ultrasonic echoes received by the ultrasonic probe.

[0093] In step S13, a first image and a second image are determined from a plurality of images according to the vibration signal.

[0094] Among them, the vibration signal is used to control the ultrasonic probe to vibrate in a direction perpendicular to the target position according to a preset vibration frequency. The position of the ultrasonic probe can be determined according to the vibration signal, and the image is generated almost in real time according to the ultrasonic echo received by the ultrasonic probe. Therefore, the first image and the second image generated when the vibration of the ultrasonic probe is at a high position and the second image generated when the vibration of the ultrasonic probe is at a low position can be determined from multiple images according to the vibration signal.

[0095] In some embodiments, the step of controlling the ultrasonic probe to vibrate in a direction perpendicular to the target position at a preset vibration frequency and emit a detection ultrasonic wave according to the vibration signal (i.e., step S11) may further include the following steps: sending a vibration signal to the vibration mechanism so that the vibration mechanism pushes the ultrasonic probe to vibrate in a direction perpendicular to the target position and emit a detection ultrasonic wave at a preset vibration frequency.

[0096] In some embodiments, the vibration mechanism is a cam mechanism, and the period angle of the cam mechanism when it is in the short diameter and the period angle when it is in the long diameter are both in the interval [90°, 140°], and the period angle of the cam mechanism when it is between the short diameter and the long diameter is in the interval [80°, 180°].

[0097] The vibration mechanism drives the ultrasonic probe to vibrate up and down continuously to obtain multiple real-time images. The vibration mechanism is designed as a cam mechanism. The cam mechanism is designed so that the period angle when it is in the short axis and the period angle when it is in the long axis are both in the interval [90°, 140°]. The period angle of the cam structure when it is between the long axis and the short axis is in the interval [80°, 180°]. This ensures that the vibration of the ultrasonic probe is in a low position or a high position most of the time, and is only between the low position and the high position for a short period of time. In this way, the first image and the second image generated when the ultrasonic probe is in different positions can provide good feature representation, because the position of the preset mark and the artifact area in the first image is not too close to the position in the second image, which will interfere with the subsequent replacement processing.

[0098] The image processing method provided by the present invention is briefly described below in conjunction with a specific embodiment.

[0099] like Figure 7 As shown, the embodiment of the present invention provides an image processing device composed of a FUS transducer, an ultrasonic probe, a vibration mechanism, an ultrasonic imaging module, an image acquisition module, an image processing module and a control module, and its workflow may include the following steps:

[0100] (i) The control module transmits a vibration signal to the vibration mechanism to initiate vibration control, so that the vibration mechanism vibrates according to a preset vibration frequency. At the same time, the control module transmits the vibration signal to the image processing module.

[0101] Among them, if the vibration frequency is set too low, it will affect the real-time performance, and if it is set too high, it will cause image blur. In order to ensure image quality, the preset vibration frequency can be set between 1hz-20hz, and the vibration amplitude can be set between 3mm-15mm.

[0102] (2) The vibration mechanism drives the ultrasonic probe to vibrate up and down.

[0103] Among them, the vibration mechanism can be composed of a special cam mechanism to push the ultrasonic probe to vibrate up and down. The vibration mechanism can be designed as follows: the percentage of the short diameter (the ultrasonic probe descends to the lowest point) and the percentage of the long diameter (the ultrasonic probe rises to the highest point) are each more than 40%, and the percentage of the transition part (the ultrasonic probe is between the lowest point and the highest point) is controlled within 20%.

[0104] (3) The ultrasonic probe that continuously vibrates up and down transmits detection ultrasonic waves and receives ultrasonic echoes.

[0105] (iv) The ultrasonic probe transmits the ultrasonic echo to the ultrasonic imaging module.

[0106] (V) The image acquisition module acquires images from the ultrasonic imaging module and transmits the acquired images to the image processing module.

[0107] (VI) The image processing module receives the image transmitted by the image acquisition module in real time, and also receives the vibration signal transmitted by the control module in real time, and determines the position of the ultrasonic probe according to the control signal. According to the vibration signal, the image img1 generated when the vibration of the ultrasonic probe is at a high level is obtained respectively (refer to Figure 1a Figure A) and the image img2 generated when the ultrasound probe is in a low vibration position (see Figure 1a Find the artifact area on img1 and img2 respectively, remove the artifact area, and superimpose img1 and img2 according to the upper and lower positions to form an image, and then display it.

[0108] Among them, the step of removing the artifact area may include the following steps:

[0109] (1) Determine the vertical distance d1 between the ultrasound probe and the skin when generating img1, determine the vertical distance d2 between the ultrasound probe and the skin when generating img2, and calculate the difference between the two: dis=d2-d1.

[0110] (2) Take img1 as the base image, move img2 downward by a distance of dis, so that the skin line of img2 can overlap with the skin line of img1, and calculate the superimposable area (equivalent to finding the intersection of two sectors).

[0111] (3) Preprocess img1 and img2 so that the superimposable areas of img1 and img2 have the same brightness and contrast.

[0112] (4) The portion formed by the superposition of img1 and img2 is determined as the initial superposition region, and the initial superposition region is compared with img1 and img2. The portion of the initial superposition region corresponding to the artifact region in img1 is replaced with the non-artifact region of the corresponding position range in img2, and the portion of the initial superposition region corresponding to the artifact region in img2 is replaced with the non-artifact region of the corresponding position range in img1. For the non-artifact region in the initial superposition region, if the pixel value difference between img1 and img2 is not large, the pixel information of the non-artifact region of the corresponding position range in img1 can be directly used for replacement, or the pixel information of the non-artifact region of the corresponding position range in img1 can be used to replace the pixel information of the non-artifact region of the corresponding position range in img2. As another embodiment, if the pixel value difference between img1 and img2 is large and the non-artifact region in the initial superposition region is close to the skin, the pixel information of the non-artifact region of the corresponding position range in img2 can be used for replacement, otherwise the pixel information of the non-artifact region of the corresponding position range in img1 can be used for replacement.

[0113] Among them, this step can use the optical flow method to reduce the error caused by slight displacement of the body and find the drift direction of the artifact.

[0114] The step division of the above methods is only for the purpose of clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0115] Based on the same technical concept, in a second aspect, an embodiment of the present invention provides an image processing device. Figure 8 As shown, the image processing device may include:

[0116] The acquisition module 101 is used to acquire a first image and a second image which are generated when the ultrasound probe is located at different positions and the generation time interval is less than a preset time threshold.

[0117] The first processing module 102 is configured to determine an initial overlapping area of ​​the first image and the second image according to a preset mark of the first image and a preset mark of the second image.

[0118] The second processing module 103 is configured to determine a target image according to the non-artifact area of ​​the first image, the non-artifact area of ​​the second image, and the initial superimposed area.

[0119] In some embodiments, the first processing module 102 is used to:

[0120] identifying a preset mark of the first image and a preset mark of the second image;

[0121] Moving the first image or the second image so that a preset mark of the first image overlaps with a preset mark of the second image;

[0122] A portion formed by superimposing the first image and the second image is determined as the initial superimposed area.

[0123] In some embodiments, the first processing module 102 is further configured to:

[0124] The first image and the second image are preprocessed so that the portions where the first image and the second image can be overlapped have the same brightness and contrast.

[0125] In some embodiments, the non-artifact area of ​​the first image and the non-artifact area of ​​the second image both include an independent non-artifact area and a non-artifact area corresponding to the initial superposition area; the second processing module 103 is used to:

[0126] Performing a replacement process on the initial superimposed area according to the non-artifact area corresponding to the initial superimposed area to obtain a final superimposed area;

[0127] The independent non-artifact area of ​​the first image, the independent non-artifact area of ​​the second image, and the final superimposed area are spliced ​​to obtain the target image.

[0128] In some embodiments, the second processing module 103 is used to:

[0129] Replacing a portion of the initial superposition area corresponding to the artifact area in the first image with a non-artifact area of ​​a corresponding position range in the second image;

[0130] Replacing a portion of the initial superposition area corresponding to the artifact area in the second image with a non-artifact area of ​​a corresponding position range in the first image;

[0131] A target area is determined from the non-artifact area corresponding to the initial superimposed area according to the pixel values ​​of the first image and the pixel values ​​of the second image, and the non-artifact area in the initial superimposed area is replaced by the target area.

[0132] In some embodiments, the first image is generated when the ultrasound probe is located closer to the target monitoring position than when the second image is generated, and the second processing module 103 is used to:

[0133] When the difference between the pixel value of the first image and the pixel value of the second image is less than a preset pixel threshold, the target area is determined based on the pixel information of the first image, or the target image is determined based on the average value of the pixel information of the first image and the pixel information of the second image.

[0134] In some embodiments, the acquisition module 101 is used to:

[0135] According to the vibration signal, the ultrasonic probe is controlled to vibrate in a direction perpendicular to the target position at a preset vibration frequency and emit a detection ultrasonic wave;

[0136] generating a plurality of images according to the ultrasonic echoes received by the ultrasonic probe;

[0137] The first image and the second image are determined from the plurality of images according to the vibration signal.

[0138] In some embodiments, the acquisition module 101 is used to send a vibration signal to the vibration mechanism, so that the vibration mechanism drives the ultrasonic probe to vibrate in a direction perpendicular to the target position and emit detection ultrasonic waves according to a preset vibration frequency.

[0139] In some embodiments, the vibration mechanism is a cam mechanism, and the period angle of the cam mechanism when it is in the short diameter and the period angle when it is in the long diameter are both in the interval [90°, 140°], and the period angle of the cam mechanism when it is between the short diameter and the long diameter is in the interval [80°, 180°].

[0140] It should be clear that the present invention is not limited to the specific configurations and processes described in the above embodiments and shown in the figures. For the convenience and brevity of description, a detailed description of the known methods is omitted here, and the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, which will not be repeated here.

[0141] like Fig. 9 As shown, an embodiment of the present invention provides an electronic device, which may include:

[0142] One or more processors 901;

[0143] A memory 902 storing one or more programs, which, when executed by one or more processors, enables the one or more processors to implement any one of the above-mentioned image processing methods;

[0144] One or more I / O interfaces 903 are connected between the processor and the memory and are configured to implement information exchange between the processor and the memory.

[0145] Among them, the processor 901 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 902 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) 903 is connected between the processor 901 and the memory 902, and can realize information interaction between the processor 901 and the memory 902, including but not limited to a data bus (Bus), etc.

[0146] In some embodiments, the processor 901 , the memory 902 , and the I / O interface 903 are connected to each other through a bus, and further connected to other components of the computing device.

[0147] This embodiment further provides a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the image processing method provided by this embodiment is implemented. To avoid repeated description, the specific steps of the image processing method are not repeated here.

[0148] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the invented method described above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0149] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0150] Those skilled in the art will understand that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present embodiment and to form different embodiments.

[0151] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present invention, but the present invention is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. An image processing method, wherein: The method comprises: Acquire a first image and a second image generated when the ultrasound probe is located at different positions, wherein the time interval between the generation of the first image and the second image is less than a preset time threshold; wherein the ultrasound probe being located at different positions includes different heights of the ultrasound probe from a target detection position; determining an initial overlapping area of ​​the first image and the second image according to a preset mark of the first image and a preset mark of the second image; Determine a target image according to the non-artifact area of ​​the first image, the non-artifact area of ​​the second image, and the initial superposition area; Wherein, both the first image and the second image include the preset mark and the artifact, and the preset mark includes an object whose moving speed is less than the moving speed of the artifact in the first image and the second image when the ultrasound probe moves at different positions; the step of determining the initial overlapping area of ​​the first image and the second image according to the preset mark of the first image and the preset mark of the second image includes: identifying a preset mark of the first image and a preset mark of the second image; Moving the first image or the second image so that a preset mark of the first image overlaps with a preset mark of the second image; According to the fact that the moving speed of the body tissue in the image is less than the moving speed of the artifact in the image when the ultrasound probe moves quickly, the portion formed by superimposing the first image and the second image is determined as the initial superimposed area.

2. The method according to claim 1, wherein: Before the step of determining the portion formed by superimposing the first image and the second image as the initial superimposed area, the method further includes: The first image and the second image are preprocessed so that portions of the first image and the second image that can be overlapped have the same brightness and contrast.

3. The method according to claim 1, wherein: The non-artifact area of ​​the first image and the non-artifact area of ​​the second image both include an independent non-artifact area and a non-artifact area corresponding to the initial superposition area; The step of determining the target image according to the non-artifact area of ​​the first image, the non-artifact area of ​​the second image, and the initial superposition area comprises: Performing a replacement process on the initial superimposed area according to the non-artifact area corresponding to the initial superimposed area to obtain a final superimposed area; The independent non-artifact area of ​​the first image, the independent non-artifact area of ​​the second image, and the final superimposed area are spliced ​​to obtain the target image.

4. The method according to claim 3, wherein: The step of replacing the initial superimposed area according to the non-artifact area corresponding to the initial superimposed area comprises: Replacing a portion of the initial superposition area corresponding to the artifact area in the first image with a non-artifact area of ​​a corresponding position range in the second image; Replacing a portion of the initial superposition area corresponding to the artifact area in the second image with a non-artifact area of ​​a corresponding position range in the first image; A target area is determined from the non-artifact area corresponding to the initial superimposed area according to the pixel values ​​of the first image and the pixel values ​​of the second image, and the non-artifact area in the initial superimposed area is replaced by the target area.

5. The method according to claim 4, wherein: The first image is generated when the ultrasound probe is located closer to the target monitoring position than when the second image is generated, and the step of determining the target area from the non-artifact area corresponding to the initial superposition area according to the pixel value of the first image and the pixel value of the second image includes: When the difference between the pixel value of the first image and the pixel value of the second image is less than a preset pixel threshold, the target area is determined based on the pixel information of the first image, or the target image is determined based on the average value of the pixel information of the first image and the pixel information of the second image.

6. The method according to any one of claims 1 to 5, wherein: The step of acquiring the first image and the second image generated when the ultrasound probe is located at different positions comprises: According to the vibration signal, the ultrasonic probe is controlled to vibrate in a direction perpendicular to the target position at a preset vibration frequency and emit a detection ultrasonic wave; generating a plurality of images according to the ultrasonic echoes received by the ultrasonic probe; The first image and the second image are determined from the plurality of images according to the vibration signal.

7. The method according to claim 6, wherein: The step of controlling the ultrasonic probe to vibrate in a direction perpendicular to the target position according to a preset vibration frequency and emitting a detection ultrasonic wave according to the vibration signal comprises: The vibration signal is sent to the vibration mechanism, so that the vibration mechanism drives the ultrasonic probe to vibrate in a direction perpendicular to the target position and emit detection ultrasonic waves according to the preset vibration frequency.

8. The method according to claim 7, wherein: The vibration mechanism is a cam mechanism, and the period angle of the cam mechanism when it is in the short axis and the period angle when it is in the long axis are both in the interval [90°, 140°], and the period angle of the cam mechanism when it is between the short axis and the long axis is in the interval [80°, 180°].

9. An image processing device, wherein: The image processing device comprises: An acquisition module, configured to acquire a first image and a second image generated when the ultrasound probe is located at different positions and the generation time interval is less than a preset time threshold; wherein the ultrasound probe being located at different positions includes different heights of the ultrasound probe from a target detection position; A first processing module, configured to determine an initial overlapping area of ​​the first image and the second image according to a preset mark of the first image and a preset mark of the second image; A second processing module, configured to determine a target image according to a non-artifact area of ​​the first image, a non-artifact area of ​​the second image, and the initial superposition area; Wherein, the first image and the second image both include the preset mark and the artifact, and the preset mark includes an object whose moving speed is less than the moving speed of the artifact in the first image and the second image when the ultrasound probe moves at different positions; The first processing module is further used for: identifying a preset mark of the first image and a preset mark of the second image; Moving the first image or the second image so that a preset mark of the first image overlaps with a preset mark of the second image; According to the fact that the moving speed of the body tissue in the image is less than the moving speed of the artifact in the image when the ultrasound probe moves quickly, the portion formed by superimposing the first image and the second image is determined as the initial superimposed area.

10. An electronic device, characterized in that: include: one or more processors; A storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method according to any one of claims 1 to 8; One or more I / O interfaces are connected between the processor and the memory and are configured to implement information interaction between the processor and the memory.

11. A computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the image processing method according to any one of claims 1 to 8.

Images