Ultrasonic video image processing method, device, and computer-readable storage medium

Abstract

An embodiment of the present application provides an ultrasonic video image processing method, device, equipment and computer-readable storage medium, the method comprising: obtaining an ultrasonic video image to be processed of a target area; combining and processing each ultrasonic video image to obtain a three-dimensional complete image of the target area; processing the three-dimensional complete image according to the ultrasonic video image and a pre-stored standard video image of the target area to obtain a target ultrasonic video image; in this embodiment, the ultrasonic video image actually obtained by scanning the target area is compared with the pre-stored standard video image, and then the three-dimensional complete image is processed based on the comparison result to obtain a processed ultrasonic video image, thereby improving the quality of the ultrasonic video image.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and specifically to an ultrasound video image processing method, device, and computer-readable storage medium. Background Art

[0002] Ultrasound scanning is a relatively common imaging method in AI medical treatment. Generally, during ultrasound scanning, the target area of ​​the human body is scanned manually using ultrasound equipment. After multiple scans, multiple ultrasound video images are obtained. Due to some techniques or experience problems in the scanning process, it is difficult to ensure that there are ultrasound video images that can fully meet the scanning requirements among the multiple ultrasound video images obtained. Therefore, when doctors watch ultrasound video images, they usually need to play and watch them one by one, and repeatedly look through multiple video images to find useful diagnostic information for further observation and diagnosis. This method is time-consuming and labor-intensive, and is prone to observation fatigue and misdiagnosis. Summary of the invention

[0003] The embodiments of the present application provide an ultrasound video image processing method, apparatus, and computer-readable storage medium.

[0004] On the one hand, an embodiment of the present application provides an ultrasonic video image processing method, comprising:

[0005] Acquire the ultrasonic video image to be processed in the target area;

[0006] Combine and process each ultrasonic video image to obtain a complete three-dimensional image of the target area;

[0007] The three-dimensional complete image is processed according to the ultrasonic video image and the pre-stored standard video image of the target area to obtain the target ultrasonic video image.

[0008] In some embodiments of the present application, a three-dimensional complete image is processed according to an ultrasonic video image and a pre-stored standard video image of a target area to obtain a target ultrasonic video image, including:

[0009] Acquire a first scanning trajectory of the ultrasonic video image in the target area, and a second scanning trajectory of the standard video image in the target area;

[0010] Determine a first key frame from the ultrasonic video image, and obtain a first track coordinate point of the first key frame in the first scanning track;

[0011] Determine a second key frame from the standard video image, and obtain a second track coordinate point of the second key frame in the second scanning track;

[0012] Based on the correspondence between the first trajectory coordinate points and the second trajectory coordinate points, the target ultrasound video image is acquired from the three-dimensional complete image in combination with the ultrasound video image.

[0013] In some embodiments of the present application, based on the correspondence between the first trajectory coordinate point and the second trajectory coordinate point, the target ultrasound video image is obtained from the three-dimensional complete image in combination with the ultrasound video image, including:

[0014] Determine the video frame sequence number difference between a first key frame in the ultrasound video image and a second key frame corresponding to the standard video image according to the coordinate point sequence number of the first track coordinate point and the coordinate point sequence number of the second track coordinate point;

[0015] Based on the video frame sequence number difference, obtaining actual video frames corresponding to each standard video frame in the standard video image in the ultrasound video image;

[0016] If the standard video frame corresponds to the actual video frame, a target ultrasound video image is obtained from the three-dimensional complete image through offset processing based on the ultrasound video image;

[0017] If the standard video frame does not correspond to the actual video frame, a target ultrasound video image is obtained from the three-dimensional complete image through stitching processing based on the ultrasound video image.

[0018] In some embodiments of the present application, obtaining a target ultrasound video image from a three-dimensional complete image through offset processing based on the ultrasound video image includes:

[0019] According to the standard track coordinate points of each standard video frame in the second scanning track and the actual track coordinate points of the actual video frame corresponding to each standard video frame in the first scanning track, an image offset of each standard video frame corresponding to the actual video frame is obtained;

[0020] Based on the image offset of the actual video frame corresponding to each standard video frame in the ultrasonic video image, the target video frame corresponding to each standard video frame is obtained from the three-dimensional complete image to obtain the target ultrasonic video image.

[0021] In some embodiments of the present application, according to the standard track coordinate points of each standard video frame in the second scanning track and the actual track coordinate points of each standard video frame corresponding to the actual video frame in the first scanning track, obtaining the image offset of each standard video frame corresponding to the actual video frame includes:

[0022] Calculating a first offset between a first scanning track coordinate point and a second scanning track coordinate point;

[0023] Calculate a second offset between an actual track coordinate point of each actual video frame in the first scanning track and a standard track coordinate point of a standard video frame corresponding to each actual video frame in the second scanning track;

[0024] The image offset corresponding to each actual video frame is obtained according to the first offset and the second offset corresponding to each actual video frame.

[0025] In some embodiments of the present application, obtaining a target ultrasound video image from a three-dimensional complete image through stitching processing based on the ultrasound video image includes:

[0026] Taking any standard video frame as a target video frame in turn, if the number of actual video frames corresponding to the target video frame is two or more, determining a target actual video frame from the actual video frames corresponding to the target video frame;

[0027] If the number of actual video frames corresponding to the target video frame is zero, a blank video frame is generated as a target actual video frame corresponding to the target video frame;

[0028] If the number of actual video frames corresponding to the target video frame is one, determining a target actual video frame from the actual video frames corresponding to the target video frame;

[0029] Based on the target actual video frames corresponding to the standard video frames, the target ultrasound video image is obtained from the three-dimensional complete image.

[0030] In some embodiments of the present application, each ultrasound video image is combined and processed to obtain a three-dimensional complete image of the target area, including:

[0031] Acquire video frames in ultrasound video images;

[0032] Input each video frame into a preset tissue feature recognition model to obtain tissue feature information corresponding to each video frame; the tissue feature recognition model is obtained by inputting sample video frames of labeled scanned tissue features into a deep learning neural network for iterative training;

[0033] According to the tissue feature information corresponding to each video frame, each video frame is combined and processed to obtain a three-dimensional complete image.

[0034] On the other hand, an embodiment of the present application further provides an ultrasonic video image processing device, comprising:

[0035] An acquisition module, used for acquiring an ultrasonic video image to be processed in a target area;

[0036] A splicing module is used to combine and process the ultrasound video images to obtain a complete three-dimensional image of the target area;

[0037] The processing module is used to process the three-dimensional complete image according to the ultrasonic video image and the pre-stored standard video image of the target area to obtain the target ultrasonic video image.

[0038] On the other hand, an embodiment of the present application also provides an ultrasonic video image processing device, which includes a processor, a memory, and an ultrasonic video image processing program stored in the memory and executable on the processor. The processor executes the ultrasonic video image processing program to implement the steps in the above-mentioned ultrasonic video image processing method.

[0039] On the other hand, an embodiment of the present application further provides a computer-readable storage medium, on which an ultrasound video image processing program is stored, and the ultrasound video image processing program is executed by a processor to implement the steps in the above-mentioned ultrasound video image processing method.

[0040] Compared with the prior art ultrasound video image processing method, device, and computer readable storage medium, this method includes: obtaining the ultrasound video image to be processed in the target area; combining and processing each ultrasound video image to obtain a three-dimensional complete image of the target area; processing the three-dimensional complete image according to the ultrasound video image and the pre-stored standard video image of the target area to obtain the target ultrasound video image. By comparing the ultrasound video image actually scanned in the target area with the pre-stored standard video image, and then processing the three-dimensional complete image based on the comparison result, an ultrasound video image closer to the target is obtained, so that the video frame in the target ultrasound video image finally presented can present image information closer to the standard section of the body organ or physiological tissue, present the target area (i.e., the body organ or physiological tissue) to the greatest extent, improve the quality of the ultrasound video image, and facilitate subsequent medical staff to view and obtain information in the target area. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 is a scene schematic diagram of the ultrasound video image processing method provided in an embodiment of the present application;

[0043] Figure 2 is a flow chart of an ultrasonic video image processing method provided in an embodiment of the present application;

[0044] Figure 3 is a schematic flow chart of target ultrasound video image acquisition steps provided in an embodiment of the present application;

[0045] Figure 4 is a flowchart of another target ultrasound video image acquisition step provided in an embodiment of the present application;

[0046] Figure 5 is a schematic diagram of a first scanning trajectory and a second scanning trajectory in an embodiment of the present application;

[0047] Figure 6 is a schematic diagram of another first scanning trajectory and a second scanning trajectory in an embodiment of the present application;

[0048] Figure 7 is a schematic diagram of another first scanning trajectory and a second scanning trajectory in an embodiment of the present application;

[0049] Figure 8 is a schematic diagram of the structure of an embodiment of an ultrasonic video image processing device provided in an embodiment of the present application;

[0050] Fig. 9 It is a schematic diagram of the structure of an embodiment of the ultrasonic video image processing device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of the present invention.

[0052] In the embodiments of the present application, the word "exemplary" is used to mean "used as an example, illustration or description". Any embodiment described as "exemplary" in the embodiments of the present application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid unnecessary details that make the description of the present invention obscure. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in the embodiments of the present application.

[0053] In the embodiments of the present application, a method, device, equipment and computer-readable storage medium for processing ultrasonic video images are provided, which are described in detail below.

[0054] In the embodiment of the present application, the ultrasonic video image processing method is deployed on the ultrasonic video image processing device in the form of a program, and the ultrasonic video image processing device is installed in the ultrasonic video image processing equipment in the form of a processor. The ultrasonic video image processing device in the ultrasonic video image processing equipment executes the following steps by running the program corresponding to the ultrasonic video image processing method: obtaining the ultrasonic video image to be processed of the target area; combining and processing each of the ultrasonic video images to obtain a three-dimensional complete image of the target area; processing the three-dimensional complete image according to the ultrasonic video image and the pre-stored standard video image of the target area to obtain the target ultrasonic video image.

[0055] like Figure 1 As shown, Figure 1 The schematic diagram of the implementation scenario of the ultrasonic video image processing of the embodiment of the present application includes an ultrasonic video image processing device 100 and a shooting device 200. The shooting device 200 is mainly used to shoot and obtain an ultrasonic video image of the target area, and the ultrasonic video image processing device 100 runs a computer storage medium corresponding to the ultrasonic video image processing method to perform the steps of ultrasonic video image processing. The shooting device 200 includes but is not limited to an ultrasonic scanner.

[0056] It should be noted that Figure 1 The scene diagram of ultrasound video image processing shown is only an example. The scene of ultrasound video image processing described in the embodiment of the present application is to more clearly illustrate the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application.

[0057] Based on the above-mentioned schematic diagram of the implementation scenario of ultrasound video image processing, a specific embodiment of the ultrasound video image processing method is proposed.

[0058] like Figure 2 As shown, Figure 2 The following is a schematic flow chart of a method for processing ultrasonic video images provided in an embodiment of the present application. The method for processing ultrasonic video images in an embodiment of the present application includes steps S210 to S230, specifically:

[0059] Step S210: Acquire the ultrasonic video image of the target area to be processed.

[0060] Among them, the target area refers to the area where the scanning object of the ultrasound video image is located; in actual application scenarios, the target area can be but is not limited to the area where the body organs or physiological tissues are located. For example, the target area can be the area where the heart, liver, breast tissue, bones and other body organs or physiological tissues are located. The target area can also be the area where the diseased tissue in the heart, liver, breast tissue, bones and other body organs or physiological tissues is located or the lesion area and other lesion areas, which are not limited here.

[0061] Among them, ultrasonic video images refer to videos captured by medical imaging equipment based on a target area where the scanning object is located with a certain shooting trajectory. It can be understood that the ultrasonic video images include multiple video frames, and each video frame is an ultrasonic image including different positions of the target area.

[0062] In this step, the target area where the body organ or physiological tissue to be detected is located can be scanned by an ultrasonic scanning device, and multiple video frames can be obtained by scanning different scanning positions in the target area to form an ultrasonic video image. Specifically, a standard scanning trajectory corresponding to the target area (such as the area where the body organ, physiological tissue or diseased tissue is located) can be first obtained, and then the target area can be scanned by the ultrasonic scanning device according to the standard scanning trajectory to obtain multiple video frames at different scanning positions on the standard scanning trajectory to obtain an ultrasonic video image.

[0063] Furthermore, in one embodiment, the ultrasonic video image may be a two-dimensional ultrasonic video image obtained by scanning using a two-dimensional ultrasonic scanning device, or may be a three-dimensional ultrasonic video image obtained by scanning using a three-dimensional ultrasonic scanning device or a four-dimensional ultrasonic scanning device.

[0064] Step S220: Combine and process the ultrasonic video images to obtain a three-dimensional complete image of the target area.

[0065] Among them, the three-dimensional complete image refers to an image including all positions in the target area. It is understandable that, due to the small scanning probe of the ultrasonic scanning instrument, the ultrasonic scanning of the target area often requires one or more scans of the target area so that the scanned ultrasonic video image can cover all positions of the target area. Therefore, after the ultrasonic video image is acquired, based on each video frame in the ultrasonic video image, each video frame is combined and processed so that the video frames in the ultrasonic video image are combined into an ultrasonic image that can display all positions in the target area.

[0066] Specifically, in one embodiment, each ultrasound video image is combined and processed to obtain a three-dimensional complete image of the target area, including: acquiring video frames in the ultrasound video image; inputting each video frame into a preset tissue feature recognition model to obtain tissue feature information corresponding to each video frame; the tissue feature recognition model is obtained by inputting sample video frames of marked scanned tissue features into a deep learning neural network for iterative training; and combining each video frame according to the tissue feature information corresponding to each video frame to obtain a three-dimensional complete image.

[0067] The tissue feature recognition model refers to a deep learning model for identifying tissue feature information contained in a video frame, such as a convolutional neural network, a deconvolutional neural network, a deep neural network, etc., which is not limited here. It is understandable that the tissue feature recognition model here is a trained deep learning model.

[0068] Specifically, the video frames in the ultrasound video image are input into the tissue feature recognition model to identify the tissue feature information in each video frame, and then the matching information between the video frames is determined by comparing the tissue feature information in each video frame, so that the video frames are spliced ​​and combined based on the matching information to obtain a three-dimensional complete image.

[0069] Taking the target area as physiological tissue as an example, the video frames of the ultrasound image video segment are input into the object recognition model to obtain the tissue feature information of the physiological tissue in the ultrasound image video, and then by comparing the tissue feature information in each video frame, the matching information between each video frame is determined, and the video frames are spliced ​​and combined based on the matching information to obtain a complete three-dimensional image that can fully display the physiological tissue.

[0070] Step S230: Processing the three-dimensional complete image according to the ultrasonic video image and the pre-stored standard video image of the target area to obtain the target ultrasonic video image.

[0071] Among them, the standard video image refers to the ultrasound video image obtained by ultrasonic scanning the target area with a standard scanning trajectory, and may refer to the ultrasound video image that can present the clearest target area or the target area that medical staff most want to observe. For example, it may include a standard cross-section of an organic organ or physiological tissue, which can see the target area (i.e., an organic organ or physiological tissue) to the greatest extent.

[0072] Among them, there is an error between the ultrasound video image obtained by ultrasonic scanning of the target area and the standard video image. Therefore, after obtaining the three-dimensional complete image, the ultrasound video image can be optimized based on the difference between the ultrasound video image and the standard video image in combination with the three-dimensional complete image to obtain a target ultrasound video image that can represent more corresponding image information of the target area.

[0073] Specifically, in one embodiment, Figure 3 As shown, according to the ultrasonic video image and the pre-stored standard video image of the target area, the three-dimensional complete image is processed to obtain the target ultrasonic video image, including:

[0074] Step S310, obtaining a first scanning trajectory of the ultrasonic video image in the target area, and a second scanning trajectory of the standard video image in the target area.

[0075] Among them, the scanning trajectory refers to a coordinate sequence composed of coordinate data corresponding to multiple coordinate points; the second scanning trajectory of the standard video image in the target area refers to the standard scanning trajectory composed of the corresponding position coordinates of each video frame in the standard video image in the target area; specifically, the standard video images corresponding to different areas (i.e., different organic organs or physiological tissues) and the standard scanning trajectory of the standard video image can be obtained in advance, and after the target area of ​​the ultrasound video image is acquired, the standard video image of the corresponding area and the standard scanning trajectory of the standard video image are searched based on the target area.

[0076] Similarly, as mentioned above, the first scanning trajectory of the ultrasound video image in the target area refers to the scanning trajectory formed by the corresponding position coordinates of each video frame in the ultrasound video image in the target area. Specifically, the first scanning trajectory of the ultrasound video image can be determined by motion tracking. For example, the optical flow information of each video frame in the ultrasound video image can be obtained, and the relative position corresponding to each video frame in the ultrasound video image can be determined according to the optical flow information to obtain the first scanning trajectory.

[0077] Step S320: determine a first key frame from the ultrasound video image, and obtain a first track coordinate point of the first key frame in the first scanning track.

[0078] Step S330 , determining a second key frame from the standard video image, and obtaining a second track coordinate point of the second key frame in the second scanning track.

[0079] Step S340 , based on the correspondence between the first trajectory coordinate points and the second trajectory coordinate points, the target ultrasound video image is acquired from the three-dimensional complete image in combination with the ultrasound video image.

[0080] The first key frame refers to a video frame in the ultrasound video image that includes image information corresponding to the key object in the target area; the second key frame refers to a video frame in the standard video image that includes image information corresponding to the key object in the target area. It can be understood that since both the first key frame and the second key frame include image information of the key object, the first key frame and the second key frame are corresponding video frames between the ultrasound video image and the standard video image.

[0081] After determining the first key frame and the second key frame, the first track coordinate point of the first key frame in the first scanning track can be obtained, and the second track coordinate point of the second key frame in the second scanning track can be obtained. It can be understood that the correspondence between the coordinate points of the first scanning track and the second scanning track can be determined according to the correspondence between the coordinate points of the first scanning track and the coordinate points of the second scanning track. Further, the correspondence between the coordinate points of the first scanning track and the second scanning track can be used to represent the correspondence between each video frame in the ultrasound video image and each video frame in the standard video image; therefore, after obtaining the first track coordinate point and the second track coordinate point, the target ultrasound video image can be obtained from the three-dimensional complete image based on the correspondence between the coordinate points of the first tracking track and the coordinate points of the second tracking track, combined with the ultrasound video image. Among them, the correspondence between the coordinate points of the first tracking track and the coordinate points of the second tracking track may include the following situations: (1) each of the coordinate points of the second tracking track has only one corresponding first track coordinate point, in which case the first track coordinate point corresponds to the second track coordinate point; (2): the coordinate point of the second tracking track has two or more first track coordinate points or no corresponding first track coordinate point, in which case the coordinate point of the first tracking track does not correspond to the coordinate point of the second tracking track. After obtaining the correspondence between the first trajectory coordinate point and the second trajectory coordinate point, if the first trajectory coordinate point corresponds to the second trajectory coordinate point, the target ultrasound video image can be obtained from the three-dimensional complete image through offset processing based on the ultrasound video image; if the first trajectory coordinate point does not correspond to the second trajectory coordinate point, the target ultrasound video image can be obtained from the three-dimensional complete image through offset processing based on the ultrasound video image.

[0082] Furthermore, the subsequent processing method of the ultrasonic video image obtained only by the correspondence between the first trajectory coordinate point and the second trajectory coordinate point will result in a large difference between the target ultrasonic video image finally obtained and the standard video image; therefore, Figure 4 As shown, in one embodiment, Figure 4 As shown, based on the correspondence between the first trajectory coordinate point and the second trajectory coordinate point, the target ultrasound video image is obtained from the three-dimensional complete image in combination with the ultrasound video image, including:

[0083] Step S410, determining the video frame number difference between the first key frame in the ultrasound video image and the corresponding second key frame in the standard video image according to the coordinate point number of the first trajectory coordinate point and the coordinate point number of the second trajectory coordinate point.

[0084] Among them, the coordinate point serial number can refer to the serial number of the first trajectory coordinate point or the second trajectory coordinate point in the corresponding scanning trajectory, or it can be the frame serial number of the first key frame corresponding to the first trajectory coordinate point in the ultrasound video image, or the frame serial number of the second key frame corresponding to the second trajectory coordinate point in the standard video image.

[0085] For example, the coordinate point number of the second trajectory coordinate point is k, the coordinate point number corresponding to the first trajectory coordinate point is k′, and the video frame number difference between the first key frame in the ultrasound video image and the second key frame corresponding to the standard video image is determined as Δk=k′-k.

[0086] Step S420 , based on the video frame sequence number difference, actual video frames corresponding to each standard video frame in the standard video image are obtained in the ultrasound video image.

[0087] The actual video frame refers to the video frame actually captured in the ultrasound video image, and the standard video frame refers to the video frame expected to be captured in the standard video image.

[0088] In this step, after obtaining the video frame number difference between the first key frame in the ultrasound video image and the corresponding second key frame in the standard video image, the correspondence between each actual video frame in the ultrasound video image and each standard video frame in the standard video image is obtained based on the video frame number difference.

[0089] For example, for the i-th standard video frame in the standard video image, the j-th actual video frame corresponds to it in the ultrasound video image, where j=Δk+i.

[0090] Step S430: If the standard video frame corresponds to the actual video frame, a target ultrasound video image is obtained from the three-dimensional complete image through offset processing based on the ultrasound video image.

[0091] Step S440: If the standard video frame does not correspond to the actual video frame, a target ultrasound video image is obtained from the three-dimensional complete image through stitching processing based on the ultrasound video image.

[0092] Among them, after obtaining the actual video frames corresponding to each standard video frame in the standard video image, the correspondence between the standard video frames and the actual video frames can be determined according to each standard video frame and its corresponding actual video frame. Specifically, if each standard video frame and its corresponding actual video frame correspond one-to-one, that is, each standard video frame has and only has one actual video frame corresponding to it, then the standard video frame corresponds to the actual video frame; if each standard video frame and its corresponding actual video frame do not correspond one-to-one, that is, some standard video frames have two or more actual video frames corresponding to them, or some standard video frames have no actual video frames corresponding to them, then the standard video frame does not correspond to the actual video frame.

[0093] By comparing all standard video frames in the standard video image with all actual video frames in the ultrasound video image, the subsequent processing method of the ultrasound video image is determined to improve the degree of proximity between the final target ultrasound video image and the standard video image, so that the target ultrasound video image can present the target area (i.e., the organ or physiological tissue of the organism) to the greatest extent, thereby improving the quality of the target ultrasound video image.

[0094] Among them, when the standard video frame corresponds to the actual video frame, the target ultrasound video image can be obtained from the three-dimensional complete image through offset processing based on the ultrasound video image. Specifically, for each video frame in the ultrasound video image, a target video frame whose coordinate position is obtained from the three-dimensional complete image and offset to the standard video image can be obtained based on the coordinate position of the video frame as the video frame corresponding to the target video image.

[0095] In one embodiment, obtaining a target ultrasonic video image from a three-dimensional complete image through offset processing based on the ultrasonic video image may specifically include: obtaining an image offset of each standard video frame corresponding to the actual video frame based on the standard trajectory coordinate points of each standard video frame in the second scanning trajectory and the actual trajectory coordinate points of the actual video frame corresponding to each standard video frame in the first scanning trajectory; obtaining a target video frame corresponding to each standard video frame from the three-dimensional complete image based on the image offset of the actual video frame corresponding to each standard video frame in the ultrasonic video image, thereby obtaining a target ultrasonic video image.

[0096] The target video frame refers to an image that is closer to a corresponding standard video frame in a standard video image after translation optimization is performed on an actual video frame in an ultrasound video image.

[0097] After obtaining the actual video frames corresponding to each standard video frame, for any standard video frame and its corresponding actual video frame, the image offset of the actual video frame corresponding to the standard video frame can be calculated according to the standard track coordinate point of the standard video frame in the second scanning track and the actual track coordinate point of the corresponding actual video frame in the first scanning track. After obtaining the image offset of the actual video frame corresponding to all standard video frames, the actual track coordinate point of the actual video frame is used as the starting coordinate and the image offset is used as the distance to obtain the corresponding image information in the three-dimensional complete image as the target video frame corresponding to the standard video frame (or the standard video frame corresponding to the actual video frame).

[0098] Continuing to take the i-th standard video frame in the standard video image as an example, the corresponding one in the ultrasound video image is the j-th actual video frame, where j=Δk+i; for the j-th actual video frame corresponding to the i-th standard video frame, its image offset can be obtained by the following formula:

[0099] ΔTi =P i -P j , i=1,2,3,…;j=Δk+i

[0100] Where, ΔT i It refers to the image offset of the jth actual video frame corresponding to the i-th standard video frame; P i refers to the standard track coordinate point (or coordinate data) of the standard video frame of the i-th frame in the second scanning track; P j It refers to the actual track coordinate point (or coordinate data) of the actual video frame of the jth frame in the first scanning track.

[0101] Furthermore, in one embodiment, the step of obtaining the image offset of each standard video frame corresponding to the actual video frame according to the standard trajectory coordinate points of each standard video frame in the second scanning trajectory and the actual trajectory coordinate points of the actual video frame corresponding to each standard video frame in the first scanning trajectory may also include: calculating the first offset between the first scanning trajectory coordinate points and the second scanning trajectory coordinate points; calculating the second offset between the actual trajectory coordinate points of each actual video frame in the first scanning trajectory and the standard trajectory coordinate points of the standard video frame corresponding to each actual video frame in the second scanning trajectory; and obtaining the image offset corresponding to each actual video frame according to the first offset and the second offset corresponding to each actual video frame.

[0102] Since the first key frame in the ultrasound video image and the second key frame in the standard video image are corresponding video frames, the first offset can be calculated based on the first scanning trajectory coordinate point of the first key frame and the second scanning trajectory coordinate point of the second key frame, and then the image offsets of other video frames in the ultrasound video image (or standard video image) are corrected and adjusted based on the first offset, so that the image offset of the actual video frame corresponding to the standard video frame is more accurate, and the proximity between the subsequently obtained target video image and the standard video image is improved, so that the final target video image is closer to the standard section of the body organ or physiological tissue, and the target area (i.e., the body organ or physiological tissue) can be seen to the greatest extent.

[0103] Continuing to take the i-th standard video frame in the standard video image as an example, the corresponding one in the ultrasound video image is the j-th actual video frame, where j=Δk+i; for the j-th actual video frame corresponding to the i-th standard video frame, its image offset can be obtained by the following formula:

[0104] T=P k -P k′

[0105] ΔT i =P i +TP j, i=1,2,3,…;j=Δk+i

[0106] Where T represents the first offset, P k represents the second scanning track coordinate point (or coordinate data) of the second key frame in the standard video image in the second scanning track, P k′ represents the first scanning track coordinate point (or coordinate data) of the first key frame in the ultrasound video image in the first scanning track; ΔT i It refers to the image offset of the jth actual video frame corresponding to the i-th standard video frame; P i refers to the standard track coordinate point (or coordinate data) of the standard video frame of the i-th frame in the second scanning track; P j It refers to the actual track coordinate point (or coordinate data) of the actual video frame of the jth frame in the first scanning track.

[0107] In addition, in the case where the standard video frame does not correspond to the actual video frame, the target ultrasound video image can be obtained from the three-dimensional complete image through stitching processing based on the ultrasound video image. Specifically, in one embodiment, the target ultrasound video image is obtained from the three-dimensional complete image through stitching processing based on the ultrasound video image, including: taking any standard video frame as the target video frame in turn, if the number of actual video frames corresponding to the target video frame is two or more, determining the target actual video frame from the actual video frame corresponding to the target video frame; if the number of actual video frames corresponding to the target video frame is zero, generating a blank video frame as the target actual video frame corresponding to the target video frame; if the number of actual video frames corresponding to the target video frame is one, determining the target actual video frame from the actual video frame corresponding to the target video frame; based on the target actual video frames corresponding to each standard video frame, obtaining the target ultrasound video image from the three-dimensional complete image.

[0108] The standard video frames in the standard video image are determined as target video frames in sequence, and then the number of actual video frames corresponding to the target video frames is determined.

[0109] If the number of actual video frames corresponding to the target video frame is two or more, it can be confirmed that the scanning tracks corresponding to different ultrasound video images overlap, such as Figure 5As shown, at this time, one frame can be selected from the multiple actual video frames corresponding to the target video frame to be determined as the target actual video frame corresponding to the target video frame; further, among the multiple actual video frames corresponding to the target video frame, the actual video frame closest to the target standard video can be selected as the target actual video frame corresponding to the target video frame. More specifically, the distance between the target video frame and each corresponding actual video frame can be calculated based on the track coordinate points of the target video frame in the second scanning track and the track coordinate points of each corresponding actual video frame in the first scanning track; among the multiple actual video frames corresponding to the target video frame, the actual video frame with the clearest image and the highest image quality can be obtained as the target video frame corresponding to the target video frame. It can be understood that there are many methods for selecting the target video frame corresponding to the target video frame among the multiple actual video frames corresponding to the target video frame, which are not limited here.

[0110] If the number of actual video frames corresponding to the target video frame is zero, the scanning tracks corresponding to the multiple ultrasound video images are broken, such as Figure 6 As shown, a blank video frame can be generated at this time, and the blank video frame is used as the target actual video frame of the target video frame. The coordinate data of the blank video frame is the same as the coordinate data of the track coordinate point of the target video frame in the second scanning track.

[0111] If the data of the actual video frame corresponding to the target video frame is one, it can be confirmed that the number of ultrasound video images is one (such as Figure 7 As shown), or the number of ultrasound video images is multiple but the corresponding scanning tracks are not overlapped or broken, at this time, the actual video frame corresponding to the target video frame can be directly determined as the target video frame corresponding to the target video frame.

[0112] After obtaining the target actual video frames corresponding to each standard video frame in the standard video image, the corresponding image information can be obtained from the three-dimensional complete image and spliced ​​into a target video image based on the scanning coordinate points (or coordinate data) of each target actual video frame in the first scanning trajectory; it is also possible to obtain the image offset of the target actual video frame corresponding to each standard video frame based on the standard trajectory coordinate points of each standard video frame in the second scanning trajectory and the actual trajectory coordinate points of the target actual video frame corresponding to each standard video frame in the first scanning trajectory, and then obtain the target video frame corresponding to each standard video frame from the three-dimensional complete image based on the image offset of the target actual video frame corresponding to each standard video frame in the ultrasonic video image to obtain the target ultrasonic video image.

[0113] It is understandable that, in one embodiment, after the target actual video frames corresponding to the respective standard video frames in the standard video image are acquired, the respective target actual video frames may be directly spliced ​​into the target video image.

[0114] In the above-mentioned ultrasonic video image processing method, the ultrasonic video image to be processed of the target area is obtained; each of the ultrasonic video images is combined and processed to obtain a three-dimensional complete image of the target area; the three-dimensional complete image is processed according to the ultrasonic video image and the pre-stored standard video image of the target area to obtain the target ultrasonic video image. By comparing the ultrasonic video image actually scanned by the target area with the pre-stored standard video image, the three-dimensional complete image is processed based on the comparison result to obtain a target ultrasonic video image that is closer to the standard video image, so that the video frame in the target ultrasonic video image finally presented can present image information that is closer to the standard section of the body organ or physiological tissue, present the target area (i.e., the body organ or physiological tissue) to the greatest extent, improve the quality of the ultrasonic video image, and facilitate subsequent medical staff to view and obtain information in the target area.

[0115] In order to better implement the ultrasonic video image processing method in the embodiment of the present application, based on the ultrasonic video image processing method, an ultrasonic video image processing device is also provided in the embodiment of the present application. Figure 8 As shown, Figure 8 800 is a schematic diagram of the structure of an ultrasonic video image processing device provided in an embodiment of the present application. The ultrasonic video image processing device 800 includes:

[0116] An acquisition module 810 is used to acquire an ultrasonic video image of a target area to be processed;

[0117] A stitching module 820 is used to combine and process the ultrasound video images to obtain a three-dimensional complete image of the target area;

[0118] The processing module 830 is used to process the three-dimensional complete image according to the ultrasonic video image and the pre-stored standard video image of the target area to obtain the target ultrasonic video image.

[0119] In some embodiments of the present application, the processing module is specifically used to obtain a first scanning trajectory of an ultrasonic video image in a target area, and a second scanning trajectory of a standard video image in the target area; determine a first key frame from the ultrasonic video image, and obtain a first trajectory coordinate point of the first key frame in the first scanning trajectory; determine a second key frame from the standard video image, and obtain a second trajectory coordinate point of the second key frame in the second scanning trajectory; based on the correspondence between the first trajectory coordinate point and the second trajectory coordinate point, obtain a target ultrasonic video image from a three-dimensional complete image in combination with the ultrasonic video image.

[0120] In some embodiments of the present application, the processing module is specifically used to determine the difference in video frame numbers between a first key frame in an ultrasonic video image and a second key frame corresponding to a standard video image based on the coordinate point numbers of the first trajectory coordinate points and the coordinate point numbers of the second trajectory coordinate points; based on the video frame number differences, actual video frames corresponding to each standard video frame in the standard video image are obtained in the ultrasonic video image; if the standard video frame corresponds to the actual video frame, a target ultrasonic video image is obtained from the three-dimensional complete image through offset processing based on the ultrasonic video image; if the standard video frame does not correspond to the actual video frame, a target ultrasonic video image is obtained from the three-dimensional complete image through stitching processing based on the ultrasonic video image.

[0121] In some embodiments of the present application, the processing module is specifically used to obtain the image offset of each standard video frame corresponding to the actual video frame based on the standard trajectory coordinate points of each standard video frame in the second scanning trajectory and the actual trajectory coordinate points of the actual video frame corresponding to each standard video frame in the first scanning trajectory; based on the image offset of the actual video frame corresponding to each standard video frame in the ultrasonic video image, obtain the target video frame corresponding to each standard video frame from the three-dimensional complete image to obtain the target ultrasonic video image.

[0122] In some embodiments of the present application, the processing module is specifically used to calculate a first offset between a first scanning trajectory coordinate point and a second scanning trajectory coordinate point; calculate a second offset between an actual trajectory coordinate point of each actual video frame in the first scanning trajectory and a standard trajectory coordinate point of a standard video frame corresponding to each actual video frame in the second scanning trajectory; and obtain an image offset corresponding to each actual video frame based on the first offset and the second offset corresponding to each actual video frame.

[0123] In some embodiments of the present application, the processing module is specifically used to take any standard video frame as the target video frame in turn. If the number of actual video frames corresponding to the target video frame is two or more, the target actual video frame is determined from the actual video frames corresponding to the target video frame; if the number of actual video frames corresponding to the target video frame is zero, a blank video frame is generated as the target actual video frame corresponding to the target video frame; if the number of actual video frames corresponding to the target video frame is one, the target actual video frame is determined from the actual video frames corresponding to the target video frame; based on the target actual video frames corresponding to each standard video frame, the target ultrasound video image is acquired from the three-dimensional complete image.

[0124] In some embodiments of the present application, the stitching module is specifically used to obtain video frames in the ultrasound video image; each video frame is input into a preset tissue feature recognition model to obtain tissue feature information corresponding to each video frame; the tissue feature recognition model is obtained by inputting sample video frames of marked scanned tissue features into a deep learning neural network for iterative training; and each video frame is combined and processed according to the tissue feature information corresponding to each video frame to obtain a three-dimensional complete image.

[0125] The present application also provides an ultrasonic video image processing device, such as Fig. 9 As shown, Fig. 9 It is a structural schematic diagram of an ultrasonic video image processing device provided in an embodiment of the present application.

[0126] The ultrasonic video image processing device includes a memory, a processor, and an ultrasonic video image processing program stored in the memory and executable on the processor. When the processor executes the ultrasonic video image processing program, the steps in the ultrasonic video image processing method provided in any embodiment of the present application are implemented.

[0127] Specifically, the ultrasonic video image processing device may include one or more processing core processors 901, one or more storage media memories 902, a power supply 903, an input unit 904 and other components. Those skilled in the art will understand that Fig. 9 The structure of the ultrasonic video image processing device shown in the figure does not constitute a limitation on the ultrasonic video image processing device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0128] The processor 901 is the control center of the ultrasonic video image processing device. It uses various interfaces and lines to connect various parts of the entire ultrasonic video image processing device. By running or executing software programs and / or modules stored in the memory 902, and calling data stored in the memory 902, it performs various functions of the ultrasonic video image processing device and processes data, thereby monitoring the ultrasonic video image processing device as a whole. Optionally, the processor 901 may include one or more processing cores; preferably, the processor 901 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 901.

[0129] The memory 902 can be used to store software programs and modules. The processor 901 executes various functional applications and data processing by running the software programs and modules stored in the memory 902. The memory 902 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the ultrasonic video image processing device, etc. In addition, the memory 902 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 902 may also include a memory controller to provide the processor 901 with access to the memory 902.

[0130] The ultrasonic video image processing device also includes a power supply 903 for supplying power to various components. Preferably, the power supply 903 can be logically connected to the processor 901 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 903 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0131] The ultrasonic video image processing device may further include an input unit 904, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0132] Although not shown, the ultrasonic video image processing device may also include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 901 in the ultrasonic video image processing device will load the executable files corresponding to the processes of one or more application programs into the memory 902 according to the following instructions, and the processor 901 will run the application programs stored in the memory 902, thereby implementing the steps in the ultrasonic video image processing method provided in any embodiment of the present application.

[0133] To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. The computer-readable storage medium stores an ultrasound video image processing program, and when the ultrasound video image processing program is executed by a processor, the steps in the ultrasound video image processing method provided in any embodiment of the present application are implemented.

[0134] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.

[0135] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.

[0136] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0137] The above is a detailed introduction to an ultrasonic video image processing method provided in an embodiment of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for processing ultrasonic video images, characterized in that: The ultrasonic video image processing method comprises: Acquire the ultrasonic video image to be processed in the target area; Combining and processing the ultrasonic video images to obtain a three-dimensional complete image of the target area; Processing the three-dimensional complete image according to the ultrasonic video image and the pre-stored standard video image of the target area to obtain a target ultrasonic video image includes: Acquire a first scanning trajectory of the ultrasonic video image in the target area, and a second scanning trajectory of the standard video image in the target area; Determine a first key frame from the ultrasound video image, and obtain a first track coordinate point of the first key frame in the first scanning track; Determine a second key frame from the standard video image, and obtain a second track coordinate point of the second key frame in the second scanning track; Determine, according to the coordinate point serial number of the first trajectory coordinate point and the coordinate point serial number of the second trajectory coordinate point, a video frame serial number difference between a first key frame in the ultrasound video image and a second key frame corresponding to the standard video image; Based on the video frame sequence number difference, obtaining in the ultrasound video image actual video frames corresponding to each standard video frame in the standard video image; If the standard video frame corresponds to the actual video frame, obtaining a target ultrasound video image from the three-dimensional complete image through an offset process based on the ultrasound video image; If the standard video frame does not correspond to the actual video frame, a target ultrasound video image is acquired from the three-dimensional complete image through stitching processing based on the ultrasound video image.

2. The ultrasonic video image processing method according to claim 1, characterized in that: The step of obtaining a target ultrasound video image from the three-dimensional complete image through offset processing based on the ultrasound video image includes: According to the standard track coordinate points of each standard video frame in the second scanning track and the actual track coordinate points of the actual video frame corresponding to each standard video frame in the first scanning track, obtaining the image offset of each standard video frame corresponding to the actual video frame; Based on the image offset of the actual video frame corresponding to each of the standard video frames in the ultrasonic video image, the target video frame corresponding to each of the standard video frames is acquired from the three-dimensional complete image to obtain the target ultrasonic video image.

3. The ultrasonic video image processing method according to claim 2, characterized in that: The step of obtaining the image offset of each standard video frame corresponding to the actual video frame according to the standard track coordinate points of each standard video frame in the second scanning track and the actual track coordinate points of each standard video frame corresponding to the actual video frame in the first scanning track comprises: Calculating a first offset between the first scanning trajectory coordinate point and the second scanning trajectory coordinate point; Calculating a second offset between an actual track coordinate point of each actual video frame in the first scanning track and a standard track coordinate point of a standard video frame corresponding to each actual video frame in the second scanning track; The image offset corresponding to each of the actual video frames is obtained according to the first offset and the second offset corresponding to each of the actual video frames.

4. The ultrasonic video image processing method according to claim 2, characterized in that: The step of obtaining a target ultrasound video image from the three-dimensional complete image through stitching based on the ultrasound video image includes: Taking any of the standard video frames as target video frames in sequence, if the number of actual video frames corresponding to the target video frames is two or more, determining a target actual video frame from the actual video frames corresponding to the target video frames; If the number of actual video frames corresponding to the target video frame is zero, generating a blank video frame as a target actual video frame corresponding to the target video frame; If the number of actual video frames corresponding to the target video frame is one, determining a target actual video frame from the actual video frames corresponding to the target video frame; Based on the target actual video frames corresponding to the standard video frames, a target ultrasound video image is acquired from the three-dimensional complete image.

5. The ultrasonic video image processing method according to any one of claims 1 to 4, characterized in that: The combining and processing of the ultrasonic video images to obtain a three-dimensional complete image of the target area includes: Acquire a video frame in the ultrasound video image; Input each of the video frames into a preset tissue feature recognition model to obtain tissue feature information corresponding to each of the video frames; the tissue feature recognition model is obtained by inputting sample video frames of labeled scanned tissue features into a deep learning neural network for iterative training; The video frames are combined and processed according to the tissue feature information corresponding to each of the video frames to obtain a three-dimensional complete image.

6. An ultrasonic video image processing device, characterized in that: The ultrasonic video image processing device comprises: An acquisition module, used for acquiring an ultrasonic video image to be processed in a target area; A splicing module, used for combining and processing the ultrasonic video images to obtain a three-dimensional complete image of the target area; A processing module is used to process the three-dimensional complete image according to the ultrasonic video image and the pre-stored standard video image of the target area to obtain a target ultrasonic video image; wherein the processing module is also used to: Acquire a first scanning trajectory of the ultrasonic video image in the target area, and a second scanning trajectory of the standard video image in the target area; Determine a first key frame from the ultrasound video image, and obtain a first track coordinate point of the first key frame in the first scanning track; Determine a second key frame from the standard video image, and obtain a second track coordinate point of the second key frame in the second scanning track; Determine, according to the coordinate point serial number of the first trajectory coordinate point and the coordinate point serial number of the second trajectory coordinate point, a video frame serial number difference between a first key frame in the ultrasound video image and a second key frame corresponding to the standard video image; Based on the video frame sequence number difference, obtaining in the ultrasound video image actual video frames corresponding to each standard video frame in the standard video image; If the standard video frame corresponds to the actual video frame, obtaining a target ultrasound video image from the three-dimensional complete image through an offset process based on the ultrasound video image; If the standard video frame does not correspond to the actual video frame, a target ultrasound video image is acquired from the three-dimensional complete image through stitching processing based on the ultrasound video image.

7. An ultrasonic video image processing device, characterized in that: The ultrasonic video image processing device includes: a processor, a memory, and an ultrasonic video image processing program stored in the memory and executable on the processor, and the processor executes the ultrasonic video image processing program to implement the steps in the ultrasonic video image processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an ultrasonic video image processing program, and the ultrasonic video image processing program is executed by a processor to implement the steps in the ultrasonic video image processing method according to any one of claims 1 to 5.

Images