Ultrasonic image regularization method, device, equipment and computer readable storage medium

Abstract

The embodiments of the present application provide an ultrasound image regularization method, device, equipment and computer-readable storage medium, the method comprising: obtaining an ultrasound image to be regularized; processing the video frames in the ultrasound image to obtain type information corresponding to each video frame; the type information includes at least one of location information and lesion information; regularizing each video frame according to the type information corresponding to each video frame to obtain a target ultrasound image corresponding to each type. The embodiments of the present application process the video frames in the acquired multiple ultrasound images to obtain the type information corresponding to each video frame, and then regularize each video frame according to the type information corresponding to each video frame, so that video frames of the same type can be combined together, so that medical practitioners can quickly select the segments that need to be viewed from the multiple ultrasound images for continuous viewing, effectively improving the doctor's diagnosis effect.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of medical image processing technology, and specifically to an ultrasound image regularization method, device, equipment, and computer-readable storage medium. Background Art

[0002] Ultrasound scanning is a commonly used imaging method in the medical field. Usually, a handheld ultrasound device is used to press the human body, slowly scan it, and then generate an ultrasound scan video image. Usually, the generated ultrasound scan video image will contain images of multiple areas of the human body, so that doctors can make a complete diagnosis of the patient's condition based on the scanned ultrasound scan video image. In order to better improve the diagnostic effect, it is often necessary to scan the human body multiple times to obtain multiple ultrasound scan video images.

[0003] However, after scanning the human body to obtain multiple ultrasound scan video images, doctors still need to analyze the ultrasound scan video images one by one, and cannot view a certain target area in a coherent manner, which affects the diagnosis effect. Summary of the invention

[0004] The embodiments of the present application provide an ultrasound image regularization method, device, equipment and computer-readable storage medium, which are intended to solve the technical problem in the prior art that after scanning a human body to obtain multiple ultrasound scanning video images, a doctor cannot view a certain target area in a coherent manner, thus affecting the diagnosis effect.

[0005] On the one hand, an embodiment of the present application provides an ultrasound image regularization method, comprising:

[0006] Acquiring ultrasound images to be organized;

[0007] Processing the video frames in the ultrasound image to obtain type information corresponding to each of the video frames; the type information includes at least one of location information and lesion information;

[0008] The video frames are sorted according to the type information corresponding to each of the video frames to obtain target ultrasound images corresponding to each type.

[0009] As a feasible embodiment of the present application, the video frames are sorted according to the type information corresponding to each video frame to obtain target ultrasound images corresponding to each type, including:

[0010] Dividing the ultrasound image according to the part information corresponding to each of the video frames to obtain a plurality of sub-ultrasound images, and combining the sub-ultrasound images corresponding to the same part to obtain a target ultrasound image corresponding to each part; and / or

[0011] Lesion video frames containing lesions are extracted from each of the initial ultrasound images according to the lesion information corresponding to each of the video frames, and video frames corresponding to the same lesion are combined according to the matching results between the lesion video frames to obtain target ultrasound images corresponding to each lesion.

[0012] As a feasible embodiment of the present application, according to the matching results between the lesion video frames, the video frames corresponding to the same lesion are combined to obtain the target ultrasound image corresponding to each lesion, including:

[0013] According to the part information corresponding to each of the video frames, image stitching is performed on each of the video frames to reconstruct the part image corresponding to each of the part information;

[0014] According to the position information of each lesion video frame in the part image, the lesion video frames are matched to obtain the target ultrasound image corresponding to each lesion.

[0015] As a feasible embodiment of the present application, according to the matching results between the lesion video frames, the video frames corresponding to the same lesion are combined to obtain the target ultrasound image corresponding to each lesion, including:

[0016] Extracting image features of each of the lesion video frames, and calculating a transformation matrix between each of the lesion video frames according to the image features of each of the lesion video frames;

[0017] Extracting corresponding physiological reference points from the lesion video frames respectively according to the transformation matrix between the lesion video frames;

[0018] Each of the lesion video frames is matched according to the relative position relationship between the lesion and the physiological reference point in each of the lesion video frames to obtain a target ultrasound image corresponding to each lesion.

[0019] As a feasible embodiment of the present application, the processing of the video frames in the ultrasound image to obtain the type information corresponding to each of the video frames includes:

[0020] Dividing the video frames in the ultrasound image according to a preset number of frames to obtain video frame groups;

[0021] Processing the video frame group according to a preset part recognition model to obtain part information corresponding to each video frame in the video frame group; and / or

[0022] The video frame group is processed according to a preset lesion recognition model to obtain lesion information corresponding to each video frame in the video frame group.

[0023] As a feasible embodiment of the present application, before dividing the video frames in the ultrasound image according to the preset number of frames to obtain the video frame groups, the method further includes:

[0024] Determining a quality monitoring result corresponding to the ultrasound image according to a scanning operation indicator corresponding to each video frame in the ultrasound image;

[0025] A preset database is queried to obtain the number of associated frames corresponding to the quality monitoring result, and the number of associated frames is used as the preset number of frames.

[0026] As a feasible embodiment of the present application, after the video frames are sorted according to the type information corresponding to each video frame to obtain the target ultrasound image corresponding to each type, the method further includes:

[0027] The target ultrasound images corresponding to each type are respectively associated and stored with each type, and a query interface is generated according to the type;

[0028] A query request triggered based on the query interface is received, and according to the type selected on the query interface, a target ultrasound image corresponding to the type is output.

[0029] As a feasible embodiment of the present application, the outputting of the target ultrasound image corresponding to the type includes:

[0030] generating a playback interface according to the sub-ultrasound image contained in the target ultrasound image;

[0031] A play request triggered based on the play interface is received, and according to the target identifier selected on the play interface, a target sub-ultrasound image corresponding to the target identifier is output.

[0032] As a feasible embodiment of the present application, after processing the video frames in the ultrasound image to obtain the type information corresponding to each of the video frames, the method further includes:

[0033] According to the part information in the type information corresponding to each of the video frames, the lesion information in the type information corresponding to each of the video frames is matched, and the association relationship between the part information and the lesion information is determined and output.

[0034] As a feasible embodiment of the present application, the step of obtaining the ultrasound image to be organized includes:

[0035] Scanning the target area according to different preset angles or preset scanning intervals to obtain at least two initial ultrasound images corresponding to the target area;

[0036] The at least two initial ultrasound images are used as the ultrasound images to be normalized.

[0037] On the other hand, an embodiment of the present application further provides an ultrasonic image integration device, comprising:

[0038] An acquisition module, used for acquiring the ultrasound image to be organized;

[0039] An identification module, used to process the video frames in the ultrasound image to obtain type information corresponding to each of the video frames; the type information includes at least one of location information and lesion information;

[0040] The sorting module is used to sort the video frames according to the type information corresponding to each video frame to obtain the target ultrasound image corresponding to each type.

[0041] On the other hand, an embodiment of the present application also provides an ultrasonic image regularization device, which includes a processor, a memory, and an ultrasonic image regularization program stored in the memory and executable on the processor, and the processor executes the ultrasonic image regularization program to implement the steps in the above-mentioned ultrasonic image regularization method.

[0042] On the other hand, an embodiment of the present application also provides a computer-readable storage medium, wherein the ultrasound image regularization device includes a processor, a memory, and an ultrasound image regularization program stored in the memory and executable on the processor, and the processor executes the ultrasound image regularization program to implement the steps in the above-mentioned ultrasound image regularization method.

[0043] The ultrasound image regularization method provided in the embodiment of the present application processes the video frames in the acquired multiple ultrasound images to obtain type information corresponding to each video frame, and then organizes each video frame according to the type information corresponding to each video frame, so that video frames of the same type can be combined together, thereby facilitating medical practitioners to quickly select the segments that need to be viewed from the multiple ultrasound images for continuous viewing, thereby effectively improving the doctor's diagnostic effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 A schematic diagram of an implementation scenario of an ultrasound image regularization method provided in an embodiment of the present application;

[0046] Figure 2 A schematic diagram of the steps of an ultrasound image regularization method provided in an embodiment of the present application;

[0047] Figure 3 A schematic diagram of a process flow for regularizing a video frame provided in an embodiment of the present application;

[0048] Figure 4 A schematic diagram of a process flow based on combining video frames of lesions provided in an embodiment of the present application;

[0049] Figure 5 A schematic diagram of another step flow of combining video frames based on lesions provided in an embodiment of the present application;

[0050] Figure 6 A schematic flow chart of steps for processing a video frame to obtain a type provided in an embodiment of the present application;

[0051] Figure 7 A schematic diagram of a process flow for grouping video frames based on quality provided in an embodiment of the present application;

[0052] Figure 8 A schematic diagram of a process flow for displaying a regularized video frame provided in an embodiment of the present application;

[0053] Fig. 9 A schematic diagram of a process flow of playing ultrasound images based on user selection provided in an embodiment of the present application;

[0054] Fig.10 A schematic diagram of a process flow for obtaining an ultrasound image provided in an embodiment of the present application;

[0055] Fig.11 A schematic diagram of the structure of an ultrasonic image regularization device provided in an embodiment of the present application;

[0056] Fig.12 A schematic diagram of the structure of an ultrasonic image regularization device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of the present invention.

[0058] In the embodiments of the present application, the word "exemplary" is used to mean "used as an example, illustration or description". Any embodiment described as "exemplary" in the embodiments of the present application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid unnecessary details that make the description of the present invention obscure. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in the embodiments of the present application.

[0059] In the embodiments of the present application, a method, device, equipment and computer-readable storage medium for regularizing ultrasound images are provided, which are described in detail below.

[0060] In the embodiment of the present application, the ultrasonic image regularization method is deployed on the ultrasonic image regularization device in the form of a program, and the ultrasonic image regularization device is installed in the ultrasonic image regularization device in the form of a processor. The ultrasonic image regularization device in the ultrasonic image regularization device executes the query and output process of the target ultrasonic image by running the program corresponding to the ultrasonic image regularization method.

[0061] like Figure 1 As shown, Figure 1 A schematic diagram of an implementation scenario of an ultrasonic image regularization method provided in an embodiment of the present application is provided, and the implementation scenario provided in the embodiment of the present application includes an ultrasonic image regularization device 100 and an ultrasonic image acquisition device 200. The ultrasonic image acquisition device 200 is mainly used to scan and acquire multiple ultrasonic images and transmit them to the ultrasonic image regularization device 100. After receiving the multiple ultrasonic images transmitted by the ultrasonic image acquisition device 200, the ultrasonic image regularization device 100 executes the ultrasonic image regularization method by running the computer storage medium corresponding to the ultrasonic image regularization method, thereby completing the regularization of the multiple ultrasonic images, so that the user can select the regularized ultrasonic images according to actual needs.

[0062] It should be noted that Figure 1 The schematic diagram of the implementation scenario of the ultrasound image regularization method shown is merely an example. The scenario of the ultrasound image regularization method described in the embodiment of the present application is intended to more clearly illustrate the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application.

[0063] Based on the above implementation scenario diagram of the ultrasound image regularization method, a specific implementation example of the ultrasound image regularization method is proposed.

[0064] like Figure 2 As shown, Figure 2 A schematic diagram of the steps of an ultrasound image regularization method provided in an embodiment of the present application. The ultrasound image regularization method in the embodiment of the present application includes steps 201 to 203:

[0065] 201, obtaining an ultrasound image to be organized.

[0066] In the embodiments of the present application, the acquired ultrasound images to be organized generally refer to multiple ultrasound images obtained by scanning the same area at different scanning angles in different time periods. For example, as an optional embodiment of the present application, the acquired ultrasound images to be organized include ultrasound images obtained in real time during the review process, and historical ultrasound images obtained during the initial diagnosis process. By organizing these two ultrasound images, extracting video clips of the same part or lesion in the two ultrasound images and combining them together, it is convenient for medical practitioners to understand the changes in the lesion during this period of time, thereby improving the efficiency of subsequent diagnosis. Of course, in addition to the above-mentioned implementation scenarios, ultrasound images can also be obtained by scanning the same area at different scanning angles, which can more specifically, completely and clearly display the scanning conditions of the area, thereby improving the diagnostic accuracy of medical practitioners. For specific implementation schemes, please refer to the subsequent figures.

[0067] 202 : Process the video frames in the ultrasound image to obtain type information corresponding to each of the video frames.

[0068] In the embodiment of the present application, the type information includes at least one of the part information and the lesion information. Specifically, the video frame is processed to obtain the part information and the lesion information, which can be realized based on a network model obtained by deep learning training, and the embodiment of the present application is not repeated here.

[0069] Furthermore, in the process of processing the video frames in the ultrasound image, in order to improve the recognition accuracy of the video frame type information, the video frames in the ultrasound image may be divided into a plurality of groups in a grouping manner, and the type information corresponding to the video frames in each group is determined. For a specific implementation scheme, please refer to the subsequent Figure 6 The embodiments of the present application will not be described in detail here.

[0070] 203 , organize the video frames according to the type information corresponding to the video frames to obtain target ultrasound images corresponding to each type.

[0071] In the embodiment of the present application, by recombining video frames corresponding to the same type of information, target ultrasound images corresponding to different types of information can be obtained. For example, as an optional embodiment of the present application, when video frames corresponding to the same part, such as video frames corresponding to the liver, are combined together, a target ultrasound image of the liver part can be obtained, and subsequent medical practitioners can quickly find the complete scan image related to the liver part. For specific implementation schemes, please refer to the subsequent Figure 3 and its explanation.

[0072] In addition, after obtaining the target ultrasound images corresponding to each type, the ultrasound image regularization device will further generate a corresponding query interface based on the identified type information to facilitate subsequent medical practitioners to view. Figure 8 and its explanatory contents.

[0073] The ultrasound image regularization method provided in the embodiment of the present application obtains at least one target type information of the target part type and the target lesion type selected on the preset selection interface, screens the initial video frames in the ultrasound image, thereby obtaining target video frames that match the corresponding target type information, and combines the target video frames for display, thereby facilitating medical practitioners to quickly select the segments they need to view from multiple ultrasound images for continuous viewing, thereby effectively improving the doctor's diagnostic effect.

[0074] like Figure 3 As shown, Figure 3 A schematic flow chart of a step of regularizing a video frame provided in an embodiment of the present application, specifically, comprising steps 301 to 302:

[0075] 301 , dividing the ultrasound image according to the part information corresponding to each of the video frames to obtain a plurality of sub-ultrasound images, and combining the sub-ultrasound images corresponding to the same part to obtain a target ultrasound image corresponding to each part.

[0076] In the embodiment of the present application, after obtaining the part information corresponding to each video frame, the ultrasound image is divided using the part information corresponding to each video frame, so that a plurality of sub-ultrasound images corresponding to different part information can be obtained, and then all the sub-ultrasound images are integrated, and the sub-ultrasound images corresponding to the same part are combined together, so that the target ultrasound image corresponding to each part can be obtained. In other words, the part information corresponding to each sub-ultrasound image in the target ultrasound image is the same.

[0077] To facilitate understanding of the technical solution provided in the embodiments of the present application, specifically, taking the following as an example, frames 1 to 30 in the first video image are the liver, frames 20 to 60 and frames 90 to 130 in the second video image are all the liver. At this time, frames 1 to 30 in the first video image and frames 20 to 60 and frames 90 to 130 in the second video image can be combined together as sub-video images to form a target ultrasound image associated with the liver area.

[0078] 302, extracting lesion video frames where lesions exist from each of the initial ultrasound images according to the lesion information corresponding to each of the video frames, and combining video frames corresponding to the same lesion according to the matching results between the lesion video frames to obtain target ultrasound images corresponding to each lesion.

[0079] In the embodiment of the present application, similar to the scheme provided above, after obtaining the lesion information corresponding to each video frame, the ultrasound image regularization device will extract the lesion video frame where the lesion exists, but it should be noted that there may be multiple different lesions of the same type in the same part. For example, there may be multiple infarctions in the ultrasound image corresponding to the heart, and infarctions in different areas should be regarded as different lesions. Therefore, it is also necessary to match the lesions in the lesion video frame to determine whether they are the same lesion, that is, according to the matching results between the lesion video frames, the video frames corresponding to the same lesion are combined to obtain the target ultrasound images corresponding to each lesion.

[0080] Specifically, as a feasible embodiment of the present application, matching the lesions in the lesion video frame to determine whether they are the same lesion can be achieved by reconstructing the part image and based on the position information of the lesion. The specific implementation scheme can be referred to in the subsequent Figure 4 and its explanatory contents.

[0081] Of course, in addition to the aforementioned method of reconstructing the part image and matching the lesion based on the location information of the lesion, in the absence of a part as a reference, the lesion can also be matched through the physiological reference points in the ultrasound image. Figure 5 and its explanatory contents.

[0082] like Figure 4 As shown, Figure 4 A schematic flowchart of the steps of combining video frames based on lesions is provided in an embodiment of the present application, which is described in detail as follows.

[0083] In the embodiment of the present application, a solution is provided for matching lesions based on image reconstruction and using the location information of the lesions, which specifically includes steps 401 to 402:

[0084] 401 , performing image stitching on each of the video frames according to the part information corresponding to each of the video frames, and reconstructing a part image corresponding to each of the part information.

[0085] In the embodiment of the present application, after obtaining the part information corresponding to each video frame, the video frames corresponding to the same part are stitched to reconstruct the part image of the part. The image stitching can be implemented based on the existing image stitching technology, which will not be described in detail in the embodiment of the present application.

[0086] 402 , matching the lesion video frames according to the position information of each lesion video frame in the part image to obtain a target ultrasound image corresponding to each lesion.

[0087] In an embodiment of the present application, after reconstructing the part image, the lesion video frames are mapped to the part image, and the position information of each lesion video frame and each lesion therein can be obtained. Based on this position information, the lesions with corresponding position information are regarded as the same lesion, and the lesion video frames are matched to obtain the target ultrasound image corresponding to each lesion.

[0088] To facilitate understanding of the technical solution provided by the embodiment of the present application, specifically, taking the following as an example, after forming a target ultrasound image associated with the liver part, the ultrasound image regularization device will further perform liver segmentation on each frame of the image, and parse the liver segmentation information in each frame of the image, thereby reconstructing the corresponding liver image. Specifically, taking the reconstruction of the first video image as region A of the liver from frames 1 to 30, the second video image as region A′ of the liver from frames 20 to 60, and the liver region B of frames 90 to 130 as an example, it can be determined that the liver image of frames 20 to 60 in the second video image is the same region as the liver image of frames 1 to 30 in the first video image, and the liver image of frames 90 to 130 in the second image is another region of the liver. Therefore, the lesions in frames 1 to 30 in the first video image and frames 20 to 60 in the second video image are the same lesion, such as lesion X, and the liver image of frames 20 to 60 in the second video image is another lesion, such as lesion Y. At this time, the 1st to 30th frames in the first video image and the 20th to 60th frames in the second video image can be further combined together to form a target ultrasound image associated with the lesion X.

[0089] like Figure 5 As shown, Figure 5 Another step flow chart of combining video frames based on lesions provided in an embodiment of the present application is described in detail as follows.

[0090] In the embodiment of the present application, a solution for matching lesions based on image transformation and physiological reference points is proposed, which specifically includes steps 501 to 503:

[0091] 501, extracting image features of each of the lesion video frames, and calculating a transformation matrix between each of the lesion video frames according to the image features of each of the lesion video frames.

[0092] In the embodiment of the present application, since the multiple video images obtained by scanning and acquisition are essentially adjusted in scanning angle or position, the image features of each lesion video frame are extracted, and the correlation relationship of the same coordinates between the multiple video images can be calculated according to the image features of each lesion video frame. This correlation relationship is the determined transformation matrix. Generally, the transformation matrix usually includes a rotation matrix and a translation matrix.

[0093] 502 , extracting corresponding physiological reference points from the lesion video frames respectively according to the transformation matrix between the lesion video frames.

[0094] In an embodiment of the present application, after determining the transformation matrix between each lesion video frame, that is, the correlation relationship between the same coordinates of multiple video images, the ultrasound image regularization device will extract at least two physiological reference points from each video image, wherein the physiological reference point is a reference point in the human body that can express the properties or types of special physiological tissue structures, and the type and number of physiological reference points extracted from each video image should be the same.

[0095] 503 , matching each of the lesion video frames according to the relative position relationship between the lesion in each of the lesion video frames and the physiological reference point, to obtain a target ultrasound image corresponding to each lesion.

[0096] In an embodiment of the present application, after extracting the physiological reference points in each video image, the ultrasound image regularization device calculates the relative position relationship between the lesions and the physiological reference points in each lesion video frame, and determines whether they are the same lesion through the relative coordinates between the lesions and the physiological reference points in multiple video images, that is, the lesion video frames can be matched, thereby obtaining the target ultrasound images corresponding to each lesion.

[0097] like Figure 6 As shown, Figure 6 A schematic flow chart of the steps of processing a video frame to obtain a type provided in an embodiment of the present application is described in detail as follows.

[0098] In the embodiment of the present application, steps 601 to 603 are specifically included:

[0099] 601 , dividing the video frames in the ultrasound image according to a preset number of frames to obtain video frame groups.

[0100] In the embodiment of the present application, in order to improve the accuracy of video frame type recognition, the ultrasound image regularization device will first divide the video frames in the ultrasound image according to a preset number of frames to obtain a number of video frame groups. Each video frame group includes a number of video frames, for example, 4 to 8 video frames.

[0101] Further, as an optional embodiment of the present application, the number of frames may be a preset fixed value, or may be set according to the quality of the ultrasound image. For specific implementation schemes, please refer to the subsequent Figure 7 and its explanatory contents.

[0102] 602 , processing the video frame group according to a preset part recognition model to obtain part information corresponding to each video frame in the video frame group.

[0103] In the embodiment of the present application, the preset part recognition model refers to a network model trained based on deep learning technology using a large number of training sample images and their corresponding part labels, such as the more common deep network model, convolutional network model, recurrent network model, etc. The embodiment of the present application does not elaborate on the specific implementation scheme of the part recognition model trained based on deep learning technology. Specifically, since the part recognition model is trained based on a large number of training sample images and their corresponding part labels, the part recognition model is used to process the video frame group to accurately identify the part information corresponding to each video frame.

[0104] 603 , processing the video frame group according to a preset lesion recognition model to obtain lesion information corresponding to each video frame in the video frame group.

[0105] In the embodiment of the present application, similar to the aforementioned part recognition model, the preset lesion recognition model refers to a network model trained based on deep learning technology using a large number of training sample images and their corresponding lesion labels, such as a relatively common deep network model, convolutional network model, recurrent network model, etc. The embodiment of the present application does not elaborate on the specific implementation scheme of the lesion recognition model trained based on deep learning technology. Specifically, since the lesion recognition model is trained based on a large number of training sample images and their corresponding lesion labels, the lesion recognition model is used to process the video frame group to accurately identify the lesion information corresponding to each video frame.

[0106] like Figure 7 As shown, Figure 7 A schematic flow chart of the steps of grouping video frames based on quality provided in an embodiment of the present application is described in detail as follows.

[0107] In an embodiment of the present application, a technical solution is provided for setting the number of associated frames based on the quality of the acquired ultrasound image, so as to be used for subsequent division to obtain a video frame group. Specifically, the technical solution includes steps 701 to 702:

[0108] 701. Determine a quality monitoring result corresponding to the ultrasound image according to a scanning operation index corresponding to each video frame in the ultrasound image.

[0109] In the embodiment of the present application, the scanning operation index mainly includes the index information used when collecting ultrasound images, for example, it may include the scanning depth, pressure, scanning speed and other indicators used in the scanning process. Of course, in addition to the above, the scanning operation index may also include information such as the brightness of the video frame.

[0110] In the embodiment of the present application, when determining the scanning operation index corresponding to each video frame, the quality monitoring result corresponding to the ultrasound image can be determined based on the information such as scanning depth, pressing force, scanning speed, brightness, etc. Specifically, the quality monitoring result can be described in a graded manner, such as excellent, good, passing, etc.

[0111] 702 , query a preset database to obtain the number of associated frames corresponding to the quality monitoring result, and use the number of associated frames as the preset number of frames.

[0112] In an embodiment of the present application, for ultrasound images with different quality monitoring results, corresponding different numbers of frames can be used for division to facilitate subsequent type identification. Specifically, the higher the quality of the ultrasound image, the smaller the number of frames used can be, and conversely, the worse the quality of the ultrasound image, the more frames can be used. For example, as an optional embodiment of the present application, for ultrasound images with excellent quality monitoring results, the number of frames selected can be 4, that is, the ultrasound image is divided into several video frame groups in a manner of dividing every 4 frames, for ultrasound images with good quality monitoring results, the number of frames selected can be 6, that is, the ultrasound image is divided into several video frame groups in a manner of dividing every 6 frames, and for ultrasound images with qualified quality monitoring results, the number of frames selected can be 8, that is, the ultrasound image is divided into several video frame groups in a manner of dividing every 8 frames.

[0113] Of course, the mapping relationship between the specific quality monitoring results and the number of associated frames can be pre-set and stored in a preset database. At this time, when the quality monitoring results corresponding to the ultrasound image are obtained, the corresponding number of associated frames can be directly obtained by querying this preset database for subsequent division.

[0114] like Figure 8 As shown, Figure 8A schematic flowchart of the steps for displaying regularized video frames provided in an embodiment of the present application is described in detail as follows.

[0115] In order to facilitate medical practitioners to better search and select regularized video frames, as an embodiment of the present application, a solution for displaying regularized video frames is provided, which specifically includes steps 801 to 802:

[0116] 801 , respectively associate target ultrasound images corresponding to each type with each type and store them, and generate a query interface according to the type.

[0117] In the embodiment of the present application, after the target ultrasound images corresponding to each type are determined, the target ultrasound images are associated with each type and stored, and the types are displayed on a preset interface, so that a query interface for querying can be obtained.

[0118] 802 , receiving a query request triggered by the query interface, and outputting a target ultrasound image corresponding to the type selected on the query interface.

[0119] In the embodiment of the present application, after displaying the type on the preset interface and obtaining the query interface for query, the medical practitioner can select the ultrasound image of his / her needs on the query interface. Specifically, the ultrasound image regularization device receives the query request triggered on the query interface, and outputs the target ultrasound image corresponding to the type according to the type selected on the query interface. For example, when the user selects the liver type on the query interface, the image regularization device outputs all ultrasound images related to the liver type obtained by the aforementioned regularization.

[0120] Furthermore, considering that the target ultrasound image is composed of several sub-ultrasound images, in order to further improve the efficiency of medical practitioners in querying ultrasound images, after outputting the target ultrasound image, a playback interface will be generated based on the sub-ultrasound images contained in the target ultrasound image. The specific implementation scheme can be found in the subsequent Fig. 9 and its explanatory contents.

[0121] like Fig. 9 As shown, Fig. 9 A schematic diagram of a flow chart of a step of playing an ultrasound image based on a user's selection is provided in an embodiment of the present application, specifically, including steps 901 to 902:

[0122] 901 : Generate a playback interface according to the sub-ultrasound image included in the target ultrasound image.

[0123] In the embodiment of the present application, after determining the target ultrasound image, the ultrasound image regularization device generates a playback interface based on the sub-ultrasound images contained in the target ultrasound image, wherein the playback interface includes an identifier corresponding to each sub-ultrasound image. Medical practitioners can select the sub-ultrasound image to be played from the target ultrasound image by selecting the identifier on the playback interface.

[0124] 902 , receiving a play request triggered by the play interface, and outputting a target sub-ultrasound image corresponding to the target identifier selected on the play interface.

[0125] In an embodiment of the present application, the ultrasound image regularization device receives a playback request triggered by a playback interface, and outputs a target sub-ultrasound image corresponding to the target identifier according to the target identifier selected on the playback interface.

[0126] like Fig.10 As shown, Fig.10 A schematic diagram of a process flow for obtaining an ultrasound image provided in an embodiment of the present application, specifically, includes steps 1001 to 1002:

[0127] 1001. Scan a target area according to preset different angles or preset scanning intervals to obtain at least two initial ultrasound images corresponding to the target area.

[0128] In an embodiment of the present application, the ultrasound images to be organized are usually obtained by scanning the same area at different scanning angles. For example, as an optional embodiment of the present application, the liver area may be scanned from the upper right to the lower left, and from the upper left to the lower right, so as to obtain two initial ultrasound images. Of course, the liver area may also be scanned separately during the initial examination and the reexamination, so as to obtain two initial ultrasound images. Among them, the ultrasound image obtained during the initial examination may be pre-stored in a database, and when the reexamination scan obtains the first ultrasound image, the corresponding second ultrasound image during the initial examination is obtained from the database.

[0129] 1002. Use the at least two initial ultrasound images as the ultrasound images to be integrated.

[0130] In order to better implement the ultrasound image regularization method provided in the embodiment of the present application, on the basis of the ultrasound image regularization method, an ultrasound image regularization device is also provided in the embodiment of the present application. Fig.11 As shown, Fig.11 A schematic diagram of the structure of an ultrasonic image regularization device provided in an embodiment of the present application. Specifically, the ultrasonic image regularization device includes:

[0131] An acquisition module 1101 is used to acquire the ultrasound image to be organized;

[0132] The identification module 1102 is used to process the video frames in the ultrasound image to obtain type information corresponding to each of the video frames; the type information includes at least one of the part information and the lesion information;

[0133] The sorting module 1103 is used to sort the video frames according to the type information corresponding to each video frame to obtain the target ultrasound image corresponding to each type.

[0134] In some embodiments of the present application, the above-mentioned integration module includes:

[0135] a part regularization submodule, for dividing the ultrasound image according to the part information corresponding to each of the video frames to obtain a plurality of sub-ultrasound images, and combining the sub-ultrasound images corresponding to the same part to obtain a target ultrasound image corresponding to each part; and / or

[0136] The lesion regularization module is used to extract lesion video frames containing lesions from each of the initial ultrasound images according to the lesion information corresponding to each of the video frames, and to combine video frames corresponding to the same lesion according to the matching results between the lesion video frames to obtain target ultrasound images corresponding to each lesion.

[0137] In some embodiments of the present application, the above-mentioned lesion regularization submodule includes:

[0138] A stitching unit, configured to stitch the video frames according to the part information corresponding to the video frames, and reconstruct part images corresponding to the part information;

[0139] The first matching unit is used to match the lesion video frames according to the position information of each lesion video frame in the part image to obtain the target ultrasound image corresponding to each lesion.

[0140] In some embodiments of the present application, the above-mentioned lesion regularization submodule includes:

[0141] A transformation matrix calculation unit, used for extracting image features of each lesion video frame, and calculating the transformation matrix between each lesion video frame according to the image features of each lesion video frame;

[0142] A feature point extraction unit, used for extracting corresponding physiological reference points from the lesion video frames respectively according to the transformation matrix between the lesion video frames;

[0143] The second matching unit is used to match each of the lesion video frames according to the relative position relationship between the lesion and the physiological reference point in each of the lesion video frames to obtain a target ultrasound image corresponding to each lesion.

[0144] In some embodiments of the present application, the above-mentioned identification module includes:

[0145] A division submodule, used for dividing the video frames in the ultrasound image according to a preset number of frames to obtain video frame groups;

[0146] a part recognition submodule, used to process the video frame group according to a preset part recognition model to obtain part information corresponding to each video frame in the video frame group; and / or

[0147] The lesion recognition submodule is used to process the video frame group according to a preset lesion recognition model to obtain lesion information corresponding to each video frame in the video frame group.

[0148] In some embodiments of the present application, the above identification module further includes:

[0149] A quality monitoring submodule, used to determine a quality monitoring result corresponding to the ultrasound image according to a scanning operation index corresponding to each video frame in the ultrasound image;

[0150] The frame quantity setting submodule is used to query a preset database, obtain the associated frame quantity corresponding to the quality monitoring result, and use the associated frame quantity as the preset frame quantity.

[0151] In some embodiments of the present application, the ultrasonic image regularization device further includes a display module, and the display module includes:

[0152] An interface generation submodule, used to associate and store target ultrasound images corresponding to each type with each type, and generate a query interface according to the type;

[0153] The image output submodule is used to receive a query request triggered based on the query interface, and output a target ultrasound image corresponding to the type selected on the query interface.

[0154] In some embodiments of the present application, the image output submodule includes:

[0155] An interface generating unit, configured to generate a playback interface according to the sub-ultrasound image contained in the target ultrasound image;

[0156] The sub-image selection unit is used to receive a playback request triggered by the playback interface, and output a target sub-ultrasound image corresponding to the target identifier selected on the playback interface.

[0157] In some embodiments of the present application, the above-mentioned ultrasound image regularization device also includes an association module, which is used to match the lesion information in the type information corresponding to each of the video frames according to the part information in the type information corresponding to each of the video frames, and determine and output the association relationship between the part information and the lesion information.

[0158] In some embodiments of the present application, the acquisition module includes:

[0159] A repeated scanning submodule is used to scan the target area according to different preset angles or preset scanning interval periods to obtain at least two initial ultrasonic images corresponding to the target area;

[0160] A submodule is set to use the at least two initial ultrasound images as the ultrasound images to be organized.

[0161] The present application also provides an ultrasonic image regularization device, such as Fig.12 As shown, Fig.12 A schematic diagram of the structure of an ultrasonic image regularization device provided in an embodiment of the present application.

[0162] The ultrasonic image regularization device includes a memory, a processor, and an ultrasonic image regularization program stored in the memory and executable on the processor. When the processor executes the ultrasonic image regularization program, the steps in the ultrasonic image regularization method provided in any embodiment of the present application are implemented.

[0163] Specifically, the ultrasound image regularization device may include one or more processing core processors 1201, one or more storage media memories 1202, a power supply 1203, an input unit 1204 and other components. Those skilled in the art will understand that Fig.12 The structure of the ultrasound image regularization device shown in the figure does not constitute a limitation on the X-ray ultrasound image regularization device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0164] The processor 1201 is the control center of the ultrasonic image regularization device. It uses various interfaces and lines to connect various parts of the entire ultrasonic image regularization device. By running or executing software programs and / or modules stored in the memory 1202, and calling data stored in the memory 1202, it performs various functions of the ultrasonic image regularization device and processes data, thereby monitoring the ultrasonic image regularization device as a whole. Optionally, the processor 1201 may include one or more processing cores; preferably, the processor 1201 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1201.

[0165] The memory 1202 can be used to store software programs and modules. The processor 1201 executes various functional applications and data processing by running the software programs and modules stored in the memory 1202. The memory 1202 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the ultrasound image regularization device, etc. In addition, the memory 1202 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 1202 may also include a memory controller to provide the processor 1201 with access to the memory 1202.

[0166] The ultrasonic image regularization device also includes a power supply 1203 for supplying power to various components. Preferably, the power supply 1203 can be logically connected to the processor 1201 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 1203 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0167] The ultrasound image regularization device may also include an input unit 1204, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0168] Although not shown, the ultrasound image regularization device may also include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 1201 in the ultrasound image regularization device will load the executable files corresponding to the processes of one or more application programs into the memory 1202 according to the following instructions, and the processor 1201 will run the application programs stored in the memory 1202, thereby implementing the steps in the ultrasound image regularization method provided in any embodiment of the present application.

[0169] To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. The computer-readable storage medium stores an ultrasound image regularization program, and when the ultrasound image regularization program is executed by a processor, the steps of the ultrasound image regularization method provided in any embodiment of the present application are implemented.

[0170] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.

[0171] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.

[0172] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0173] The above is a detailed introduction to an ultrasonic image regularization method provided in an embodiment of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for normalizing ultrasonic images, characterized in that: include: Acquire ultrasound images to be organized; the ultrasound images include at least two initial ultrasound images; Processing the video frames in the ultrasound image to obtain type information corresponding to each of the video frames; the type information includes at least one of location information and lesion information; Arrange each of the video frames according to the type information corresponding to each of the video frames to obtain a target ultrasound image corresponding to each type; The step of arranging the video frames according to the type information corresponding to each video frame to obtain target ultrasound images corresponding to each type includes: Extracting lesion video frames containing lesions from each of the initial ultrasound images according to the lesion information corresponding to each of the video frames, and combining video frames corresponding to the same lesion according to the matching results between the lesion video frames to obtain target ultrasound images corresponding to each lesion; The step of combining video frames corresponding to the same lesion according to the matching results between the lesion video frames to obtain target ultrasound images corresponding to the lesions includes: According to the part information corresponding to each of the video frames, image stitching is performed on each of the video frames to reconstruct the part image corresponding to each of the part information; According to the position information of each lesion video frame in the part image, the lesion video frames are matched to obtain the target ultrasound image corresponding to each lesion.

2. The ultrasound image normalization method according to claim 1, characterized in that: The step of arranging the video frames according to the type information corresponding to each video frame to obtain target ultrasound images corresponding to each type includes: The ultrasonic image is divided according to the part information corresponding to each of the video frames to obtain a plurality of sub-ultrasonic images, and the sub-ultrasonic images corresponding to the same part are combined to obtain a target ultrasonic image corresponding to each part.

3. The ultrasound image normalization method according to claim 1, characterized in that: The step of combining video frames corresponding to the same lesion according to the matching results between the lesion video frames to obtain target ultrasound images corresponding to the lesions includes: Extracting image features of each of the lesion video frames, and calculating a transformation matrix between each of the lesion video frames according to the image features of each of the lesion video frames; Extracting corresponding physiological reference points from the lesion video frames respectively according to the transformation matrix between the lesion video frames; Each of the lesion video frames is matched according to the relative position relationship between the lesion and the physiological reference point in each of the lesion video frames to obtain a target ultrasound image corresponding to each lesion.

4. The ultrasound image normalization method according to claim 1, characterized in that: The processing of the video frames in the ultrasound image to obtain type information corresponding to each of the video frames includes: Dividing the video frames in the ultrasound image according to a preset number of frames to obtain video frame groups; Processing the video frame group according to a preset part recognition model to obtain part information corresponding to each video frame in the video frame group; and / or The video frame group is processed according to a preset lesion recognition model to obtain lesion information corresponding to each video frame in the video frame group.

5. The ultrasound image normalization method according to claim 4, characterized in that: Before dividing the video frames in the ultrasound image according to the preset number of frames to obtain the video frame groups, the method further includes: Determining a quality monitoring result corresponding to the ultrasound image according to a scanning operation indicator corresponding to each video frame in the ultrasound image; A preset database is queried to obtain the number of associated frames corresponding to the quality monitoring result, and the number of associated frames is used as the preset number of frames.

6. The ultrasound image normalization method according to claim 1, characterized in that: After the video frames are sorted according to the type information corresponding to each of the video frames to obtain the target ultrasound images corresponding to each type, the method further includes: The target ultrasound images corresponding to each type are respectively associated and stored with each type, and a query interface is generated according to the type; A query request triggered based on the query interface is received, and according to the type selected on the query interface, a target ultrasound image corresponding to the type is output.

7. The ultrasound image normalization method according to claim 6, characterized in that: The outputting of the target ultrasound image corresponding to the type includes: generating a playback interface according to the sub-ultrasound image contained in the target ultrasound image; A play request triggered based on the play interface is received, and according to the target identifier selected on the play interface, a target sub-ultrasound image corresponding to the target identifier is output.

8. The ultrasound image normalization method according to claim 1, characterized in that: After processing the video frames in the ultrasound image to obtain type information corresponding to each of the video frames, the method further includes: According to the part information in the type information corresponding to each of the video frames, the lesion information in the type information corresponding to each of the video frames is matched, and the association relationship between the part information and the lesion information is determined and output.

9. The ultrasound image normalization method according to any one of claims 1 to 8, characterized in that: The step of obtaining the ultrasound image to be sorted includes: Scanning the target area according to different preset angles or preset scanning intervals to obtain at least two initial ultrasound images corresponding to the target area; The at least two initial ultrasound images are used as the ultrasound images to be normalized.

10. An ultrasonic image integration device, characterized in that: include: An acquisition module, used for acquiring ultrasound images to be organized; the ultrasound images include at least two initial ultrasound images; An identification module, used to process the video frames in the ultrasound image to obtain type information corresponding to each of the video frames; the type information includes at least one of location information and lesion information; A sorting module, used for sorting each of the video frames according to the type information corresponding to each of the video frames to obtain a target ultrasound image corresponding to each type; The step of arranging the video frames according to the type information corresponding to each video frame to obtain target ultrasound images corresponding to each type includes: Extracting lesion video frames containing lesions from each of the initial ultrasound images according to the lesion information corresponding to each of the video frames, and combining video frames corresponding to the same lesion according to the matching results between the lesion video frames to obtain target ultrasound images corresponding to each lesion; The step of combining video frames corresponding to the same lesion according to the matching results between the lesion video frames to obtain target ultrasound images corresponding to the lesions includes: According to the part information corresponding to each of the video frames, image stitching is performed on each of the video frames to reconstruct the part image corresponding to each of the part information; According to the position information of each lesion video frame in the part image, the lesion video frames are matched to obtain the target ultrasound image corresponding to each lesion.

11. An ultrasonic image integration device, characterized in that: The ultrasound image normalization device includes a processor, a memory, and an ultrasound image normalization program stored in the memory and executable on the processor. The processor executes the ultrasound image normalization program to implement the steps in the ultrasound image normalization method described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an ultrasound image regularization program, and the ultrasound image regularization program is executed by a processor to implement the steps in the ultrasound image regularization method according to any one of claims 1 to 9.

Images