Methods, apparatus, computing devices, and storage media for assisting with ultrasound scanning

Abstract

A method, apparatus, computing device, and storage medium for assisting ultrasound scanning are provided. The method may include: obtaining a sequence of scanned images comprising one or more images obtained by ultrasound scans; performing anomaly detection on the sequence of scanned images; in response to determining that the result of the anomaly detection indicates an anomaly in a human body part targeted by the sequence of scanned images, obtaining image quality of an image region associated with the anomaly; and displaying an image representation of the anomaly, the image representation of the anomaly having display parameters indicating the image quality.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent healthcare, and in particular to a method, apparatus, computing device, and storage medium for assisting ultrasound scanning. Background Technology

[0002] Currently, doctors frequently use medical image sequences obtained from medical scanning equipment for medical diagnosis. Among these, ultrasound acquisition equipment is one of the most common medical scanning devices. However, because ultrasound acquisition equipment often requires the operator to manually move the probe during scanning, issues such as excessively fast scanning speeds in localized areas and unclear images may occur, and the process is highly dependent on the operator's experience. Therefore, a method to assist in ultrasound scanning is desired.

[0003] The methods described in this section are not necessarily methods that had been previously conceived or adopted. Unless otherwise specified, no method described in this section should be assumed to be prior art simply because it is included in this section. Similarly, unless otherwise specified, the issues mentioned in this section should not be considered to be accepted in any prior art. Summary of the Invention

[0004] According to one aspect of this disclosure, a method for assisting ultrasound scanning is provided, comprising: obtaining a scanned image sequence comprising one or more images obtained by ultrasound scanning; performing anomaly detection on the scanned image sequence; in response to determining that the result of the anomaly detection indicates that an anomaly exists in a human body part targeted by the scanned image sequence, obtaining image quality of an image region associated with the anomaly; and displaying an image representation of the anomaly, the image representation of the anomaly having display parameters indicating the image quality.

[0005] According to another aspect of this disclosure, an apparatus for assisting ultrasound scanning is provided, comprising: a scan image sequence acquisition unit for acquiring a scan image sequence, the scan image sequence comprising one or more images acquired by ultrasound scanning; an anomaly detection unit for performing anomaly detection on the scan image sequence; an image quality acquisition unit for acquiring image quality of an image region associated with the anomaly in response to determining that the result of the anomaly detection indicates that an anomaly exists in a human body part targeted by the scan image sequence; and a display unit for displaying an image representation of the anomaly, the image representation of the anomaly having display parameters indicating the image quality.

[0006] According to another aspect of this disclosure, a computing device is provided, comprising: a memory, a processor, and a computer program stored on the memory, wherein the processor is configured to execute the computer program to implement a method for assisting ultrasound scanning according to one or more embodiments of this disclosure.

[0007] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a method for assisting ultrasound scanning according to one or more embodiments of this disclosure.

[0008] According to another aspect of this disclosure, a computer program product is provided, including a computer program, wherein the computer program, when executed by a processor, implements a method for assisting ultrasound scanning according to one or more embodiments of this disclosure.

[0009] These and other aspects of this disclosure will be apparent from the embodiments described below, and will be elucidated with reference to the embodiments described below. Attached Figure Description

[0010] Further details, features, and advantages of this disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0011] Figure 1 This is a schematic diagram illustrating an example system in which various methods described herein may be implemented according to exemplary embodiments;

[0012] Figure 2 This is a flowchart illustrating a method for assisting ultrasound scanning according to an exemplary embodiment;

[0013] Figures 3A-3D This is a schematic diagram illustrating a scanned image according to an exemplary embodiment;

[0014] Figure 4 This is a schematic block diagram illustrating an apparatus for assisting ultrasound scanning according to an exemplary embodiment;

[0015] Figure 5 This is a block diagram illustrating an exemplary computer device that can be applied to an exemplary embodiment. Detailed Implementation

[0016] In this disclosure, unless otherwise stated, the use of terms such as "first," "second," etc., to describe various elements is not intended to limit the positional, temporal, or importance relationships of these elements; such terms are merely used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of that element, while in other cases, based on the context, they may refer to different instances.

[0017] The terminology used in the description of the various examples described in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context explicitly indicates otherwise, an element may be one or more unless the number of elements is specifically limited. As used herein, the term "multiple" means two or more, and the term "based on" should be interpreted as "at least partially based on". Furthermore, the terms "and / or" and "at least one of..." cover any one of the listed items and all possible combinations thereof.

[0018] Exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0019] Figure 1 This is a schematic diagram illustrating an example system 100 in which various methods described herein may be implemented according to exemplary embodiments.

[0020] refer to Figure 1 The system 100 includes a client device 110, a server 120, and a network 130 that communicatively couples the client device 110 and the server 120.

[0021] Client device 110 includes a display 114 and a client application (APP) 112 that can be displayed on the display 114. Client application 112 can be an application that needs to be downloaded and installed before running, or a lightweight application (liteapp). If client application 112 is an application that needs to be downloaded and installed before running, client application 112 can be pre-installed on client device 110 and activated. If client application 112 is a mini-app, user 102 can directly run client application 112 on client device 110 by searching for client application 112 in the host application (e.g., by the name of client application 112) or by scanning the graphic code of client application 112 (e.g., barcode, QR code, etc.), without installing client application 112. In some embodiments, client device 110 can be any type of mobile computing device, including mobile computers, mobile phones, wearable computing devices (e.g., smartwatches, head-mounted devices including smart glasses, etc.), or other types of mobile devices. In some embodiments, the client device 110 may alternatively be a stationary computer device, such as a desktop computer, server computer, or other type of stationary computer device. In some alternative embodiments, the client device 110 may also be or may include a medical image printing device.

[0022] Server 120 is typically a server deployed by an Internet Service Provider (ISP) or Internet Content Provider (ICP). Server 120 can represent a single server, a cluster of multiple servers, a distributed system, or a cloud server providing basic cloud services such as cloud databases, cloud computing, cloud storage, and cloud communications. It will be understood that, although... Figure 1 The diagram shows that server 120 communicates with only one client device 110, but server 120 can provide background services to multiple client devices simultaneously.

[0023] Examples of network 130 include combinations of local area networks (LANs), wide area networks (WANs), personal area networks (PANs), and / or communication networks such as the Internet. Network 130 can be wired or wireless. In some embodiments, technologies and / or formats including Hypertext Markup Language (HTML), Extensible Markup Language (XML), etc., are used to process data exchanged through network 130. Furthermore, encryption technologies such as Secure Sockets Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), and Internet Protocol Security (IPsec) can be used to encrypt all or some of the links. In some embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0024] System 100 may further include image acquisition device 140. In some embodiments, Figure 1 The image acquisition device 140 shown may be a medical scanning device, including but not limited to scanning or imaging devices used in positron emission tomography (PET), positron emission tomography with computerized tomography (PET / CT), single photon emission computed tomography with computerized tomography (SPECT / CT), computed tomography (CT), medical ultrasonography, nuclear magnetic resonance imaging (NMRI), magnetic resonance imaging (MRI), cardiovascular angiography (CA), digital radiography (DR), etc. For example, image acquisition device 140 may include digital subtraction angiography scanner, magnetic resonance angiography scanner, computed tomography angiography scanner, positron emission tomography scanner, positron emission tomography (PET) scanner, single photon emission computed tomography (SPT) scanner, computed tomography scanner, medical ultrasound examination equipment, magnetic resonance imaging (MRI) scanner, digital radiography scanner, etc. Image acquisition device 140 may be connected to a server (e.g., Figure 1 The system connects to server 120 (or a separate server of the imaging system, not shown in the figure) to process image data, including but not limited to converting scan data (e.g., converting it into a medical image sequence), compressing it, correcting pixels, and reconstructing it in three dimensions.

[0025] Image acquisition device 140 may be connected to client device 110, for example, via network 130, or otherwise directly connected to client device to communicate with client device.

[0026] Optionally, the system may also include an intelligent computing device or a computing card 150. The image acquisition device 140 may include or be connected (e.g., detachably connected) to such a computing card 150. As an example, the computing card 150 can perform image data processing, including but not limited to conversion, compression, pixel correction, reconstruction, etc. As another example, the computing card 150 can implement a method for assisting ultrasound scanning according to embodiments of the present disclosure.

[0027] The system may also include other components not shown, such as a data storage unit. The data storage unit may be a database, data repository, or other form of device for data storage; it may be a conventional database, or it may include a cloud database, a distributed database, etc. For example, direct image data generated by the image acquisition device 140, or medical image sequences or three-dimensional image data obtained through image processing, may be stored in the data storage unit for subsequent retrieval by the server 120 and client device 110. Furthermore, the image acquisition device 140 may also directly provide direct image data or medical image sequences or three-dimensional image data obtained through image processing to the server 120 or client device 110, etc.

[0028] Users can use client device 110 to view acquired images or videos, including preliminary image data or images after analysis and processing, view analysis results, interact with the acquired images or analysis results, and input acquisition commands, configuration data, etc. Client device 110 can send configuration data, commands, or other information to image acquisition device 140 to control the acquisition and data processing of the image acquisition device. In some embodiments, the client device can be physically, communicatively, or otherwise connected to the acquisition device, such as an ultrasound acquisition device, enabling the client device to receive, process, and / or display images acquired by the acquisition device. In some embodiments, the client device may include the acquisition device, or the client device and the acquisition device may be integrated in some way, and this disclosure is not limited thereto.

[0029] For the purposes of this disclosure's embodiments, Figure 1In the example, client application 112 can be an image sequence management application that provides various functions, such as storage management, indexing, sorting, and classification of acquired image sequences. Correspondingly, server 120 can be a server used in conjunction with the image sequence management application. Server 120 can provide image sequence management services to client application 112 running on client device 110 based on user requests or instructions generated according to embodiments of this disclosure. For example, it can manage image sequence storage in the cloud, store and classify image sequences according to specified indexes (including, but not limited to, sequence type, patient identifier, body part, acquisition target, acquisition stage, acquisition machine, presence of lesions, severity, etc.), and retrieve and provide image sequences to client devices according to specified indexes, etc. Alternatively, server 120 can also provide or allocate such service capabilities or storage space to client device 110, whereby client application 112 running on client device 110 provides corresponding image sequence management services based on user requests or instructions generated according to embodiments of this disclosure, etc. It is understood that the above is only one example, and this disclosure is not limited thereto.

[0030] Figure 2 This is a flowchart illustrating a method 200 for assisting ultrasound scanning according to an exemplary embodiment. Method 200 can be performed on a client device (e.g., Figure 1 The execution is performed at the client device 110 shown, that is, the execution entity of each step of method 200 can be... Figure 1 The client device 110 shown. In some embodiments, method 200 can be performed on a server (e.g., Figure 1 The method 200 is executed at server 120 (as shown in the figure). In some embodiments, the method 200 may be executed in combination by a client device (e.g., client device 110) and a server (e.g., server 120).

[0031] In the following text, taking the client device 110 as the execution subject as an example, the steps of method 200 are described in detail.

[0032] refer to Figure 2 In step 210, a scan image sequence is obtained, the scan image sequence comprising one or more images obtained by ultrasound scan.

[0033] In step 220, anomaly detection is performed on the scanned image sequence.

[0034] In step 230, in response to determining that the result of the anomaly detection indicates that there is an anomaly in the human body part targeted by the scanned image sequence, the image quality of the image region associated with the anomaly is obtained.

[0035] In step 240, an image representation of the anomaly is displayed, the image representation of the anomaly having display parameters indicating the image quality.

[0036] During ultrasound scanning, the operator often moves the probe by hand, leading to problems such as uneven scanning speed, incomplete scanning of certain areas, blurriness, and poor image quality. Since the key to ultrasound scanning lies in acquiring images of abnormal areas, the image quality of these abnormal areas is particularly important. According to embodiments of this disclosure, by identifying abnormalities and displaying an image representation of the abnormalities using display parameters indicating image quality, it is possible to intuitively show whether the current abnormal area has been effectively and / or completely scanned, thereby achieving the effect of intelligent assisted scanning.

[0037] It is understood that the sequence of scanned images can be images acquired in a single ultrasound scan targeting a region of the human body. In some embodiments, at least one scanned image is acquired via a rapid ultrasound scan or "fast ultrasound scan" operation. Furthermore, it is understood that the sequence of scanned images can also be images in other forms than ultrasound. For example, at least one scanned image can be an image acquired via a scanning probe—as long as such a scanning probe can be held by the scanner or otherwise moved according to the operator's manipulation, the operator can benefit from such an auxiliary display.

[0038] At step 220, anomaly detection can be performed on the scanned image sequence. It is understood that the obtained anomaly detection results can be, for example, numerical, enumerated, textual, or other types. The obtained anomaly detection results may include anomaly scores, the number or proportion of images with abnormal regions, the type of anomaly (e.g., lesion), the severity level of the anomaly (e.g., area or proportion of the abnormal region, stage of lesion development, degree of disease), etc., and this disclosure is not limited thereto. At step 230, determining that the anomaly detection result indicates an anomaly in the human body part targeted by the scanned image sequence may include detecting an abnormal region in at least one image, for example, determining that the probability of an abnormal region in at least one image is higher than a threshold. For example, determining that the anomaly detection result indicates an anomaly in the human body part targeted by the scanned image sequence may include the probability of an abnormal region in at least one image (or at least a predetermined number of images) being higher than a first threshold, the area of ​​the abnormal region reaching a second threshold, or the severity of the anomaly (e.g., lesion) being higher than a third threshold, etc., and this disclosure is not limited thereto. Furthermore, it is understood that an actual abnormality in the human body (or an abnormal site, such as a lesion, foreign body, blockage, plaque, etc.) can correspond to one or more abnormal regions. For example, a region corresponding to the abnormality may exist in multiple scan images. Furthermore, it is understood that one or more abnormalities may exist, and each of these abnormalities may correspond to one or more scan images, and so on.

[0039] At step 240, an image representation of the anomaly may be displayed. It is understood that the image representation of the anomaly can be a two-dimensional or three-dimensional image, such as an original acquired image or image sequence, a processed image or image sequence, or a further generated three-dimensional model representation. Processing may include basic or enhancement processing such as cropping, rotating, and adjusting brightness of the image, or it may include medical knowledge-based processing such as human organ and / or tissue segmentation and lesion identification, etc., and this disclosure is not limited thereto. The image representation of the anomaly may include marking the anomaly with lines, highlights, or other effects in the original acquired (or processed) scan image sequence, or it may additionally or separately display an image of the anomaly, such as a magnified image of the anomaly, etc.

[0040] According to some embodiments, the graphical representation of the anomaly is obtained by three-dimensional modeling the anomaly region. In other words, the method of this disclosure may include modeling the anomaly region based on acquired images, particularly three-dimensional modeling. Exemplarily, the modeling process may be real-time modeling.

[0041] In some exemplary embodiments, the method may include: firstly, detecting lesions and / or abnormalities based on currently acquired two-dimensional images. Then, constructing a three-dimensional image of the lesion or abnormal region based on multiple two-dimensional images. This step can be implemented, for example, by segmenting the acquired two-dimensional image sequence to obtain two-dimensional lesion images, abnormal region images, abnormal parts, etc., and then constructing a three-dimensional image of the abnormal or lesion region based on the segmented image parts. This process can be a real-time modeling process, for example, real-time modeling can be performed simultaneously during scanning. As subsequent images are received, the image quality of the three-dimensional image can be updated in real time, for example, by updating various visual indicators. By utilizing various identifiers, such as visual identifiers, to provide feedback to the user on the image quality of the lesion or abnormal region (e.g., three-dimensional image quality), deficiencies can be clearly and in real time displayed, further enabling intelligent assistance to the user's scanning and facilitating the acquisition of complete, high-quality images of lesions, abnormalities, and other areas of concern.

[0042] Real-time modeling facilitates quality feedback, allowing users to clearly identify areas with scan defects, thus benefiting subsequent scanning processes. Furthermore, according to some examples of this disclosure, during real-time scanning, modeling can be performed only on lesions or abnormal areas, rather than the entire scanned area. This reduces the demands on scanning equipment and algorithms, and saves time. This is because complete 3D modeling often requires complex solutions, costly peripheral equipment (e.g., detecting the position of ultrasound sensors using external electromagnetic devices; or using larger, wider ultrasound probes, 3D probes, etc.); or imposes very stringent requirements on scanning parameters and the scanner's technique. According to one or more examples of this disclosure, the implementation is simple and achieves the goal of enabling doctors to obtain high-quality images.

[0043] In some examples, after the scan is completed, the doctor can be shown a sequence of two-dimensional scan images so that the doctor can select abnormal areas in the sequence to view the two-dimensional images. At the same time, a three-dimensional model of the abnormal area can also be displayed for the doctor to compare and view (or other interactive options such as buttons can be displayed so that the three-dimensional model of the abnormal area can be displayed to the user when the interactive option is selected).

[0044] According to one or more embodiments of this disclosure, real-time and convenient scanning assistance can be achieved, which is particularly beneficial for applications that integrate scanning and diagnosis, where the scanning user (e.g., a doctor) is also a diagnosing doctor, and can diagnose while scanning. In such a scenario, real-time display of an image representation (e.g., a three-dimensional representation) of abnormal areas is particularly convenient and advantageous, and eliminates the complex requirements for external devices.

[0045] According to some embodiments, method 200 may further include, after displaying a graphical representation of the anomaly: obtaining one or more additional scan images; and updating the graphical representation of the anomaly based on the one or more additional scan images.

[0046] According to this embodiment, image quality can be displayed in real time and updated at any time.

[0047] It is understood that at least one additional scan image may be obtained after the at least one scan image, targeting substantially the same human body region as the at least one scan image, for example, a subsequent image obtained in the same scanning operation. Here, "same scanning operation" can mean a scanning operation targeting the same patient and the same scanning target within a relatively close time period, such as a scanning operation performed by the same operator holding the same scanning instrument, which may include unidirectional movement and repeated back-and-forth movement, etc., and this disclosure is not limited thereto. For example, different operators (e.g., doctors of different levels, interns, etc.) may hold ultrasound probes to perform scanning operations targeting the same site, but since such operations target substantially the same site and, for example, may generate a merged image sequence, it can still be considered "same scanning operation" or the same scanning operation. It is understood that this disclosure is not limited thereto.

[0048] According to some embodiments, obtaining the image quality of the image region associated with the anomaly may include obtaining a first image quality of a first sub-region and a second image quality of a second sub-region of the image region associated with the anomaly, wherein the first image quality is different from the second image quality, and wherein displaying an image representation of the anomaly includes displaying the first sub-region and the second sub-region with different display parameters.

[0049] According to this embodiment, different image quality sub-regions of an abnormal area can be displayed with different display parameters. For example, for an identified lesion, part of the lesion may have been well captured, while another part may not be clearly displayed according to the currently captured image, the currently captured image may not cover a specific angle of the lesion, or the image captured for that part may have blurry image quality due to equipment limitations or movement during the acquisition process. By identifying at least two sub-regions of the image area associated with the abnormality, and displaying these at least two sub-regions with different image qualities using different display parameters (each corresponding to a different image quality), the part of the lesion that needs to be filled in can be intuitively displayed, facilitating operator positioning and scanning.

[0050] For example, sub-regions can be divided based on at least one quality threshold. For instance, a region with image quality higher than a first quality threshold is divided into a first sub-region and displayed with normal display parameters (e.g., default or default brightness, color, texture, etc.), and an image with image quality lower than the first quality threshold is divided into a second sub-region and displayed with different display parameters, such as different brightness, color, texture, or for example, by outlining the region with lines, etc.

[0051] It is understood that the above are merely examples, and there can be more than one quality threshold and more than two image quality levels, and sub-regions of different quality levels can be displayed with more than two different display parameters. As another example, image quality can be linearly varied rather than hierarchically represented; for example, an image quality score can be determined, and regions of different image quality can be displayed with gradient display parameters (e.g., gradient colors, brightness, mosaic patterns of different granularities, or other masks).

[0052] Furthermore, it is understood that image quality may include, for example, sharpness, resolution, depth, and the ability to distinguish different parts. Alternatively, for example, if the device has 3D reconstruction capabilities or other 3D display capabilities, image quality may include whether there are enough acquisition pixels within a certain spatial range, acquisition density, etc., to achieve the desired 3D reconstruction or 3D display, whether the currently acquired image is sufficient to ensure that the 3D reconstruction or 3D display result has sufficient quality, sharpness, depth, etc., and this disclosure is not limited to these aspects.

[0053] According to some embodiments, the display parameters are selected from the group consisting of: color, brightness, and texture. This allows for the visual identification of locations with poor quality.

[0054] It is understood that such display parameters are merely examples, and display parameters may also include other marks, symbols, markers, masks, etc. For example, areas with poor quality can be outlined with lines, while areas with satisfactory quality are left unmarked, and so on. Furthermore, other examples of display parameters are given throughout this document, and those skilled in the art will understand that this disclosure is not limited thereto.

[0055] According to some embodiments, obtaining the image quality of the image region associated with the anomaly includes: in response to determining that the result of the anomaly detection indicates that there is an anomaly in the human body part targeted by the scanned image sequence, performing image quality detection on the image in the scanned image sequence associated with the anomaly.

[0056] According to such an embodiment, image quality detection can be performed only in response to the presence of abnormal regions, thereby saving computational resources, improving computational speed, and achieving more real-time rendering. Exemplarily, image quality detection can be performed only on the image region where the detected abnormality is located, or on the abnormal image region and its adjacent regions. As a specific non-limiting example, if an abnormality is detected in images numbered 2 and 3 in an image sequence, quality detection can be performed on images 2 and 3, or on images 1-4 in the image sequence, which include adjacent images. Alternatively, quality detection can be performed only on the abnormal regions (and their surrounding pixels) in images 2 and 3, without needing to perform quality detection on other regions in images 2 and 3. As yet another specific non-limiting example, different strategies can be selected based on user preferences, the number of images, image size, device computing power, etc., to achieve a balance between image quality and computational speed, and this disclosure is not limited thereto.

[0057] In other embodiments, image quality detection may be performed first, followed by lesion identification, or image quality detection and lesion identification may be performed in parallel, and this disclosure is not limited thereto.

[0058] According to some embodiments, anomaly detection may include: detecting lesions in each image of the scanned image sequence with a first confidence level; in response to determining that a suspected lesion region exists in a first image of the scanned image sequence based on the first confidence level, confirming the suspected lesion region with a second confidence level different from the first confidence level; and obtaining the anomaly index based on the confirmation result. According to such embodiments, secondary lesion confirmation based on different confidence levels can be performed, thereby obtaining a more accurate identification result. It is understood that the first confidence level can be lower than the second confidence level. For example, identifying lesions with a lower confidence level requires less time, thus being more advantageous for screening multiple images; for the identified suspected lesion region, secondary confirmation can be performed with a higher confidence level. Anomaly indicators, such as the finally confirmed lesion region, the image number where the lesion is located, the lesion type, grade, severity, etc., can be determined based on the results of the secondary confirmation, thereby obtaining a more accurate identification result and saving time.

[0059] In some other embodiments, only one confirmation may be performed, including, for example, confirming with only a lower confidence level in the case of rate priority, or confirming with a higher confidence level in the case of quality priority or a slower scan rate, and this disclosure is not limited thereto. Exemplarily, whether to perform secondary confirmation and the confidence level used for each confirmation can be selected based on user preferences or user settings. As another example, the confidence level can be determined based on the current scan rate; for example, anomaly detection and / or confirmation with a higher confidence level when the current operator is detected moving the ultrasound probe at a slower speed, and / or when the scan frequency is set to a lower frequency, and anomaly detection and / or confirmation with a lower confidence level when the current operator is detected moving the ultrasound probe at a faster speed, and / or when the scan frequency is set to a higher frequency, or detection with only low confidence and no secondary confirmation, etc.

[0060] It is understood that the anomaly identification and detection steps according to one or more embodiments of this disclosure can be performed using various existing or post-dated anomaly identification algorithms, and this disclosure is not limited thereto. As an example, identification can be performed using a pre-trained neural network model. During training, images with and without lesions can be used as positive and negative samples, respectively, to enable the model to identify anomalies (and, exemplarily, the type, area, stage, severity, etc. of the lesion). In other examples, images without labeled positive and negative samples can also be used as samples for self-supervised learning training, and this disclosure is not limited thereto.

[0061] According to some embodiments, method 200 may further include generating a three-dimensional model of the anomaly based on the scanned image sequence, wherein displaying an image representation of the anomaly includes displaying the three-dimensional model of the anomaly.

[0062] In such an embodiment, poor-quality or missing parts of the image can be displayed in a three-dimensional area, thereby facilitating further rescanning and completion by the operator.

[0063] For example, a complete 3D model of the region corresponding to the scanned image sequence can be generated, and the abnormal parts can be displayed within it using highlights, different colors, outlines with boxes or lines, or other methods. Alternatively, a 3D model of the abnormal part can be generated and displayed separately. As another example, a complete graphical representation of the region corresponding to the scanned image sequence, such as a 2D or 3D graphical representation, can be displayed, and a 2D or 3D graphical representation of the abnormal region, such as a magnified representation, can be displayed alongside (e.g., side-by-side), so that the operator can simultaneously view the overall image sequence and the scanning progress of one or more abnormalities.

[0064] According to some embodiments, determining that the result of the anomaly detection indicates the presence of an anomaly in the human body part targeted by the scanned image sequence may include determining the presence of at least two anomalies, and wherein the method further includes: determining the corresponding importance levels of the at least two anomalies; and displaying the corresponding graphical representations of the at least two anomalies according to the importance levels.

[0065] According to this embodiment, abnormalities can be displayed sequentially based on their severity level, enabling a more user-friendly interaction and prompting the operator to focus on the most important abnormalities in order to obtain scan images that are more conducive to subsequent diagnosis.

[0066] In some embodiments, the importance level or indicator of the abnormality can be determined based on the size, type, and confidence level of the identified lesions. Exemplarily, the importance level or indicator of the abnormality can also be determined based on the extent of the lesion region or the number of images in which the lesion can be identified. The level of the abnormality may also include the stage of lesion development, severity, etc.

[0067] According to some embodiments, method 200 may further include displaying an identifier of the scanning position corresponding to the current scanning probe in association with the graphical representation of the anomaly.

[0068] It can locate the current scanning probe relative to the abnormal position, making it easier for the operator to locate the current probe and more easily fill in poor quality or missing positions.

[0069] On the one hand, for the algorithm, because the lesion is smaller in scale, the difficulty of locating the lesion locally is lower than the difficulty of locating it within the entire organ, which will lead to smaller errors; on the other hand, this also helps the operator to see which angle and which area of ​​the lesion has not yet been clearly scanned, thus making it easier to adjust the scanning probe to obtain an image at the desired position / angle.

[0070] Although the various operations are depicted in the accompanying drawings in a specific order, this should not be construed as requiring that these operations be performed in the specific order shown or in chronological order, nor should it be construed as requiring that all the operations shown be performed to obtain the desired result. For example, as described above, in some embodiments, the step of detecting image quality may occur before, after, or in parallel with the step of identifying anomalies, and this disclosure is not limited thereto.

[0071] It is understood that throughout this disclosure, image sequences may be or may include two-dimensional image data, or may be or may include three-dimensional image data. Image sequences may be image data that is directly acquired and stored or otherwise transmitted to a terminal device for user use. Image sequences may also be processed image data after various image processing steps. Image sequences may also undergo other analytical processes (e.g., analysis for the presence of lesion features or lesions) and include analytical results (e.g., delineation of regions of interest, tissue segmentation results, etc.). It is understood that this disclosure is not limited thereto.

[0072] According to one or more embodiments of this disclosure, images that are more conducive to subsequent diagnosis can be obtained from the perspective of doctors, examiners, analysts, or other operators. For example, lesions can be ranked, and the abnormal areas can be presented to the doctor based on the ranking results. A three-dimensional model of the lesions can be presented to the doctor, showing both those that meet and do not meet the quality requirements, and updating in real time. Exemplarily, the area corresponding to the current section can be displayed to facilitate operator localization.

[0073] According to one or more embodiments of this disclosure, real-time modeling and updating are possible.

[0074] According to one or more embodiments of this disclosure, quality can be displayed in a user-friendly and lesion-oriented manner. According to some exemplary embodiments, the current scan location can also be displayed for more convenient interactive location of the scan, facilitating subsequent scans by the operator.

[0075] According to one or more embodiments of this disclosure, image quality can be judged and / or displayed at the lesion level based on abnormal detection algorithms such as lesion detection, thereby achieving an intelligent auxiliary effect of "scanning wherever it is missing".

[0076] The following is combined Figures 3A-3D A schematic diagram illustrating a scanned image according to an exemplary embodiment of the present disclosure. References Figure 3A The diagram illustrates a schematic image 310 of the body region to be scanned, and shows a schematic cross-section 311 of an ultrasound scan. It is understood that the number, angles, and positions of the cross-sections shown herein are merely examples. Furthermore, it is understood that the schematic image 310 may be a two-dimensional image representation, a three-dimensional image representation, or it may not be displayed to the user. In other examples, the cross-section 311 may be interactive, for example, zooming in on the corresponding image upon receiving a user selection, etc., and this disclosure is not limited thereto.

[0077] refer to Figure 3B An exemplary cross-sectional image 320 is shown, for example, corresponding to... Figure 3AThe cross-sectional image of section 311 in the image. For example, section image 320 can be a frame image acquired by the ultrasound probe at a certain angle and position. Specifically, Figure 3B The image illustrates two exemplary abnormal regions 321 and 322, which may be, for example, lesions, plaques, or other abnormalities. Exemplarily, different image qualities can be displayed in image 320 with different display parameters. Additionally or alternatively, image quality can be displayed in other images, as will be referred to below. Figure 3C As described.

[0078] Figure 3C Examples of some exemplary embodiments of the present disclosure are shown. Figure 3B The magnified image 330 shows the abnormal region 321 in the image. It is understandable that, although... Figure 3B The image is shown in two-dimensional form, but image 330 can be a three-dimensional image. In other words, image 330 can be a magnified portion of the original acquired image, or it can be a view of the generated three-dimensional model from a certain rotational perspective. Figure 3B As shown, the first sub-region 331 and the second sub-region 332 of the identified anomalies can be displayed using different display parameters to correspond to different qualities. For example, sub-region 332 can be displayed with a lighter color or highlighted to indicate that the area has defects and requires further scanning. Furthermore, it is understood that although... Figure 3C The image shows two sub-regions with different image qualities, but this disclosure is not limited to this. It may include more image quality levels, or it may not distinguish between sub-regions and display different image qualities with gradually varying parameters. Alternatively, after the corresponding abnormality or lesion has been fully scanned, the entire abnormal area may be displayed with the same display parameters, and a message "Current lesion has been fully scanned" may also be displayed.

[0079] Furthermore, it is understandable that, such as Figure 3C The image quality shown can change in real time or near real time. For example, after receiving subsequent scanned images, the image quality display can be updated. For instance, as new images supplement some of the image quality deficiencies, the reduced sub-region 332 and the increased sub-region 331 can be displayed to indicate that the area that needs to be supplemented is gradually decreasing. Alternatively, as the image quality corresponding to sub-region 332 reaches the standard, the image sub-region 332 can no longer be displayed, etc.

[0080] refer to Figure 3DThe illustration shows a schematic image 340, which displays an identifier indicating the scanning position corresponding to the current scanning probe according to some exemplary embodiments of the present disclosure. An exemplary lesion cross-section 341 is shown, along with an identifier 342 indicating the current probe position and angle. For simplicity, only an exemplary lesion cross-section 341 is shown here comprising a sub-region; however, it is understood that the present disclosure is not limited thereto. It is understood that the features described in conjunction with image 340 can be combined with images 320, 330, etc., or image 340 can be displayed in place of image 330. Furthermore, it is understood that the identifier 342 indicating the current probe position and angle can take various forms, such as arrows, boxes, or enclosing lines. Although a generally elongated probe cross-section region is shown as an example with the current probe angle substantially perpendicular to the angle of the currently viewed image, the cross-section currently scanned by the probe in the displayed image can have a circular, fan-shaped, elliptical, or various other shapes and coverage areas depending on the current probe angle and the angle of the currently viewed image, and the present disclosure is not limited thereto. Exemplarily, the identifier corresponding to the scanning position of the scanning probe may also include the current probe angle, direction of movement, etc. (not shown).

[0081] It is understandable that the above combination Figures 3A-3D One or more of the described images 310-340 may be displayed to the user together, or only a portion or one of them may be displayed, and this disclosure is not limited thereto. For example, only image 330 or 340 may be displayed during the scanning process, and the entire image 310 or 320 may be displayed after the scanning is substantially completed, and so on. Furthermore, it is understood that in the presence of multiple abnormalities or lesions, multiple images of the abnormalities or lesions corresponding to image 330 or 340 may be displayed side by side, in a priority order, or otherwise, so that each identified abnormality can obtain a composite desired image quality.

[0082] Figure 4This is a schematic block diagram illustrating an apparatus 400 for assisting ultrasound scanning according to an exemplary embodiment. The apparatus 400 for assisting ultrasound scanning may include a scan image sequence acquisition unit 410, an anomaly detection unit 420, an image quality acquisition unit 430, and a display unit 440. The scan image sequence acquisition unit 410 is configured to acquire a scan image sequence comprising one or more images acquired by ultrasound scanning. The anomaly detection unit 420 is configured to perform anomaly detection on the scan image sequence. The image quality acquisition unit 430 is configured to acquire the image quality of an image region associated with the anomaly in response to determining that the result of the anomaly detection indicates an anomaly in a human body part targeted by the scan image sequence. The display unit 440 is configured to display an image representation of the anomaly, the image representation of the anomaly having display parameters indicating the image quality.

[0083] It should be understood that Figure 4 The various modules of the device 400 shown can be connected to the reference. Figure 2 The steps in method 200 described correspond to each other. Therefore, the operations, features, and advantages described above for method 200 also apply to apparatus 400 and its included modules. For the sake of brevity, some operations, features, and advantages will not be repeated here.

[0084] According to embodiments of the present disclosure, a computing device is also disclosed, including a memory, a processor, and a computer program stored on the memory, wherein the processor is configured to execute the computer program to implement the steps of the method for assisting ultrasound scanning according to embodiments of the present disclosure and variations thereof.

[0085] According to embodiments of the present disclosure, a non-transitory computer-readable storage medium is also disclosed, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the method for assisting ultrasound scanning according to embodiments of the present disclosure and variations thereof.

[0086] According to embodiments of the present disclosure, a computer program product is also disclosed, including a computer program, wherein when executed by a processor, the computer program implements the steps of the method for assisting ultrasound scanning according to embodiments of the present disclosure and variations thereof.

[0087] While specific functions have been discussed above with reference to specific modules, it should be noted that the functions of the modules discussed herein can be divided into multiple modules, and / or at least some functions of multiple modules can be combined into a single module. The specific module performing an action discussed herein includes the specific module itself performing the action, or alternatively, the specific module calling or otherwise accessing another component or module that performs the action (or performs the action in conjunction with the specific module). Therefore, a specific module performing an action can include the specific module performing the action itself and / or another module that the specific module calls or otherwise accesses to perform the action. As used herein, the phrase "Entity A initiates action B" can mean that entity A issues an instruction to perform action B, but entity A itself does not necessarily perform action B. For example, the phrase "Display unit 440 can be used to display a graphical representation of the anomaly" can mean that display unit 440 instructs a display (not shown) to present first navigation information, while display unit 440 itself does not need to perform the action of displaying or presenting.

[0088] It should also be understood that this article can describe various technologies in the general context of software and hardware components or program modules. The above regarding... Figure 4 The various modules described can be implemented in hardware or in hardware in combination with software and / or firmware. For example, these modules can be implemented as computer program code / instructions configured to execute in one or more processors and stored in a computer-readable storage medium. Alternatively, these modules can be implemented as hardware logic / circuit. For example, in some embodiments, one or more of the modules or units according to embodiments of this disclosure can be implemented together in a System on Chip (SoC). The SoC may include an integrated circuit chip (which includes a processor (e.g., a Central Processing Unit (CPU), microcontroller, microprocessor, digital signal processor (DSP), etc.), memory, one or more communication interfaces, and / or one or more other components of circuitry), and may optionally execute received program code and / or include embedded firmware to perform functions.

[0089] According to one aspect of this disclosure, a computing device is provided, including a memory, a processor, and a computer program stored in the memory. The processor is configured to execute the computer program to implement the steps of any of the method embodiments described above.

[0090] According to one aspect of this disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the method embodiments described above.

[0091] According to one aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the steps of any of the method embodiments described above.

[0092] In the following text, combined with Figure 5 Illustrative examples describing such computer devices, non-transitory computer-readable storage media, and computer program products.

[0093] Figure 5 An example configuration of a computer device 500 that can be used to implement the methods described herein is shown. For example, Figure 1 The server 120 and / or client device 110 shown may include an architecture similar to computer device 500. The aforementioned devices / apparatus for assisting ultrasound scanning may also be implemented wholly or at least partially by computer device 500 or similar devices or systems.

[0094] Computer device 500 can be a variety of different types of devices, such as a service provider's server, a device associated with a client (e.g., a client device), a system-on-a-chip, and / or any other suitable computer device or computing system. Examples of computer device 500 include, but are not limited to: desktop computers, server computers, laptop or netbook computers, mobile devices (e.g., tablets, cellular or other wireless phones (e.g., smartphones), notebook computers, mobile stations), wearable devices (e.g., glasses, watches), entertainment devices (e.g., entertainment appliances, set-top boxes communicatively coupled to a display device, game consoles), televisions or other display devices, automotive computers, and so on. Therefore, the range of computer device 500 can be from full-resource devices with large amounts of memory and processor resources (e.g., personal computers, game consoles) to low-resource devices with limited memory and / or processing resources (e.g., traditional set-top boxes, handheld game consoles).

[0095] Computer device 500 may include at least one processor 502, memory 504, multiple communication interfaces 506, display device 508, other input / output (I / O) devices 510, and one or more mass storage devices 512 capable of communicating with each other, such as via system bus 514 or other suitable connections.

[0096] Processor 502 may be a single processing unit or multiple processing units, and all processing units may include single or multiple computing units or multiple cores. Processor 502 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that manipulates signals based on operating instructions. Among other capabilities, processor 502 may be configured to acquire and execute computer-readable instructions stored in memory 504, mass storage device 512, or other computer-readable media, such as program code of operating system 516, program code of application program 518, program code of other program 520, etc.

[0097] Memory 504 and mass storage device 512 are examples of computer-readable storage media for storing instructions executed by processor 502 to perform the various functions described above. For example, memory 504 may generally include both volatile and non-volatile memory (e.g., RAM, ROM, etc.). Furthermore, mass storage device 512 may generally include hard disk drives, solid-state drives, removable media, including external and removable drives, memory cards, flash memory, floppy disks, optical disks (e.g., CDs, DVDs), storage arrays, network-attached storage, storage area networks, etc. Both memory 504 and mass storage device 512 may be collectively referred to herein as memory or computer-readable storage media, and may be non-transitory media capable of storing computer-readable, processor-executable program instructions as computer program code, which may be executed by processor 502 as a specific machine configured to perform the operations and functions described in the examples herein.

[0098] Multiple program modules may be stored on mass storage device 512. These programs include operating system 516, one or more application programs 518, other programs 520, and program data 522, and they may be loaded into memory 504 for execution. Examples of such application programs or program modules may include, for example, computer program logic (e.g., computer program code or instructions) for implementing components / functions such as method 200 (including any suitable steps of method 200), and / or other embodiments described herein.

[0099] Although Figure 5 The modules 516, 518, 520, and 522, or portions thereof, are illustrated as being stored in memory 504 of computer device 500; however, modules 516, 518, 520, and 522 may be implemented using any form of computer-readable medium accessible by computer device 500. As used herein, “computer-readable medium” includes at least two types of computer-readable media: computer storage media and communication media.

[0100] Computer storage media includes volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, DVD, or other optical storage devices, magnetic cassettes, magnetic tapes, disk storage devices or other magnetic storage devices, or any other non-transfer medium that can be used to store information for access by computer equipment.

[0101] In contrast, communication media can embody computer-readable instructions, data structures, program modules, or other data within modulated data signals such as carrier waves or other transmission mechanisms. Computer storage media as defined herein do not include communication media.

[0102] Computer device 500 may also include one or more communication interfaces 506 for exchanging data with other devices, such as via a network, direct connection, etc., as discussed above. Such communication interfaces can be one or more of the following: any type of network interface (e.g., a network interface card (NIC)), wired or wireless (such as IEEE 802.11 Wireless LAN (WLAN)) wireless interface, Wi-MAX interface, Ethernet interface, Universal Serial Bus (USB) interface, cellular network interface, Bluetooth. TM Interfaces include near-field communication (NFC) interfaces. Communication interface 506 can facilitate communication across various network and protocol types, including wired networks (e.g., LAN, cable, etc.) and wireless networks (e.g., WLAN, cellular, satellite, etc.), the Internet, etc. Communication interface 506 can also provide communication with external storage devices (not shown) such as storage arrays, network-attached storage, storage area networks, etc.

[0103] In some examples, a display device 508, such as a monitor, may be included for displaying information and images to the user. Other I / O devices 510 may be devices that receive various inputs from the user and provide various outputs to the user, and may include touch input devices, gesture input devices, cameras, keyboards, remote controls, mice, printers, audio input / output devices, and so on.

[0104] Although this disclosure has been described and illustrated in detail in the accompanying drawings and the foregoing description, such description and illustration should be considered illustrative and suggestive, not restrictive; this disclosure is not limited to the disclosed embodiments. By studying the drawings, the disclosure, and the appended claims, those skilled in the art will be able to understand and implement variations of the disclosed embodiments in practicing the claimed subject matter. In the claims, the word "comprising" does not exclude other elements or steps not listed, and the words "a" or "an" do not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be beneficial.

Claims

1. A device for assisting ultrasound scanning, comprising: A scanned image sequence acquisition unit is used to acquire a scanned image sequence, the scanned image sequence comprising one or more images acquired by ultrasound scans; An anomaly detection unit is used to perform anomaly detection on the scanned image sequence; An image quality acquisition unit is configured to, in response to determining that the result of the anomaly detection indicates that there is an anomaly in the human body part targeted by the scanned image sequence, acquire the image quality of the image region associated with the anomaly; as well as A display unit is configured to display an image representation of the anomaly, the image representation of the anomaly having display parameters indicating the image quality. Specifically, obtaining the image quality of the image region associated with the anomaly includes obtaining a first image quality of a first sub-region and a second image quality of a second sub-region of the image region associated with the anomaly, wherein the first image quality is different from the second image quality, and wherein displaying the image representation of the anomaly includes displaying the first sub-region and the second sub-region with different display parameters, and The device also includes a unit that displays an identifier of the scanning position corresponding to the current ultrasound scanning probe in association with the graphical representation of the anomaly.

2. The apparatus according to claim 1, wherein, The graphical representation of the anomaly is a three-dimensional representation obtained by three-dimensional modeling of the image region associated with the anomaly.

3. The apparatus of claim 1, further comprising performing the following operations after displaying a graphical representation of the anomaly: Obtain one or more newly added scanned images; and The image representation of the anomaly is updated based on the one or more newly added scan images.

4. The apparatus according to any one of claims 1-3, wherein, The display parameters are selected from the group consisting of: color, brightness, and texture.

5. The apparatus according to any one of claims 1-3, wherein, Obtaining the image quality of the image region associated with the anomaly includes: in response to determining that the result of the anomaly detection indicates that there is an anomaly in the human body part targeted by the scanned image sequence, performing image quality detection on the images in the scanned image sequence associated with the anomaly.

6. The apparatus according to any one of claims 1-3, further comprising generating a three-dimensional model of the anomaly based on the scanned image sequence, wherein, The graphical representation of the anomaly includes the three-dimensional model that displays the anomaly.

7. The apparatus according to any one of claims 1-3, wherein, Determining that the result of the anomaly detection indicates the presence of an anomaly in the human body portion targeted by the scanned image sequence includes determining the presence of at least two anomalies, and further includes the following operations: Determine the corresponding importance levels of the at least two anomalies; and Display the corresponding graphical representation of the at least two anomalies according to the importance level.

8. A computing device, comprising: Memory, processor, and computer program stored on said memory, The processor is configured to execute the computer program to perform the following steps: Obtain a sequence of scanned images, the sequence comprising one or more images obtained from ultrasound scans; Anomaly detection is performed on the scanned image sequence; In response to determining that the result of the anomaly detection indicates an anomaly in the human body portion targeted by the scanned image sequence, the image quality of the image region associated with the anomaly is obtained; and The image representation of the anomaly is displayed, and the image representation of the anomaly has display parameters indicating the image quality. Specifically, obtaining the image quality of the image region associated with the anomaly includes obtaining a first image quality of a first sub-region and a second image quality of a second sub-region of the image region associated with the anomaly, wherein the first image quality is different from the second image quality, and wherein displaying the image representation of the anomaly includes displaying the first sub-region and the second sub-region with different display parameters, and It also includes an identifier that displays the scanning position corresponding to the current ultrasound scanning probe in association with the graphical representation of the anomaly.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it performs the following steps: Obtain a sequence of scanned images, the sequence comprising one or more images obtained from ultrasound scans; Anomaly detection is performed on the scanned image sequence; In response to determining that the result of the anomaly detection indicates that there is an anomaly in the human body part targeted by the scanned image sequence, the image quality of the image region associated with the anomaly is obtained; as well as The image representation of the anomaly is displayed, and the image representation of the anomaly has display parameters indicating the image quality. Specifically, obtaining the image quality of the image region associated with the anomaly includes obtaining a first image quality of a first sub-region and a second image quality of a second sub-region of the image region associated with the anomaly, wherein the first image quality is different from the second image quality, and wherein displaying the image representation of the anomaly includes displaying the first sub-region and the second sub-region with different display parameters, and It also includes an identifier that displays the scanning position corresponding to the current ultrasound scanning probe in association with the graphical representation of the anomaly.

10. A computer program product comprising a computer program, wherein, When the computer program is executed by the processor, it performs the following steps: Obtain a sequence of scanned images, the sequence comprising one or more images obtained from ultrasound scans; Anomaly detection is performed on the scanned image sequence; In response to determining that the result of the anomaly detection indicates that there is an anomaly in the human body part targeted by the scanned image sequence, the image quality of the image region associated with the anomaly is obtained; as well as The image representation of the anomaly is displayed, and the image representation of the anomaly has display parameters indicating the image quality. Specifically, obtaining the image quality of the image region associated with the anomaly includes obtaining a first image quality of a first sub-region and a second image quality of a second sub-region of the image region associated with the anomaly, wherein the first image quality is different from the second image quality, and wherein displaying the image representation of the anomaly includes displaying the first sub-region and the second sub-region with different display parameters, and It also includes an identifier that displays the scanning position corresponding to the current ultrasound scanning probe in association with the graphical representation of the anomaly.

Images