Ultrasonic examination report generation method, equipment and device

Abstract

The invention relates to the technical field of ultrasonic artificial intelligence, in particular to an ultrasonic examination report generation method, equipment and device. The method comprises the following steps: detecting a target detection part corresponding to a collected target ultrasonic image; based on the target detection part, screening out at least one reference ultrasonic examination report corresponding to the target detection part from the candidate ultrasonic examination report set; determining a first measurement parameter of a target area where the target detection part is located in the target ultrasonic image, and extracting a first ROI image from the target ultrasonic image; determining a target reference ultrasound examination report from the at least one reference ultrasound examination report based on the first measurement parameter and the first ROI image; and on the basis of the first measurement parameter, the first ROI image and the target reference ultrasonic inspection report, the ultrasonic inspection report corresponding to the target ultrasonic image is generated, so that the accuracy and timeliness of the inspection result corresponding to the target ultrasonic image are ensured.

Description

Technical Field

[0001] The present application relates to the field of ultrasonic artificial intelligence technology, and in particular to a method, equipment and device for generating an ultrasonic examination report. Background Art

[0002] Ultrasonic imaging is widely used in the analysis of various diseases. Ultrasonic imaging can be performed on various biological tissues, and then the ultrasonic images are analyzed to generate an ultrasonic examination report. The characteristics of the biological tissues are analyzed based on the ultrasonic examination report to detect the status of the biological tissues.

[0003] Hospitals need to analyze a large number of ultrasound images and generate ultrasound examination reports every day. For example, departments such as physical examinations and obstetrics need to generate a large number of ultrasound examination reports.

[0004] In the related art, one method is that the doctor analyzes the ultrasound image and then generates the ultrasound examination report manually through the keyboard, which not only requires a lot of manpower and time costs, but is also prone to input errors and has poor timeliness. Another method is to use a preset ultrasound examination report template, and then fill in the corresponding places in the template after analyzing the ultrasound image. However, the ultrasound template in this method cannot meet a variety of ultrasound examination application scenarios, and the examination results are inaccurate, requiring the doctor to further confirm the examination results.

[0005] However, when analyzing the characteristics of biological tissues, ultrasound examination reports require high timeliness and accuracy. Therefore, there is an urgent need for an ultrasound examination report generation method that can timely analyze ultrasound images and generate ultrasound examination reports to ensure the timeliness and accuracy of ultrasound examination report generation. Summary of the invention

[0006] The purpose of this application is to provide a method, device and apparatus for generating an ultrasonic examination report, so as to solve the problem of low timeliness and accuracy in generating an ultrasonic examination report.

[0007] In a first aspect, the present application provides a method for generating an ultrasound examination report, the method being applied to an ultrasound device, comprising:

[0008] Detecting a target detection part corresponding to the acquired target ultrasound image;

[0009] Based on the target detection part, screening out at least one reference ultrasound examination report corresponding to the target detection part from a set of candidate ultrasound examination reports;

[0010] Determine a first measurement parameter of a target region where the target detection part is located in the target ultrasound image, and extract a first ROI image from the target ultrasound image;

[0011] determining a target reference ultrasound examination report from the at least one reference ultrasound examination report based on the first measurement parameter and the first ROI image;

[0012] An ultrasound examination report corresponding to the target ultrasound image is generated based on the first measurement parameter and the first ROI image, as well as the target reference ultrasound examination report.

[0013] In a possible implementation, determining a first measurement parameter of a target region in the target ultrasound image where the target detection part is located, and extracting a first ROI image from the target ultrasound image includes:

[0014] Segmenting the target ultrasound image based on a semantic segmentation model corresponding to the target detection part to obtain a segmented ultrasound image corresponding to the target ultrasound image;

[0015] Performing contour detection on the segmented ultrasonic image to determine an edge contour of a target area where the target detection part in the segmented ultrasonic image is located;

[0016] The target region is measured to obtain a first measurement parameter; and a first ROI image is extracted from the target ultrasound image according to a position of the target region in the segmented ultrasound image.

[0017] In a possible implementation manner, measuring the target area to obtain a first measurement parameter includes:

[0018] Determining a minimum bounding rectangle of the target area in the segmented ultrasound image;

[0019] The first measurement parameter is determined based on a minimum circumscribed rectangle of the target area and an edge contour of the target area.

[0020] In a possible implementation, extracting a first ROI image from the target ultrasound image according to the position of the target region in the segmented ultrasound image includes:

[0021] Determine a rectangular bounding box of an edge contour of the target area in the segmented ultrasound image;

[0022] The target ultrasound image is cropped according to the position of the rectangular bounding box in the segmented ultrasound image to obtain the first ROI image.

[0023] In a possible implementation, determining a target reference ultrasound examination report from the at least one reference ultrasound examination report based on the first measurement parameter and the first ROI image includes:

[0024] The following operations are performed for any reference ultrasound examination report in the at least one reference ultrasound examination report: obtaining a second measurement parameter and a second ROI image corresponding to the reference ultrasound examination report; determining a first similarity based on the first measurement parameter and the second measurement parameter; determining a second similarity based on the first ROI image and the second ROI image; determining a target similarity between the ultrasound image corresponding to the reference ultrasound examination report and the target ultrasound image according to the first similarity and the second similarity;

[0025] The reference ultrasonic examination report with the highest target similarity is selected from the at least one reference ultrasonic examination report as the target reference ultrasonic examination report.

[0026] In a possible implementation manner, determining the second similarity based on the first ROI image and the second ROI image includes:

[0027] Performing multi-dimensional feature extraction and feature perception on the first ROI image and the second ROI image respectively to obtain a first feature vector corresponding to the first ROI image and a second feature vector corresponding to the second ROI image;

[0028] Performing feature fusion on the first feature vector and the second feature vector to obtain an intermediate feature vector;

[0029] The intermediate feature vector is processed by a fully connected layer and an activation function to obtain a second similarity between the first ROI image and the second ROI image.

[0030] In a possible implementation, performing feature fusion on the first feature vector and the second feature vector to obtain an intermediate feature vector includes:

[0031] Performing a pooling operation of multiple dimensions on the first feature vector to obtain a plurality of sub-feature vectors corresponding to the first feature vector; concatenating the plurality of sub-feature vectors corresponding to the first feature vector with the first feature vector to obtain a first concatenated feature vector corresponding to the first feature vector; and

[0032] Performing the pooling operation of the multiple dimensions on the second feature vector to obtain a plurality of sub-feature vectors corresponding to the second feature vector; concatenating the plurality of sub-feature vectors corresponding to the second feature vector with the second feature vector to obtain a second concatenated feature vector corresponding to the second feature vector;

[0033] The first concatenated feature vector and the second concatenated feature vector are fused to obtain the intermediate feature vector.

[0034] In a possible implementation, the detecting the target detection part corresponding to the acquired target ultrasound image includes:

[0035] Performing feature extraction on the target ultrasonic image to obtain a feature vector of the target ultrasonic image;

[0036] Acquire feature vectors corresponding to a plurality of preset detection parts, and respectively calculate similarities between the feature vectors corresponding to the plurality of preset detection parts and the feature vector of the target ultrasound image;

[0037] The preset detection part corresponding to the feature vector with the highest similarity is used as the target detection part corresponding to the target ultrasound image.

[0038] In a second aspect, an embodiment of the present application provides an ultrasound device, the device comprising an image acquisition unit, a processor, and a display screen;

[0039] The image acquisition unit is used to acquire a target ultrasound image;

[0040] The processor is used to detect a target detection part corresponding to a captured target ultrasound image; based on the target detection part, screen out at least one reference ultrasound inspection report corresponding to the target detection part from a set of candidate ultrasound inspection reports; determine a first measurement parameter of a target region in the target ultrasound image where the target detection part is located, and extract a first ROI image from the target ultrasound image; based on the first measurement parameter and the first ROI image, determine a target reference ultrasound inspection report from the at least one reference ultrasound inspection report; based on the first measurement parameter and the first ROI image, and the target reference ultrasound inspection report, generate an ultrasound inspection report corresponding to the target ultrasound image.

[0041] The display screen is used to display the ultrasound examination report in a display interface.

[0042] In a third aspect, an embodiment of the present application provides an ultrasonic examination report generating device, the device comprising:

[0043] An acquisition module, used for acquiring a target ultrasonic image;

[0044] A detection part determination module is used to detect a target detection part corresponding to the acquired target ultrasound image;

[0045] A first screening module is used to screen out at least one reference ultrasound examination report corresponding to the target detection part from a set of candidate ultrasound examination reports based on the target detection part;

[0046] An information extraction module, used to determine a first measurement parameter of a target region where the target detection part is located in the target ultrasound image, and to extract a first ROI image from the target ultrasound image;

[0047] a second screening module, configured to determine a target reference ultrasound examination report from the at least one reference ultrasound examination report based on the first measurement parameter and the first ROI image;

[0048] A generating module is used to generate an ultrasound examination report corresponding to the target ultrasound image based on the first measurement parameter and the first ROI image, and the target reference ultrasound examination report.

[0049] In a fourth aspect, the present application provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by an electronic device, the electronic device is enabled to execute the ultrasound examination report generation method as described in any one of the first aspects above.

[0050] In a fifth aspect, the present application provides a computer program product, comprising a computer program: when the computer program is executed by a processor, the method for generating an ultrasound examination report as described in any one of the first aspects above is implemented.

[0051] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:

[0052] In an embodiment of the present application, a target detection part corresponding to a captured target ultrasound image is detected; based on the target detection part, at least one reference ultrasound examination report corresponding to the target detection part is screened out from a set of candidate ultrasound examination reports; a first measurement parameter of a target region in the target ultrasound image where the target detection part is located is determined, and a first ROI image is extracted from the target ultrasound image; based on the first measurement parameter and the first ROI image, a target reference ultrasound examination report is determined from at least one reference ultrasound examination report; based on the first measurement parameter and the first ROI image, and the target reference ultrasound examination report, an ultrasound examination report corresponding to the target ultrasound image is generated.

[0053] Thus, the target ultrasound image is segmented to obtain the first measurement parameter and the first ROI image, and the candidate ultrasound examination report set is screened based on the target detection site, the first measurement parameter and the first ROI image to obtain the target reference ultrasound examination report, which is replaced with the target reference ultrasound examination report to obtain the ultrasound examination report of the target ultrasound image, and combined with the examination results in the candidate ultrasound examination reports, the accuracy of the examination results corresponding to the target ultrasound image can be guaranteed, and the quality of the ultrasound examination report can be improved; at the same time, the target ultrasound image can be analyzed to directly generate the ultrasound examination report, thereby improving the timeliness of the generation of the ultrasound examination report.

[0054] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. Obviously, the drawings introduced below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0056] Figure 1 This is a structural block diagram of an ultrasonic device according to an embodiment of the present application;

[0057] Figure 2 This is a schematic diagram of the principle of realizing ultrasonic imaging by an ultrasonic device according to an embodiment of the present application;

[0058] Figure 3 This is an application scenario diagram of a method for generating an ultrasonic examination report according to an embodiment of the present application;

[0059] Figure 4 A schematic diagram of a flow chart of a method for generating an ultrasonic examination report provided in an embodiment of the present application;

[0060] Figure 5 A schematic diagram of a target ultrasound image provided in an embodiment of the present application;

[0061] Figure 6 A schematic diagram of a target ultrasound image after data enhancement provided in an embodiment of the present application;

[0062] Figure 7 A schematic diagram of a flow chart of a method for acquiring a target detection part provided in an embodiment of the present application;

[0063] Figure 8 A schematic diagram of a detection model for a target detection part provided in an embodiment of the present application;

[0064] Fig. 9 A schematic flow chart of a method for acquiring a first measurement parameter and a first ROI image provided in an embodiment of the present application;

[0065] Fig.10 A schematic diagram of a detection model for measurement parameters and ROI images provided in an embodiment of the present application;

[0066] Fig.11A schematic diagram of a minimum circumscribed rectangle and a rectangular bounding box provided in an embodiment of the present application;

[0067] Fig.12 A schematic diagram of a cropping process of a first ROI image provided in an embodiment of the present application;

[0068] Fig.13 A schematic diagram of a flow chart of a method for determining a target reference ultrasound examination report provided in an embodiment of the present application;

[0069] Fig.14 A schematic diagram of a flow chart of a method for determining a second similarity provided in an embodiment of the present application;

[0070] Fig.15 A schematic diagram of a feature perception module provided in an embodiment of the present application;

[0071] Fig.16 A schematic diagram of the structure of a spatial pyramid pooling provided in an embodiment of the present application;

[0072] Fig.17 A schematic diagram of the structure of a second similarity calculation model provided in an embodiment of the present application;

[0073] Fig.18 A schematic diagram of the overall architecture of a method for generating an ultrasonic examination report provided in an embodiment of the present application;

[0074] Fig.19 A schematic diagram of the structure of an ultrasonic device provided in an embodiment of the present application;

[0075] Fig. 20 A schematic diagram of the structure of an ultrasonic examination report generating device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Among them, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0077] Furthermore, in the description of the embodiments of the present application, unless otherwise specified, “ / ” means or. For example, A / B can mean A or B. The “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0078] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more.

[0079] Ultrasonic imaging is widely used in the analysis of various diseases. Ultrasonic imaging can be performed on various biological tissues, and then the ultrasonic images are analyzed to generate an ultrasonic examination report. The characteristics of the biological tissues are analyzed based on the ultrasonic examination report to detect the status of the biological tissues.

[0080] Hospitals need to analyze a large number of ultrasound images and generate ultrasound examination reports every day. For example, departments such as physical examinations and obstetrics need to generate a large number of ultrasound examination reports.

[0081] In the related art, one method is that the doctor analyzes the ultrasound image and then generates the ultrasound examination report manually through the keyboard, which not only requires a lot of manpower and time costs, but is also prone to input errors and has poor timeliness. Another method is to use a preset ultrasound examination report template, and then fill in the corresponding places in the template after analyzing the ultrasound image. However, the ultrasound template in this method cannot meet a variety of ultrasound examination application scenarios, and the examination results are inaccurate, requiring the doctor to further confirm the examination results.

[0082] However, when analyzing the characteristics of biological tissues, ultrasound examination reports require high timeliness and accuracy. Therefore, there is an urgent need for an ultrasound examination report generation method that can timely analyze ultrasound images and generate ultrasound examination reports to ensure the timeliness and accuracy of ultrasound examination report generation.

[0083] In view of this, the present application provides a method, device and apparatus for generating an ultrasonic examination report to solve the problem of low timeliness and accuracy in generating an ultrasonic examination report.

[0084] The inventive concept of the present application can be summarized as follows: in the embodiment of the present application, the target detection part corresponding to the collected target ultrasound image is detected; based on the target detection part, at least one reference ultrasound inspection report corresponding to the target detection part is screened out from the candidate ultrasound inspection report set; the first measurement parameter of the target region where the target detection part is located in the target ultrasound image is determined, and the first ROI (region of interest) image is extracted from the target ultrasound image; based on the first measurement parameter and the first ROI image, the target reference ultrasound inspection report is determined from at least one reference ultrasound inspection report; based on the first measurement parameter and the first ROI image, and the target reference ultrasound inspection report, an ultrasound inspection report corresponding to the target ultrasound image is generated. Thus, in combination with the inspection results in the candidate ultrasound inspection reports, the accuracy of the inspection results corresponding to the target ultrasound image can be guaranteed, and the quality of the ultrasound inspection report can be improved; at the same time, the target ultrasound image can be analyzed to directly generate an ultrasound inspection report, thereby improving the timeliness of ultrasound inspection report generation.

[0085] After introducing the main inventive concept of the embodiment of the present application, the ultrasonic device used in the ultrasonic inspection report generation method provided in the embodiment of the present application is introduced in conjunction with the accompanying drawings. Figure 1 As shown, it is a structural block diagram of an ultrasonic device provided in an embodiment of the present application.

[0086] It should be understood that Figure 1 The ultrasound device shown is only an example and the ultrasound device may have more Figure 1 The more or less components shown in the figure can be combined with two or more components, or can have different component configurations. The various components shown in the figure can be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application specific integrated circuits.

[0087] Figure 1 Schematically shows a hardware configuration block diagram of an ultrasound device according to an exemplary embodiment.

[0088] like Figure 1 As shown, the ultrasound device may include, for example: a processor 110, a memory 120, a display unit 130 and a probe 140; wherein:

[0089] A probe 140, used for transmitting an ultrasonic signal;

[0090] A display unit 130, configured to display a target ultrasound image;

[0091] The memory 120 is configured to store data required for ultrasound imaging, which may include software programs, application interface data, etc.;

[0092] The processor 110 is connected to the probe 140, the display unit 130 and the memory 120, respectively, and is configured to execute: detecting a target detection part corresponding to the acquired target ultrasound image; based on the target detection part, screening at least one reference ultrasound inspection report corresponding to the target detection part from a set of candidate ultrasound inspection reports; determining a first measurement parameter of a target region where the target detection part is located in the target ultrasound image, and extracting a first ROI image from the target ultrasound image; based on the first measurement parameter and the first ROI image, determining a target reference ultrasound inspection report from at least one reference ultrasound inspection report; and generating an ultrasound inspection report corresponding to the target ultrasound image based on the first measurement parameter and the first ROI image, as well as the target reference ultrasound inspection report.

[0093] The ultrasound examination report corresponding to the target ultrasound image is displayed through the display unit 130 .

[0094] Figure 2 Schematic diagram of the application principle according to an embodiment of the present application. Figure 1 The partial modules or functional components of the ultrasonic device shown are implemented, and only the main components will be described below, while other components, such as memory, controller, control circuit, etc., will not be described here in detail.

[0095] like Figure 2 As shown, the application environment may include a user interface 210 , a display unit 220 for displaying the user interface, and a processor 230 .

[0096] The display unit 220 may include a display panel 221 and a backlight assembly 222. The display panel 221 is configured to display the target ultrasound image, the backlight assembly 222 is located at the back of the display panel 221, and the backlight assembly 222 may include a plurality of backlight partitions (not shown in the figure), each of which may emit light to illuminate the display panel 221.

[0097] The processor 230 may be configured to control the brightness of the backlight source of each backlight partition in the backlight assembly 222 , and to control the probe to transmit an ultrasonic signal, and to receive and analyze an ultrasonic echo signal to obtain an ultrasonic image.

[0098] Among them, the processor 230 can detect the target detection part corresponding to the acquired target ultrasound image; based on the target detection part, screen out at least one reference ultrasound inspection report corresponding to the target detection part from the candidate ultrasound inspection report set; determine the first measurement parameter of the target area where the target detection part is located in the target ultrasound image, and extract the first ROI image from the target ultrasound image; based on the first measurement parameter and the first ROI image, determine the target reference ultrasound inspection report from at least one reference ultrasound inspection report; based on the first measurement parameter and the first ROI image, and the target reference ultrasound inspection report, generate an ultrasound inspection report corresponding to the target ultrasound image.

[0099] like Figure 3 As shown, it is an application scenario diagram of an ultrasonic examination report generation method provided in an embodiment of the present application. The diagram includes: a user 10, an ultrasonic device 20, and a memory 30;

[0100] The user 10 uses the probe of the ultrasound device 20 to collect a target ultrasound image corresponding to the biological tissue, and stores the collected target ultrasound image in the memory 30 .

[0101] When the ultrasound device 20 processes the target ultrasound image, it first obtains the target ultrasound image and a set of candidate ultrasound inspection reports from the storage 30 through the network, and then detects the target detection part corresponding to the target ultrasound image; based on the target detection part, screens out at least one reference ultrasound inspection report corresponding to the target detection part from the set of candidate ultrasound inspection reports; determines a first measurement parameter of a target region where the target detection part in the target ultrasound image is located, and extracts a first ROI image from the target ultrasound image; based on the first measurement parameter and the first ROI image, determines a target reference ultrasound inspection report from at least one reference ultrasound inspection report; based on the first measurement parameter and the first ROI image, and the target reference ultrasound inspection report, generates an ultrasound inspection report corresponding to the target ultrasound image.

[0102] Finally, the ultrasound examination report corresponding to the target ultrasound image is displayed in the display interface of the ultrasound device 20 and presented to the user 10, so that the user 10 can analyze the biological tissue according to the ultrasound examination report to detect the state of the biological tissue.

[0103] In the description of the present application, only a single ultrasound device and a memory are described in detail, but it should be understood by those skilled in the art that the ultrasound device 20 and the memory 30 shown are intended to indicate that the technical solution of the present application involves the operation of the ultrasound device and the memory. The description of a single ultrasound device and a memory is at least for the convenience of explanation, and does not imply a limitation on the number, type, or location of the ultrasound device and the memory. It should be noted that if additional modules are added to the illustrated environment or individual modules are removed from it, the underlying concept of the exemplary embodiment of the present application will not be changed.

[0104] It should be noted that the memory 30 in the embodiment of the present application may be, for example, a cache system, a hard disk storage, a memory storage, etc. In addition, the ultrasonic inspection report generation method proposed in the present application is not only applicable to Figure 3 The application scenario shown can also be used in other possible application scenarios, and the embodiments of the present application are not limited thereto.

[0105] To further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation steps shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiment of the present application.

[0106] See also Figure 4 , is a flow chart of a method for generating an ultrasonic examination report provided in an embodiment of the present application. Figure 4 As shown, the method for generating an ultrasonic examination report provided in the embodiment of the present application includes the following steps:

[0107] In step 401, a target detection part corresponding to the acquired target ultrasound image is detected.

[0108] Optionally, if the target ultrasound image has been acquired, the target detection part corresponding to the acquired target ultrasound image is detected; if the target ultrasound image has not been acquired, it is necessary to first obtain the original ultrasound data, and then convert the original ultrasound data into an original ultrasound image; and crop the effective area of ​​the original ultrasound image, such as removing UI information, to generate a target ultrasound image, and then detect the target detection part corresponding to the target ultrasound image.

[0109] Optionally, in order to improve the contrast of the image and enhance the contour performance, before detecting the target detection part corresponding to the target ultrasound image, the input target ultrasound image can be subjected to data enhancement processing such as rotation, random brightness transformation, and contrast limited adaptive histogram equalization (CLAHE), and then the target detection part corresponding to the target ultrasound image after the data enhancement processing is detected.

[0110] For example, Figure 5As shown in FIG. 1 , it is a schematic diagram of the target ultrasound image. CLAHE is used to perform data enhancement processing on the target ultrasound image. First, the target ultrasound image is divided into several sub-regions of fixed size; each sub-region is subjected to histogram equalization processing to obtain the grayscale value of the histogram of each sub-region; each grayscale level of the histogram of each sub-region is limited using a contrast limiting threshold to obtain the sub-region after grayscale processing; finally, each grayscale processed sub-region is spliced ​​and combined into a target ultrasound image after data enhancement, as shown in FIG. Figure 6 shown.

[0111] Therefore, each area of ​​the target ultrasound image is individually subjected to histogram equalization processing, and a contrast limiting threshold is set at the same time, which can not only enhance the image contrast but also suppress the background noise.

[0112] In a possible implementation manner, the target detection part corresponding to the target ultrasound image collected in the embodiment of the present application can be executed as follows: Figure 7 Steps shown:

[0113] In step 701, feature extraction is performed on a target ultrasound image to obtain a feature vector of the target ultrasound image;

[0114] In step 702, characteristic vectors corresponding to a plurality of preset detection parts are obtained, and similarities between the characteristic vectors corresponding to the plurality of preset detection parts and the characteristic vector of the target ultrasound image are respectively calculated;

[0115] In step 703, the preset detection part corresponding to the feature vector with the highest similarity is used as the target detection part corresponding to the target ultrasound image.

[0116] For example, Figure 8 As shown, it is a schematic diagram of the detection model of the target detection part provided in the embodiment of the present application. The trained section recognition model stores feature vectors corresponding to multiple preset detection parts, that is, each feature vector corresponds to a section category, that is, the detection part. After the target ultrasound image is subjected to data enhancement processing, it is input into the section recognition model. First, the feature of the target ultrasound image is extracted through the encoder to obtain the feature vector of the target ultrasound image; then, the similarity between the feature vectors corresponding to the multiple preset detection parts stored in the section recognition model and the feature vector of the target ultrasound image is calculated respectively; the preset detection part corresponding to the feature vector with the highest similarity is used as the target detection part corresponding to the target ultrasound image.

[0117] In step 402, based on the target detection part, at least one reference ultrasound examination report corresponding to the target detection part is screened out from the candidate ultrasound examination report set.

[0118] The ultrasound examination reports in the candidate ultrasound examination report set may be all historical ultrasound examination reports within a preset historical period, or all historical ultrasound examination reports in the hospital database. The historical ultrasound examination reports include the doctor's detailed examination results, patient information, test sites, corresponding ultrasound images, measurement information, ROI images, and other information.

[0119] In specific implementation, in the embodiment of the present application, the examination part, ultrasound probe and other patient information can be collected when collecting the target ultrasound image, or step 401 can be used to determine the detection part, and then the detection parts of all historical ultrasound examination reports in the candidate ultrasound examination report set are compared with the target detection part corresponding to the target ultrasound image, and the ultrasound probe and other patient information can be further compared, so as to retain some historical examination reports with the same detection part, ultrasound probe and other patient information, and obtain at least one reference ultrasound examination report corresponding to the target detection part.

[0120] Therefore, the candidate ultrasound examination report set is first screened by using patient information such as the detection site and ultrasound probe, thereby narrowing the subsequent screening range and improving the retrieval speed of the ultrasound examination report.

[0121] In step 403, a first measurement parameter of a target region where a target detection part is located in a target ultrasound image is determined, and a first ROI image is extracted from the target ultrasound image.

[0122] Optionally, the order of step 402 and step 403 in the embodiment of the present application can be interchanged, that is, step 402 can be executed first, and then step 403; or step 403 can be executed first, and then step 402.

[0123] In a possible implementation, after screening the candidate ultrasound examination report set for the detection part in the embodiment of the present application, in order to further accurately screen, further screening can be performed based on the measurement parameters and ROI image of the target ultrasound image. Therefore, in the embodiment of the present application, determining the first measurement parameter of the target area where the target detection part is located in the target ultrasound image, and extracting the first ROI image from the target ultrasound image, can be performed as follows: Fig. 9 Steps shown:

[0124] In step 901, based on the semantic segmentation model corresponding to the target detection part, the target ultrasound image is segmented to obtain a segmented ultrasound image corresponding to the target ultrasound image;

[0125] In step 902, contour detection is performed on the segmented ultrasound image to determine the edge contour of the target area where the target detection part in the segmented ultrasound image is located;

[0126] In step 903, the target region is measured to obtain a first measurement parameter; and a first ROI image is extracted from the target ultrasound image according to the position of the target region in the segmented ultrasound image.

[0127] For example, Fig.10 As shown, it is a schematic diagram of the detection model of the measurement parameters and ROI image provided in the embodiment of the present application. After determining the target detection part, the target ultrasound image is segmented using the corresponding semantic segmentation model (Unet, DeepLab, etc.) to obtain the segmentation result, that is, the segmented ultrasound image corresponding to the target ultrasound image; the segmented ultrasound image is then post-processed to obtain the first measurement parameter and the first ROI image. Among them, during post-processing, it is necessary to use the contour retrieval algorithm cv2.findContours of OpenCV to perform contour detection on the segmented ultrasound image and determine the edge contour of the target area where the target detection part is located in the segmented ultrasound image. Then, based on the edge contour of the target area, the first measurement parameter and the first ROI image are obtained.

[0128] In a possible implementation, measuring the target area to obtain the first measurement parameter is performed by: determining a minimum bounding rectangle of the target area in the segmented ultrasound image; and determining the first measurement parameter based on the minimum bounding rectangle of the target area and an edge contour of the target area.

[0129] For example, Fig.11 As shown, the white irregular image is the target area, and the small white rectangle is the minimum enclosing rectangle. The cv2.minAreaRect function is used to calculate the edge contour of the target area to obtain the minimum enclosing rectangle of the target area. Then, the minimum enclosing rectangle and the point set on the edge contour of the target area determined by the cv2.findContours function are calculated to obtain the first measurement parameter of the target area.

[0130] The first measurement parameter includes but is not limited to measurement information such as the perimeter, area, major diameter, minor diameter, vertical depth, etc. of the target area.

[0131] In a possible implementation, extracting a first ROI image from a target ultrasound image according to a position of the target region in the segmented ultrasound image can be performed as follows: determining a rectangular bounding box of an edge contour of the target region in the segmented ultrasound image; and cropping the target ultrasound image according to a position of the rectangular bounding box in the segmented ultrasound image to obtain the first ROI image.

[0132] For example, Fig.11As shown in the figure, the white irregular image is the target area, and the gray larger rectangle is the rectangular bounding box. The bounding rectangle function cv2.boundingRect is used to find the bounding rectangle with a rotation angle of 0, determine the coordinates of the upper left corner of the target area and the width and height information of the rectangle, and obtain the position coordinates of the rectangular bounding box in the segmented ultrasound image as ((x, y), w, h). Then the position coordinates of the rectangular bounding box in the segmented ultrasound image are mapped to the target ultrasound image, so that the target ultrasound image is cropped according to the coordinate position to obtain the first ROI image.

[0133] The segmented ultrasound image and the target ultrasound image have the same size.

[0134] Edges are important features of ultrasound images. To ensure that target edge information is not lost during cropping, a pixel expansion factor n can be set in the embodiment of the present application to correct the coordinates of the position to be cropped. The corrected position coordinates are ((xn, y+n), w+2n, h+2n). The target ultrasound image is then cropped according to the corrected position coordinates to obtain a first ROI image.

[0135] like Fig.12 As shown, it is a schematic diagram of the cropping process of the first ROI image. Fig.12 The first image in the figure is the target ultrasound image, the second image is the segmented ultrasound image, the rectangle in the second image is the position coordinate of the rectangular bounding box determined according to the pixel expansion factor, and then the corresponding position in the first image is cropped according to the position coordinate to obtain the first ROI image in the third image.

[0136] Therefore, the first ROI image can be acquired through the above approach. The first ROI image can suppress the interference of the background area relative to the entire target ultrasound image, reduce the calculation complexity, and improve the recognition accuracy.

[0137] In step 404 , a target reference ultrasound examination report is determined from at least one reference ultrasound examination report based on the first measurement parameter and the first ROI image.

[0138] In a possible implementation, in the embodiment of the present application, based on the first measurement parameter and the first ROI image, determining a target reference ultrasound examination report from at least one reference ultrasound examination report may be performed as follows: Fig.13 Steps shown:

[0139] In step 1301, the following operations are performed for any reference ultrasound examination report in at least one reference ultrasound examination report: obtaining a second measurement parameter and a second ROI image corresponding to the reference ultrasound examination report; determining a first similarity based on the first measurement parameter and the second measurement parameter; determining a second similarity based on the first ROI image and the second ROI image; and determining a target similarity between the ultrasound image corresponding to the reference ultrasound examination report and the target ultrasound image based on the first similarity and the second similarity.

[0140] In a possible implementation manner, in the embodiment of the present application, determining the first similarity based on the first measurement parameter and the second measurement parameter may be performed as follows:

[0141] A distance metric between the first measurement parameter and the second measurement parameter is calculated, and the distance metric is used as the first similarity, for example, a Euclidean distance or a cosine distance is calculated.

[0142] If the first measurement parameter includes multiple parameters, the Euclidean distance between each first measurement parameter and the corresponding second measurement parameter can be calculated, and then the average value of each Euclidean distance is used as the first similarity or the Euclidean distances are weighted and summed to obtain the first similarity.

[0143] In this way, it is possible to avoid the situation where the lesion images have high similarity but large differences in measurement parameters due to adjustments to the depth, gain, etc. of the ultrasound equipment, thereby obtaining different treatment plans. For example, the degree of attention and treatment plans for uterine fibroids of different sizes in clinical examinations are different.

[0144] In a possible implementation manner, in the embodiment of the present application, determining the second similarity based on the first ROI image and the second ROI image may be performed as follows: Fig.14 Steps shown:

[0145] In step 1401, multi-dimensional feature extraction and feature perception are performed on the first ROI image and the second ROI image respectively to obtain a first feature vector corresponding to the first ROI image and a second feature vector corresponding to the second ROI image.

[0146] In specific implementation, the two feature extraction networks (Convolutional Neural Networks, CNN) of the first ROI image and the second ROI image have the same network structure, and the feature extraction network is composed of a convolution layer, a pooling layer and an activation function, and models such as VGG and ResNet can also be selected. In the embodiment of the present application, the first ROI image and the second ROI image are respectively input into the feature extraction network, and the corresponding feature mapping matrices are obtained through the convolution layer composed of multiple convolution kernels.

[0147] Among them, each convolution kernel in the convolution layer can extract specific features, and different convolution kernels can extract different features. The extracted features are multi-level, multi-scale, and multi-semantic, including not only edge, texture, color and other information, but also shape, structure, and higher-dimensional semantic information; the pooling layer can reduce the dimension of the image while retaining important information and reducing computational complexity; the activation function adds nonlinear factors to the neural network to enhance the fitting ability of the neural network. Commonly used activation functions include sigmoid function, tanh activation function, ReLU activation function, and Leaky ReLU function.

[0148] Among them, the weight parameters (Weight) of the feature extraction network are the same, thereby reducing the number of parameters and the complexity of the model; the feature vectors of two different spatial dimensions can also be mapped to the same spatial dimension, so that the data distribution remains consistent and different feature vectors are encoded on the same spatial dimension.

[0149] During feature extraction, the first ROI image and the second ROI image will generate corresponding multi-dimensional feature vectors after feature extraction, but there will be feature redundant information in the multi-dimensional feature vector, which will interfere with the accuracy of the model.

[0150] Therefore, in the embodiment of the present application, a feature perception module (FPM) is added to the model, and the structure is as follows: Fig.15 As shown in the figure, the multidimensional feature vector is first subjected to batch normalization (BN) and full connection (FC) operations, and then activated by the Sigmoid function to obtain the perception vector, and then the perception vector is multiplied by the multidimensional feature vector to obtain the first feature vector corresponding to the first ROI image and the second feature vector corresponding to the second ROI image.

[0151] The first feature vector corresponding to the first ROI image and the second feature vector corresponding to the second ROI image have the same dimension as the multi-dimensional feature vector.

[0152] Therefore, after multi-dimensional feature extraction and feature perception, the feature vector can be adaptively adjusted so that the model pays more attention to key feature information, suppresses irrelevant features, and improves the model accuracy.

[0153] In step 1402, feature fusion is performed on the first feature vector and the second feature vector to obtain an intermediate feature vector.

[0154] In a possible implementation manner, the first feature vector and the second feature vector are subjected to feature fusion in the embodiment of the present application to obtain an intermediate feature vector, which may be executed as follows:

[0155] Performing a pooling operation of multiple dimensions on the first feature vector to obtain a plurality of sub-feature vectors corresponding to the first feature vector; concatenating the plurality of sub-feature vectors corresponding to the first feature vector with the first feature vector to obtain a first concatenated feature vector corresponding to the first feature vector; and performing a pooling operation of multiple dimensions on the second feature vector to obtain a plurality of sub-feature vectors corresponding to the second feature vector; concatenating the plurality of sub-feature vectors corresponding to the second feature vector with the second feature vector to obtain a second concatenated feature vector corresponding to the second feature vector;

[0156] The first concatenated feature vector and the second concatenated feature vector are fused to obtain an intermediate feature vector.

[0157] In specific implementation, since the number of features of the fully connected layer is fixed, the image size required to be input to the network is also fixed. However, the first ROI image and the second ROI image input in the embodiment of the present application are obtained by cropping the position coordinate mapping, and the size is not fixed. In this regard, the image is usually cropped or stretched, but both methods will produce size errors and destroy the aspect ratio of the original image, causing deformation and distortion of the image.

[0158] Therefore, in the embodiment of the present application, spatial pyramid pooling (SPP) is added after the feature perception module FPM, such as Fig.16 The figure shows the schematic diagram of the SPP structure. The feature maps of any size obtained after feature extraction and feature perception of the ROI image are subjected to pooling operations (max pool) of dimensions such as 4×4, 2×2, and 1×1 to obtain Fig.16 The three sub-feature vectors in the dotted part are then concatenated (contacted) to obtain a feature vector of fixed length, i.e., a concatenated feature vector. Thus, the first feature vector is input into the spatial pyramid pooling to obtain the first concatenated feature vector, and the second feature vector is input into the spatial pyramid pooling to obtain the second concatenated feature vector.

[0159] Among them, the feature vector after the feature map is pooled in 1×1 dimension is the feature vector for feature extraction and feature perception of the ROI image.

[0160] Therefore, SPP can realize the fusion of local features and global features and enrich the expressiveness of feature maps.

[0161] like Fig.17 As shown, the first concatenated feature vector and the second concatenated feature vector output by the spatial pyramid pooling are flattened to obtain vectors X1 and X2 of the same dimension, that is, vector X1 = (x11 ,x 12 ,x 13 …x 1n ) and X2=(x 21 ,x 22 ,x 23 …x 2n ), and then in the Loss Layer layer, the distance between vectors X1 and X2 is calculated using formula (1) to obtain the intermediate feature vector:

[0162]

[0163] In step 1403, the intermediate feature vector is processed by a fully connected layer and an activation function to obtain a second similarity between the first ROI image and the second ROI image.

[0164] When implementing it, Fig.17 As shown, the first ROI image and the second ROI image are respectively subjected to the feature extraction network CNN, the feature perception module FPM, and the spatial pyramid pooling SPP to obtain the first spliced ​​feature vector corresponding to the first feature vector and the second spliced ​​feature vector corresponding to the second ROI image; then the first spliced ​​feature vector and the second spliced ​​feature vector are respectively flattened by Flatten to obtain vectors X1 and X2 of the same dimension, and then the vectors X1 and X2 are calculated in the Loss Layer layer to obtain the intermediate feature vector; finally, the intermediate feature vector output by the Loss Layer layer is subjected to the fully connected layer (FC) to obtain a scalar, and then the scalar number is converted to a value between [0,1] through the activation function Sigmoid, and this value is used as the second similarity between the first ROI image and the second ROI image.

[0165] Among them, Fig.17 When the model shown is trained, the input is a ROI image pair and an expert annotation label in the format of ((first ROI image, second ROI image), Label), Label=1 indicates that the image pair is completely similar, Label=0 indicates that the image pair is completely dissimilar, and the model output is the second similarity of the ROI image pair.

[0166] Finally, the weights corresponding to the first similarity and the second similarity are obtained, and then the target similarity between the ultrasound image corresponding to the reference ultrasound examination report and the target ultrasound image can be determined according to the first similarity and the second similarity according to formula (2):

[0167] S=S1×weight+ S2×(1-weight) (2)

[0168] Among them, S is the target similarity; S1 is the first similarity; S2 is the second similarity; weight is the weight of the first similarity; 1-weight is the weight of the second similarity.

[0169] Among them, training Fig.17 The model shown uses the Contrastive Loss function, which has the following formula:

[0170]

[0171]

[0172] Wherein, X1, X2 represent the feature vectors mapped by the convolutional neural network of the input image; Y = 1 or 0, representing the similarity between the input samples, if the model predicts that the two input images are similar, Y = 1, otherwise Y = 0; N represents the number of input images, in the embodiment of the present application, N = 2; D W is the Euclidean distance between X1 and X2, and m is the set threshold.

[0173] During the model training process, the optimizer based on the back-propagation algorithm reduces the loss function, thereby increasing the similarity between the positive sample pairs ((first ROI image, second ROI image), Label=1) and reducing the similarity between the negative sample pairs ((first ROI image, second ROI image), Label=0). Therefore, the trained model has the ability to judge the similarity of two ultrasound images.

[0174] In step 1302, a reference ultrasound examination report with the highest target similarity is selected from at least one reference ultrasound examination report as a target reference ultrasound examination report.

[0175] Therefore, after similarity calculation, the reference ultrasound examination report with the highest target similarity is used as the target reference ultrasound examination report.

[0176] In step 405, an ultrasound examination report corresponding to the target ultrasound image is generated based on the first measurement parameter and the first ROI image, as well as the target reference ultrasound examination report.

[0177] like Fig.18 The figure shows the overall architecture of the ultrasound examination report generation method provided in the embodiment of the present application. The image acquisition module is used to obtain the original ultrasound data and patient information, and convert the ultrasound data into the original ultrasound image. The image processing module is used to connect the image acquisition module, perform effective area cropping (remove UI information, etc.) on the original ultrasound image, and generate a target ultrasound image.

[0178] The automatic measurement module is used to receive the target ultrasound image converted by the image processing module, obtain the target detection part of the target ultrasound image through the section recognition model, and then obtain the first ROI image and the first measurement parameter through the semantic segmentation model and post-processing calculation.

[0179] The report retrieval module is used to combine the first ROI image output by the automatic measurement module, the first measurement parameter and the patient information of the image acquisition module, first match the patient information and the target detection part, screen out at least one reference ultrasound examination report from the candidate ultrasound examination report set, narrow the retrieval scope, and then calculate the similarity based on the first measurement parameter and the first ROI image, and further screen out the target reference ultrasound examination report from the at least one reference ultrasound examination report.

[0180] Finally, the report combination module is used to replace the first measurement parameter, the first ROI image, the target ultrasound image and other information into the corresponding position of the target reference ultrasound examination report, generate an ultrasound examination report corresponding to the target ultrasound image, and finally the doctor reviews the ultrasound examination report.

[0181] In the present application, a standard data range can also be set, and the first measurement parameter is compared with the standard data range to obtain the inspection result. For example, if the length value of part A is 10, the standard range should be >12, and the inspection conclusion is automatically displayed as: the length value is low.

[0182] Based on the foregoing description, in an embodiment of the present application, a target detection part corresponding to a collected target ultrasound image is detected; based on the target detection part, at least one reference ultrasound inspection report corresponding to the target detection part is screened out from a set of candidate ultrasound inspection reports; a first measurement parameter of a target area where the target detection part is located in the target ultrasound image is determined, and a first ROI image is extracted from the target ultrasound image; based on the first measurement parameter and the first ROI image, a target reference ultrasound inspection report is determined from at least one reference ultrasound inspection report; based on the first measurement parameter and the first ROI image, and the target reference ultrasound inspection report, an ultrasound inspection report corresponding to the target ultrasound image is generated.

[0183] Thus, the target ultrasound image is measured to obtain the first measurement parameter and the first ROI image, and the candidate ultrasound examination report set is screened based on the target detection site, the first measurement parameter and the first ROI image to obtain the target reference ultrasound examination report, which is replaced with the target reference ultrasound examination report to obtain the ultrasound examination report of the target ultrasound image, and combined with the examination results in the candidate ultrasound examination reports, the accuracy of the examination results corresponding to the target ultrasound image can be guaranteed, and the quality of the ultrasound examination report can be improved; at the same time, the target ultrasound image can be analyzed to directly generate the ultrasound examination report, thereby improving the timeliness of the generation of the ultrasound examination report.

[0184] Refer to the following Fig.19 1900 according to this embodiment of the present application will be described. Fig.19 The ultrasonic device 1900 is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0185] like Fig.19 As shown, the components of the ultrasound device 1900 may include, but are not limited to: an image acquisition unit 1901, at least one processor 1902, a display screen 1903, at least one memory 1904, and a bus 1905 connecting different system components.

[0186] Bus 1905 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus structures. Memory 1904 may include readable media in the form of volatile memory, such as random access memory (RAM) or cache memory, and may further include read-only memory (ROM).

[0187] Memory 1904 may also include a program / utility having a set (at least one) of program modules, such program modules including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0188] The ultrasound device 1900 may also communicate with one or more external devices (e.g., keyboards, pointing devices, etc.), may also communicate with one or more devices that enable a user to interact with the ultrasound device 1900, or may communicate with any device that enables the ultrasound device 1900 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface. Furthermore, the ultrasound device 1900 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), or a public network, such as the Internet) via a network adapter.

[0189] An image acquisition unit 1901 is used to acquire a target ultrasound image;

[0190] Processor 1902 is configured to detect a target detection part corresponding to the acquired target ultrasound image; based on the target detection part, screen out at least one reference ultrasound examination report corresponding to the target detection part from a set of candidate ultrasound examination reports; determine a first measurement parameter of a target region where the target detection part is located in the target ultrasound image, and extract a first ROI image from the target ultrasound image; based on the first measurement parameter and the first ROI image, determine a target reference ultrasound examination report from the at least one reference ultrasound examination report; based on the first measurement parameter and the first ROI image, and the target reference ultrasound examination report, generate an ultrasound examination report corresponding to the target ultrasound image;

[0191] Display screen 1903 is used to display the ultrasound examination report in the display interface.

[0192] In a possible implementation, the processor 1902 is specifically configured to:

[0193] Segmenting the target ultrasound image based on a semantic segmentation model corresponding to the target detection part to obtain a segmented ultrasound image corresponding to the target ultrasound image;

[0194] Performing contour detection on the segmented ultrasonic image to determine an edge contour of a target area where the target detection part in the segmented ultrasonic image is located;

[0195] The target region is measured to obtain a first measurement parameter; and a first ROI image is extracted from the target ultrasound image according to a position of the target region in the segmented ultrasound image.

[0196] In a possible implementation, the processor 1902 is specifically configured to:

[0197] Determining a minimum bounding rectangle of the target area in the segmented ultrasound image;

[0198] The first measurement parameter is determined based on a minimum circumscribed rectangle of the target area and an edge contour of the target area.

[0199] In a possible implementation, the processor 1902 is specifically configured to:

[0200] Determine a rectangular bounding box of an edge contour of the target area in the segmented ultrasound image;

[0201] The target ultrasound image is cropped according to the position of the rectangular bounding box in the segmented ultrasound image to obtain the first ROI image.

[0202] In a possible implementation, the processor 1902 is specifically configured to:

[0203] The following operations are performed for any reference ultrasound examination report in the at least one reference ultrasound examination report: obtaining a second measurement parameter and a second ROI image corresponding to the reference ultrasound examination report; determining a first similarity based on the first measurement parameter and the second measurement parameter; determining a second similarity based on the first ROI image and the second ROI image; determining a target similarity between the ultrasound image corresponding to the reference ultrasound examination report and the target ultrasound image according to the first similarity and the second similarity;

[0204] The reference ultrasonic examination report with the highest target similarity is selected from the at least one reference ultrasonic examination report as the target reference ultrasonic examination report.

[0205] In a possible implementation, the processor 1902 is specifically configured to:

[0206] Performing multi-dimensional feature extraction and feature perception on the first ROI image and the second ROI image respectively to obtain a first feature vector corresponding to the first ROI image and a second feature vector corresponding to the second ROI image;

[0207] Performing feature fusion on the first feature vector and the second feature vector to obtain an intermediate feature vector;

[0208] The intermediate feature vector is processed by a fully connected layer and an activation function to obtain a second similarity between the first ROI image and the second ROI image.

[0209] In a possible implementation, the processor 1902 is specifically configured to:

[0210] Performing a pooling operation of multiple dimensions on the first feature vector to obtain a plurality of sub-feature vectors corresponding to the first feature vector; concatenating the plurality of sub-feature vectors corresponding to the first feature vector with the first feature vector to obtain a first concatenated feature vector corresponding to the first feature vector; and

[0211] Performing the pooling operation of the multiple dimensions on the second feature vector to obtain a plurality of sub-feature vectors corresponding to the second feature vector; concatenating the plurality of sub-feature vectors corresponding to the second feature vector with the second feature vector to obtain a second concatenated feature vector corresponding to the second feature vector;

[0212] The first concatenated feature vector and the second concatenated feature vector are fused to obtain the intermediate feature vector.

[0213] In a possible implementation, the processor 1902 is specifically configured to:

[0214] Performing feature extraction on the target ultrasonic image to obtain a feature vector of the target ultrasonic image;

[0215] Acquire feature vectors corresponding to a plurality of preset detection parts, and respectively calculate similarities between the feature vectors corresponding to the plurality of preset detection parts and the feature vector of the target ultrasound image;

[0216] The preset detection part corresponding to the feature vector with the highest similarity is used as the target detection part corresponding to the target ultrasound image.

[0217] like Fig. 20 As shown, the embodiment of the present application provides an ultrasonic examination report generating device 2000, comprising:

[0218] The acquisition module 2001 is used to acquire a target ultrasound image;

[0219] The detection part determination module 2002 is used to detect the target detection part corresponding to the acquired target ultrasound image;

[0220] A first screening module 2003 is used to screen out at least one reference ultrasound examination report corresponding to the target detection part from a set of candidate ultrasound examination reports based on the target detection part;

[0221] An information extraction module 2004 is used to determine a first measurement parameter of a target region where the target detection part is located in the target ultrasound image, and to extract a first ROI image from the target ultrasound image;

[0222] A second screening module 2005 is configured to determine a target reference ultrasound examination report from the at least one reference ultrasound examination report based on the first measurement parameter and the first ROI image;

[0223] The generating module 2006 is configured to generate an ultrasound examination report corresponding to the target ultrasound image based on the first measurement parameter, the first ROI image, and the target reference ultrasound examination report.

[0224] In a possible implementation, the information extraction module 2004 is specifically used to:

[0225] Segmenting the target ultrasound image based on a semantic segmentation model corresponding to the target detection part to obtain a segmented ultrasound image corresponding to the target ultrasound image;

[0226] Performing contour detection on the segmented ultrasonic image to determine an edge contour of a target area where the target detection part in the segmented ultrasonic image is located;

[0227] The target region is measured to obtain a first measurement parameter; and a first ROI image is extracted from the target ultrasound image according to a position of the target region in the segmented ultrasound image.

[0228] In a possible implementation, the information extraction module 2004 is specifically used to:

[0229] Determining a minimum bounding rectangle of the target area in the segmented ultrasound image;

[0230] The first measurement parameter is determined based on a minimum circumscribed rectangle of the target area and an edge contour of the target area.

[0231] In a possible implementation, the information extraction module 2004 is specifically used to:

[0232] Determine a rectangular bounding box of an edge contour of the target area in the segmented ultrasound image;

[0233] The target ultrasound image is cropped according to the position of the rectangular bounding box in the segmented ultrasound image to obtain the first ROI image.

[0234] In a possible implementation manner, the second screening module 2005 is specifically used to:

[0235] The following operations are performed for any reference ultrasound examination report in the at least one reference ultrasound examination report: obtaining a second measurement parameter and a second ROI image corresponding to the reference ultrasound examination report; determining a first similarity based on the first measurement parameter and the second measurement parameter; determining a second similarity based on the first ROI image and the second ROI image; determining a target similarity between the ultrasound image corresponding to the reference ultrasound examination report and the target ultrasound image according to the first similarity and the second similarity;

[0236] The reference ultrasonic examination report with the highest target similarity is selected from the at least one reference ultrasonic examination report as the target reference ultrasonic examination report.

[0237] In a possible implementation manner, the second screening module 2005 is specifically used to:

[0238] Performing multi-dimensional feature extraction and feature perception on the first ROI image and the second ROI image respectively to obtain a first feature vector corresponding to the first ROI image and a second feature vector corresponding to the second ROI image;

[0239] Performing feature fusion on the first feature vector and the second feature vector to obtain an intermediate feature vector;

[0240] The intermediate feature vector is processed by a fully connected layer and an activation function to obtain a second similarity between the first ROI image and the second ROI image.

[0241] In a possible implementation manner, the second screening module 2005 is specifically used to:

[0242] Performing a pooling operation of multiple dimensions on the first feature vector to obtain a plurality of sub-feature vectors corresponding to the first feature vector; concatenating the plurality of sub-feature vectors corresponding to the first feature vector with the first feature vector to obtain a first concatenated feature vector corresponding to the first feature vector; and

[0243] Performing the pooling operation of the multiple dimensions on the second feature vector to obtain a plurality of sub-feature vectors corresponding to the second feature vector; concatenating the plurality of sub-feature vectors corresponding to the second feature vector with the second feature vector to obtain a second concatenated feature vector corresponding to the second feature vector;

[0244] The first concatenated feature vector and the second concatenated feature vector are fused to obtain the intermediate feature vector.

[0245] In a possible implementation manner, the detection site determination module 2002 is specifically configured to:

[0246] Performing feature extraction on the target ultrasonic image to obtain a feature vector of the target ultrasonic image;

[0247] Acquire feature vectors corresponding to a plurality of preset detection parts, and respectively calculate similarities between the feature vectors corresponding to the plurality of preset detection parts and the feature vector of the target ultrasound image;

[0248] The preset detection part corresponding to the feature vector with the highest similarity is used as the target detection part corresponding to the target ultrasound image.

[0249] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the instructions can be executed by a processor to complete the above-mentioned ultrasound inspection report generation method. Optionally, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0250] In an exemplary embodiment, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, any method of the ultrasound examination report generation method provided in the present application is implemented.

[0251] In an exemplary embodiment, various aspects of an ultrasound inspection report generation method provided by the present application can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of the ultrasound inspection report generation method according to various exemplary embodiments of the present application described above in this specification.

[0252] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0253] The program product for the method for generating an ultrasonic inspection report of the embodiment of the present application can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on an electronic device. However, the program product of the present application is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, apparatus, or device.

[0254] The readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein the readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0255] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0256] The program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as "like" language or similar programming languages. The program code may be executed entirely on the user electronic device, partially on the user device, as an independent software package, partially on the user electronic device and partially on a remote electronic device, or entirely on a remote electronic device or a server. In cases involving remote electronic devices, the remote electronic device may be connected to the user electronic device via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external electronic device (e.g., via the Internet using an Internet service provider).

[0257] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0258] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

Claims

1. An ultrasonic device, characterized in that: The device includes an image acquisition unit, a processor and a display screen; The image acquisition unit is used to acquire a target ultrasound image; The processor is configured to detect a target detection part corresponding to the acquired target ultrasound image; based on the target detection part, screen out at least one reference ultrasound examination report corresponding to the target detection part from a set of candidate ultrasound examination reports; determine a first measurement parameter of a target region in the target ultrasound image where the target detection part is located, and extract a first ROI image from the target ultrasound image; based on the first measurement parameter and the first ROI image, determine a target reference ultrasound examination report from the at least one reference ultrasound examination report; based on the first measurement parameter and the first ROI image, and the target reference ultrasound examination report, generate an ultrasound examination report corresponding to the target ultrasound image; The display screen is used to display the ultrasound examination report in a display interface.

2. The ultrasonic device according to claim 1, characterized in that The processor is specifically used for: Segmenting the target ultrasound image based on a semantic segmentation model corresponding to the target detection part to obtain a segmented ultrasound image corresponding to the target ultrasound image; Performing contour detection on the segmented ultrasonic image to determine an edge contour of a target area where the target detection part in the segmented ultrasonic image is located; The target region is measured to obtain a first measurement parameter; and a first ROI image is extracted from the target ultrasound image according to a position of the target region in the segmented ultrasound image.

3. The ultrasonic device according to claim 2, characterized in that The processor is specifically used for: Determining a minimum bounding rectangle of the target area in the segmented ultrasound image; The first measurement parameter is determined based on a minimum circumscribed rectangle of the target area and an edge contour of the target area.

4. The ultrasonic device according to claim 2, characterized in that The processor is specifically used for: Determine a rectangular bounding box of an edge contour of the target area in the segmented ultrasound image; The target ultrasound image is cropped according to the position of the rectangular bounding box in the segmented ultrasound image to obtain the first ROI image.

5. The ultrasonic device according to claim 1, characterized in that The processor is specifically used for: The following operations are performed for any reference ultrasound examination report in the at least one reference ultrasound examination report: obtaining a second measurement parameter and a second ROI image corresponding to the reference ultrasound examination report; determining a first similarity based on the first measurement parameter and the second measurement parameter; determining a second similarity based on the first ROI image and the second ROI image; determining a target similarity between the ultrasound image corresponding to the reference ultrasound examination report and the target ultrasound image according to the first similarity and the second similarity; The reference ultrasonic examination report with the highest target similarity is selected from the at least one reference ultrasonic examination report as the target reference ultrasonic examination report.

6. The ultrasonic device according to claim 5, characterized in that The processor is specifically used for: Performing multi-dimensional feature extraction and feature perception on the first ROI image and the second ROI image respectively to obtain a first feature vector corresponding to the first ROI image and a second feature vector corresponding to the second ROI image; Performing feature fusion on the first feature vector and the second feature vector to obtain an intermediate feature vector; The intermediate feature vector is processed by a fully connected layer and an activation function to obtain a second similarity between the first ROI image and the second ROI image.

7. The ultrasonic device according to claim 6, characterized in that The processor is specifically used for: Performing a pooling operation of multiple dimensions on the first feature vector to obtain a plurality of sub-feature vectors corresponding to the first feature vector; concatenating the plurality of sub-feature vectors corresponding to the first feature vector with the first feature vector to obtain a first concatenated feature vector corresponding to the first feature vector; as well as Performing the pooling operation of the multiple dimensions on the second feature vector to obtain a plurality of sub-feature vectors corresponding to the second feature vector; concatenating the plurality of sub-feature vectors corresponding to the second feature vector with the second feature vector to obtain a second concatenated feature vector corresponding to the second feature vector; The first concatenated feature vector and the second concatenated feature vector are fused to obtain the intermediate feature vector.

8. The ultrasonic device according to claim 1, characterized in that The processor is specifically used for: Performing feature extraction on the target ultrasonic image to obtain a feature vector of the target ultrasonic image; Acquire feature vectors corresponding to a plurality of preset detection parts, and respectively calculate similarities between the feature vectors corresponding to the plurality of preset detection parts and the feature vector of the target ultrasound image; The preset detection part corresponding to the feature vector with the highest similarity is used as the target detection part corresponding to the target ultrasound image.

9. A method for generating an ultrasonic examination report, characterized in that: The method is applied to an ultrasonic device, comprising: Detecting a target detection part corresponding to the acquired target ultrasound image; Based on the target detection part, screening out at least one reference ultrasound examination report corresponding to the target detection part from a set of candidate ultrasound examination reports; Determine a first measurement parameter of a target region where the target detection part is located in the target ultrasound image, and extract a first ROI image from the target ultrasound image; determining a target reference ultrasound examination report from the at least one reference ultrasound examination report based on the first measurement parameter and the first ROI image; An ultrasound examination report corresponding to the target ultrasound image is generated based on the first measurement parameter and the first ROI image, as well as the target reference ultrasound examination report.

10. An ultrasonic examination report generating device, characterized in that: The device comprises: An acquisition module, used for acquiring a target ultrasonic image; A detection part determination module is used to detect a target detection part corresponding to the collected target ultrasound image; A first screening module, configured to screen out at least one reference ultrasound examination report corresponding to the target detection part from a set of candidate ultrasound examination reports based on the target detection part; An information extraction module, used to determine a first measurement parameter of a target region where the target detection part is located in the target ultrasound image, and to extract a first ROI image from the target ultrasound image; a second screening module, configured to determine a target reference ultrasound examination report from the at least one reference ultrasound examination report based on the first measurement parameter and the first ROI image; A generating module is used to generate an ultrasound examination report corresponding to the target ultrasound image based on the first measurement parameter and the first ROI image, and the target reference ultrasound examination report.

Images