Pulmonary effusion ultrasonic image classification method, medical equipment and system

By identifying and analyzing the characteristics of pulmonary fluid in multi-frame ultrasound images, generating segmentation masks and calculating physical dimensions, the problem of automated analysis of lack of professional doctors in outdoor rescue is solved, and the rapid and accurate classification and severity of pulmonary fluid is achieved. Evaluate.

CN119048792APending Publication Date: 2024-11-29THE FIRST MEDICAL CENT CHINESE PLA GENERAL HOSPITAL
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410452408.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-04-16
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

In outdoor or wartime rescue situations, there is a lack of professional ultrasound doctors, and existing ultrasound equipment cannot automatically and quickly analyze and judge the pulmonary fluid and its severity, making it difficult for rescue personnel to take effective measures.

Method used

By acquiring continuous multi-frame ultrasound images, identifying the characteristics of the pulmonary fluid, generating a segmentation mask for the pulmonary fluid region, calculating its physical size, and outputting classification results based on the threshold, and using neural networks and three-axis robotic arms to achieve automated analysis.

Benefits of technology

The rapid and automatic classification of pulmonary effusion is achieved, the accuracy and speed of analysis is improved, and manual participation is required, making it easier for rescue personnel to evaluate and take corresponding measures.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048792A_ABST
    Figure CN119048792A_ABST
Patent Text Reader

Abstract

The invention discloses a pulmonary effusion ultrasonic image classification method, medical equipment and system, the medical equipment comprises an ultrasonic device, the ultrasonic device is used for generating ultrasonic images through scanning, and the method comprises the steps that continuous multiple frames of ultrasonic images are acquired, and whether each frame of ultrasonic image comprises pulmonary effusion features or not is recognized; when the pulmonary effusion features are identified, generating a segmentation mask of a pulmonary effusion region of the corresponding frame of the pulmonary trauma image; obtaining the physical size of the pulmonary effusion through the pixel size of the pulmonary effusion region in the multi-frame segmentation mask; when the physical size exceeds the threshold value, the corresponding classification result is output, rapid and automatic classification of the ultrasonic image containing the pulmonary effusion is achieved, and meanwhile the corresponding classification result is obtained according to the pulmonary effusion of different severity degrees.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of medical equipment, and in particular to a classification method for ultrasonic images of pulmonary effusion, medical equipment and a system. Background Art

[0002] When a person's lungs are injured and pulmonary effusion occurs, it is usually very urgent. The larger the area or volume of the pulmonary effusion, the more serious the situation is. Most pulmonary effusions need to be diagnosed in the hospital by an ultrasound doctor using ultrasound equipment.

[0003] Existing ultrasound equipment only has the function of scanning and generating ultrasound images. When a large number of people are found injured in remote outdoor places such as the wild, disaster relief, or wartime battlefields, even if the rescuers carry ultrasound equipment, there are usually no professional ultrasound doctors nearby. They cannot automatically and quickly analyze the ultrasound images to determine whether the injured have pulmonary effusion and the severity of the pulmonary effusion, thereby facilitating the rescuers to take further rescue measures. Summary of the invention

[0004] Provided are a lung effusion ultrasound image classification, medical equipment and system to solve the problem that ultrasound images cannot be automatically analyzed and quickly determined whether a wounded person has lung effusion and the severity of the lung effusion during outdoor or wartime rescue.

[0005] In order to solve the above technical problems, this application proposes the following technical solutions:

[0006] An embodiment of the present application provides a method for classifying ultrasound images of lung effusion, which is applied to medical equipment. The medical equipment includes an ultrasound device, and the ultrasound device is used to generate ultrasound images by scanning. The method includes: acquiring multiple frames of continuous ultrasound images, and identifying whether each frame of the ultrasound image includes a lung effusion feature; when the lung effusion feature is identified, generating a segmentation mask of a corresponding frame of lung trauma image containing a lung effusion area; obtaining the physical size of the lung effusion through the pixel size of the lung effusion area in the multi-frame segmentation mask; and outputting a corresponding classification result when the physical size exceeds a threshold.

[0007] Preferably, the step of acquiring a continuous plurality of frames of ultrasonic detection images and identifying whether each frame of lung trauma image includes a lung effusion feature comprises: pre-collecting a plurality of lung trauma data sets, and pre-processing the pre-collected lung trauma data sets, wherein the data sets include a plurality of frames of images, and each frame of the image includes a category and an instance to generate a data fingerprint; correspondingly, when the lung effusion feature is identified, generating a segmentation mask of the corresponding frame of lung trauma image containing a lung effusion area comprises: retaining a connected domain with the largest size to generate the segmentation mask.

[0008] Preferably, when the lung effusion feature is identified, generating a segmentation mask of the corresponding frame of lung injury image containing the lung effusion area comprises:

[0009] The acquired multiple frames of ultrasound images are used as original image inputs. After preprocessing, the first neural network is used to perform step-by-step convolution and pooling down-sampling to locate the effusion feature area, and the second neural network is used to extract the global information of the effusion feature to obtain a low-resolution effusion feature; the first neural network is used to perform step-by-step deconvolution up-sampling to map the low-resolution effusion feature into a high-resolution segmentation mask.

[0010] Preferably, obtaining the physical size of the pulmonary effusion by using the pixel size of the pulmonary effusion area in the multi-frame segmentation mask includes: stacking the boundary points of the effusion area in multiple consecutive frames of two-dimensional ultrasound detection images to obtain a three-dimensional point cloud data set; fitting the three-dimensional point cloud data set by surface fitting to obtain a three-dimensional surface equation of the corresponding effusion surface; calculating the three-dimensional surface equation by multiple integrals to obtain the pixel volume of the corresponding pulmonary effusion area; obtaining the total pixel volume of the pulmonary effusion area according to the pixel volume of the pulmonary effusion area, and obtaining the first physical volume of the pulmonary effusion by the original resolution information of the ultrasound detection image.

[0011] Preferably, the process of fitting each three-dimensional point cloud data set by surface fitting to obtain the corresponding three-dimensional surface data equation of the effusion surface includes: obtaining surface fitting parameters by using a two-variable quadratic surface formula and a least squares method to obtain the three-dimensional surface equation of the effusion surface; wherein, after converting the three-dimensional point cloud data format into an array format, generating a corresponding coefficient matrix and a target vector according to the three-dimensional point cloud data set, and minimizing the fitting error to obtain the surface fitting parameters.

[0012] Preferably, obtaining the physical size of the lung effusion by using the pixel size of the lung effusion area in the multi-frame segmentation mask also includes: when a preset position outside the effusion area in the segmentation mask includes a preset number of preset pixel values, determining the preset position as a boundary point of the effusion area.

[0013] Preferably, the method further comprises: generating a score value of the trauma level of multiple frames of ultrasound images through a third neural network; when the score value is greater than a preset value, identifying whether each frame of the lung trauma image includes a lung effusion feature.

[0014] An embodiment of the present application also provides a medical device, including: an ultrasound device, which is used to generate an ultrasound detection image by scanning; and a first processing unit, which is used to execute the above-mentioned classification method of the lung effusion ultrasound image.

[0015] Preferably, the medical device also includes: a three-axis robotic arm, through which the ultrasonic device moves and scans to generate an ultrasonic detection image, and the three-axis robotic arm is also used to generate three-dimensional posture coordinates in real time; a second processing unit, used to execute the above-mentioned classification method of the ultrasonic image of pulmonary effusion through a central processing unit; the first processing unit is also used to execute the above-mentioned classification method of the ultrasonic image of pulmonary effusion through a graphics processor.

[0016] An embodiment of the present application also provides a classification system for ultrasonic images of pulmonary effusion, comprising: a medical device according to the above; a communication device; a remote device, which is communicatively connected to the medical device through the communication device, the remote device comprising an operating device and a display device, the operating device being used to control the movement and scanning of the three-axis robotic arm through the communication device to generate an ultrasonic detection image, and the display device being used to receive and display the classification result through the communication device.

[0017] The embodiments of the present application provide a lung effusion ultrasound image classification method, medical equipment, and a lung effusion ultrasound image classification system, which have at least the following beneficial effects:

[0018] The classification, medical equipment and system of ultrasound images of pulmonary effusion provided in the embodiments of the present application perform two-dimensional recognition on ultrasound images of pulmonary effusion to determine whether there is pulmonary effusion, generate a segmentation mask corresponding to the area of ​​pulmonary effusion, then obtain the pixel size of the area of ​​pulmonary effusion in the segmentation mask, calculate the pixel size of the segmentation mask generated in the image recognition and segmentation process to obtain the physical size of the pulmonary effusion, and output the corresponding classification result when the threshold is exceeded, thereby realizing automatic classification of ultrasound images of pulmonary effusion without human intervention. Meanwhile, compared with three-dimensional recognition, two-dimensional recognition has higher accuracy for ultrasound images. Through image recognition The pixel size of the segmentation mask generated during the segmentation process is calculated to obtain the physical size of the pulmonary effusion. There is no need to extract the mask or crop the ultrasound image through the segmentation mask, and then evaluate the size of the pulmonary effusion by obtaining the final generated segmented image, which speeds up the classification of the pulmonary effusion in the ultrasound image. Therefore, the present application realizes the rapid and automatic classification of ultrasound images containing pulmonary effusion, and at the same time obtains the corresponding classification results according to the different severity of pulmonary effusion, and also realizes the classification of patients with different severity of pulmonary effusion, so as to facilitate rescue personnel to take further rescue measures as an evaluation basis when they are outdoors or standing. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0020] Figure 1 A schematic diagram of a method for classifying ultrasound images of pulmonary effusion provided in accordance with an embodiment of the present application;

[0021] Figure 2 A schematic diagram of a process for generating a segmentation mask of a lung effusion region of a lung injury image corresponding to a frame provided by an embodiment of the present application;

[0022] Figure 3 A schematic diagram of a process for obtaining the physical size of pulmonary effusion by using the pixel size of the pulmonary effusion region in a multi-frame segmentation mask provided in an embodiment of the present application Figure 1 ;

[0023] Figure 4 A schematic diagram of a coefficient matrix A and a target vector b provided for one embodiment of the present application;

[0024] Figure 5 A schematic diagram of the structure of a medical device provided for one embodiment of the present application;

[0025] Figure 6 A schematic diagram of a process for obtaining the physical size of pulmonary effusion by using the pixel size of the pulmonary effusion region in a multi-frame segmentation mask provided in an embodiment of the present application Figure 2 ;

[0026] Figure 7 A schematic diagram of a process for generating a trauma grade scoring value for multiple frames of ultrasound images provided in one embodiment of the present application;

[0027] Figure 8 A schematic diagram of a lung ultrasound scoring standard provided for one embodiment of the present application;

[0028] Fig. 9 A schematic diagram of the structure of a remote device provided in one embodiment of the present application;

[0029] Fig.10 A schematic diagram of a classification system for ultrasound images of lung effusion provided in accordance with an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0031] It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than in the flowchart. Terms in the specification, claims and the above drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0032] In the description of this application, if there is a description of first or second, it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0033] In the description of this application, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in this application based on the specific content of the technical solution.

[0034] When a patient's lungs are injured and pulmonary effusion occurs, it is usually very urgent. The larger the area or volume of the pulmonary effusion, the more serious the situation is. Most pulmonary effusions need to be diagnosed in the hospital by an ultrasound doctor using ultrasound equipment.

[0035] Existing ultrasound equipment only has the function of scanning and generating ultrasound images. When people are found injured in remote outdoor places such as the wild, disaster relief, etc., even if ultrasound equipment is carried, there are usually no professional ultrasound doctors nearby. It is difficult to analyze the ultrasound image to determine whether pulmonary effusion is present and the severity of the pulmonary effusion, so as to take further rescue measures.

[0036] Based on this, please refer to Figure 1 Embodiment 1 of the present application provides a classification method for ultrasonic images of pulmonary effusion, which is applied to a medical device, wherein the medical device comprises an ultrasonic device, and the ultrasonic device is used to generate an ultrasonic image by scanning. The method comprises the following steps:

[0037] Step S200: Acquire multiple continuous ultrasound image frames, and identify whether each ultrasound image frame includes a lung effusion feature.

[0038] Ultrasonic imaging is achieved by emitting and receiving ultrasonic waves. Since different human tissues have different impedances to sound waves, different degrees of transmission and reflection will occur at the junction of different tissues. By analyzing the reflected signals, the tissue structure information in the body can be obtained.

[0039] Compared with other medical images such as X-rays and CT, ultrasound imaging has poor image quality and low signal-to-noise ratio, but has high temporal resolution and can generate continuous multi-frame images in real time. Therefore, in this embodiment, continuous multi-frame ultrasound images are obtained to identify whether each frame of the ultrasound image includes lung effusion features.

[0040] Specifically, in the present embodiment, a plurality of continuous ultrasound image frames are first acquired, and then an image recognition model is used to identify whether each ultrasound image frame includes features of pulmonary effusion. That is, due to the low signal-to-noise ratio of ultrasound images, the present embodiment uses a 2D image recognition model to identify whether ultrasound images contain pulmonary effusion frame by frame to improve recognition accuracy, rather than directly using a 3D image recognition model to directly identify whether pulmonary effusion is contained. This is because the signal-to-noise ratio of ultrasound images is very low, and 3D recognition is also difficult to extract deep image features. When performing image recognition on ultrasound images, the accuracy of 3D recognition is poor.

[0041] In this embodiment, target detection is used to identify and locate different objects in each frame of ultrasound image, and their positions and categories are marked with bounding boxes.

[0042] Specifically, a neural network is used to extract features of pulmonary effusion in each frame of ultrasound image, and the features of pulmonary effusion are displayed as different textures, shapes, colors, and other information in each frame of image.

[0043] Step S300: When the lung effusion feature is identified, a segmentation mask of the lung effusion region of the corresponding frame of lung trauma image is generated.

[0044] After obtaining the result of target detection corresponding to each frame of ultrasound image, segmentation masks corresponding to multiple frames can be generated. For example, a segmentation mask with all zeros is first used to cover all areas in each frame of ultrasound detection image, and then the pixel value of the area corresponding to the lung effusion in the segmentation mask is set to 1 according to the position and size of the border of the lung effusion area, so as to obtain a segmentation mask containing the lung effusion area.

[0045] In this embodiment, the lung effusion in the ultrasound image is separated to generate a corresponding binary image to represent the position and shape of the lung effusion. In other embodiments, a corresponding multi-value image can also be generated. For example, the pixel value of the area corresponding to the lung effusion in the segmentation mask is set to a preset value according to the position and size of the border of the lung effusion area.

[0046] Step S400: obtaining the physical size of the pulmonary effusion by the pixel size of the pulmonary effusion region in the multi-frame segmentation mask.

[0047] After obtaining the multi-frame segmentation mask corresponding to the ultrasonic detection image, the physical size of the lung effusion is obtained according to the pixel size of the lung effusion area in the multi-frame segmentation mask, that is, the pixel size of the lung effusion area in the multi-frame two-dimensional segmentation mask is converted into the physical size of the three-dimensional lung effusion, for example, the area or number of pixels of the area with a pixel value of 1 or a preset value in each frame of the segmentation mask is obtained.

[0048] The severity of pulmonary effusion is closely related to the amount of effusion. After obtaining the area or number of pixels in the region with a pixel value of 1 or a preset value in each frame of the segmentation mask, the relative size of the effusion in the pulmonary effusion can be characterized. Because the segmentation mask and the ultrasound image have the same pixel size, when the pixel area of ​​the pulmonary effusion region in each frame of the segmentation mask is obtained, the actual area of ​​the corresponding pulmonary effusion region, that is, the physical size, can be obtained based on the ultrasound image resolution information.

[0049] For example, if the size of the lung effusion region in the segmentation mask is 640×480 pixels, and the resolution of the ultrasound image is 180 ppi (pixels per inch), its actual size corresponds to: length: 3.556 inches, width: 2.667 inches.

[0050] Step S500: When the physical size exceeds a threshold, the corresponding classification result is output.

[0051] Finally, according to the output values ​​of the physical sizes of the multiple frames of pulmonary effusion, according to the evaluation standards of pulmonary effusion of different severity, multiple evaluation results are divided, such as mild, standard, severe, dangerous, etc., and then the corresponding evaluation results are arranged and matched from low to high according to the obtained physical size of the pulmonary effusion, and the corresponding classification results are automatically output after the output physical size of the pulmonary effusion is obtained. For example, when it is detected that the amount of pulmonary effusion is greater than the upper limit of different thresholds, the corresponding severity level is generated, which serves as an evaluation basis for taking further rescue measures, or automatically triggers the corresponding warning level or directly takes corresponding help measures.

[0052] The classification method of the ultrasound image of pulmonary effusion of the present embodiment performs two-dimensional recognition on the ultrasound image of pulmonary effusion to determine whether there is pulmonary effusion, and generates a segmentation mask corresponding to the pulmonary effusion area, then obtains the pixel size of the pulmonary effusion area in the segmentation mask, calculates the pixel size through the segmentation mask generated in the image recognition and segmentation process to obtain the physical size of the pulmonary effusion, and outputs the corresponding classification result when the threshold is exceeded, and the automatic classification of the ultrasound image pulmonary effusion is realized without manual participation. At the same time, compared with three-dimensional recognition, two-dimensional recognition has higher accuracy for ultrasound images. The pixel size is calculated through the segmentation mask generated in the image recognition and segmentation process to obtain the physical size of the pulmonary effusion, and there is no need to extract or crop the ultrasound image through the segmentation mask, and then evaluate the size of the pulmonary effusion by obtaining the finally generated segmented image, which speeds up the classification speed of the pulmonary effusion in the ultrasound image. Therefore, the present application realizes the rapid automatic classification of the ultrasound image containing pulmonary effusion, and at the same time obtains the corresponding classification results according to the pulmonary effusion of different severities, and also realizes the classification of patients with pulmonary effusion of different severities, so as to be convenient as an evaluation basis for taking further rescue measures.

[0053] In this embodiment, step S200: acquiring a continuous plurality of frames of ultrasound images, and identifying whether each frame of ultrasound image includes a lung effusion feature includes: pre-collecting a plurality of lung trauma data sets, and pre-processing the pre-collected lung trauma data sets, wherein the data sets include a plurality of frames of images, and each frame of the image includes a category and an instance to generate a data fingerprint.

[0054] Correspondingly, step S300: when the lung effusion feature is identified, generating a segmentation mask of the lung effusion region of the corresponding frame of the lung trauma image includes: retaining the largest connected domain to generate the segmentation mask.

[0055] In this embodiment, a plurality of lung injury data sets are pre-collected, wherein the lung injury data sets include a plurality of lung effusion sample features and other lung injury features.

[0056] Specifically, during the data collection process, the balance of factors such as gender, age, and case type was considered to show the male-female ratio in the dataset, the distribution of different age groups, and the distribution of samples with different trauma grades and effusion conditions.

[0057] Among them, regarding the gender ratio: the ratio of men to women in the dataset is 1:1, ensuring a balanced distribution of genders. Regarding the age distribution: the dataset covers patients of all ages from 26 to 70 years old, ensuring that the model has sufficient learning ability for data from different age groups.

[0058] After collecting enough sample data, the sample data is preprocessed. Specifically, the pre-collected lung trauma data set is preprocessed to generate a data fingerprint, and the driving parameters are adaptively changed according to the data set fingerprint to obtain multiple two-dimensional lung effusion recognition models, wherein the lung trauma data set includes multiple frames of images, each frame of the image contains a category and an instance to generate a data fingerprint.

[0059] Among them, the dataset attributes mainly include: image size, number of images, label categories and their number, the proportion of instances in each category, the pixel ratio and modality of each category, etc.

[0060] Specifically, in the current dataset, there are 5535 images in total, with two sizes, 1920×1080 and 1024×768 RGB images. Each image contains only one category, pleural effusion, and the foreground pixels of this category occupy 10% of the entire image. These values ​​constitute the dataset fingerprint.

[0061] Then, the inference parameters are adaptively changed according to the dataset fingerprint. The inference parameters include: image block size, number of downsampling layers of the neural network encoder, number of samples in the training dataset, normalization method, etc.

[0062] Finally, by training multiple recognition models, the prediction results of all models are integrated and saved, and the optimal recognition model is selected through empirical parameters, and it is decided whether to retain the maximum connected domain.

[0063] In this embodiment, each image in the data set has only one instance, so only the largest connected domain is retained in the segmentation mask generated for the corresponding pulmonary effusion area, thereby speeding up the pulmonary effusion recognition and subsequent pulmonary effusion volume calculation.

[0064] Among them, the connected domain generally refers to the image area composed of pixels with the same pixel value and adjacent positions in the image. The regional position corresponding to the pulmonary effusion is obtained by finding and marking each connected area in the segmentation mask image.

[0065] Please refer to Figure 2 In this embodiment, step S300: when the lung effusion feature is identified, generating a segmentation mask of the lung effusion region of the corresponding frame of the lung trauma image includes:

[0066] Step S320: The acquired multiple frames of ultrasound images are used as original image inputs, and after preprocessing, they are down-sampled by convolution pooling step by step through a first neural network to locate the effusion feature area, and the global information of the effusion feature is extracted through a second neural network to obtain a low-resolution effusion feature.

[0067] Step S340: performing step-by-step deconvolution upsampling through the first neural network to map low-resolution effusion features into high-resolution segmentation masks.

[0068] After identifying the segmented area of ​​the feature, image segmentation is an important step in medical image analysis. Unlike ordinary image segmentation, medical images, especially ultrasound images, usually contain more noise and have blurred boundaries. Therefore, it is difficult to detect and identify the effusion area in the ultrasound image by relying solely on the underlying features of the image. At the same time, due to the lack of image detail information, accurate boundaries cannot be obtained by relying solely on image semantic features.

[0069] In existing medical image segmentation, recognition and segmentation are mainly performed through a U-shaped architecture, such as the U-Net convolutional neural network structure. The encoder part of U-Net uses pooling layers to downsample step by step, and the decoder part uses deconvolution to upsample step by step. The spatial information in the original input image and the edge information in the image will be gradually restored. As a result, the low-resolution feature map will eventually be mapped to a pixel-level segmentation result map.

[0070] Among them, the convolution of the image is actually a process of extracting features from the image using the convolution kernel. The convolution kernel is a matrix. By designing this matrix, such as the size of the matrix and the values ​​in the matrix, the corresponding image features can be extracted, such as the edge features and texture features of the image. Deconvolution, or upsampling, increases the resolution of the original image by enlarging it.

[0071] However, U-Net cannot model the contextual relationships of distant features, especially for the features of lung effusion in ultrasound images. The texture, shape and size may vary greatly among different patients. The performance is weak when U-Net is used for recognition, and it is necessary to model the global information and contextual relationships.

[0072] In this embodiment, the acquired multi-frame ultrasound images are used as original image inputs, and after preprocessing, the first neural network is used for step-by-step convolution and pooling downsampling to locate the effusion feature area, and the global information of the effusion feature is extracted by the second neural network to obtain low-resolution effusion features.

[0073] The first neural network is a U-Net neural network, and the second neural network is a neural network structure that can extract global information, such as a Transformer neural network. After data preprocessing, features are extracted in parallel through U-Net and Transformer. After extraction, these features are merged, and then upsampled through U-Net step by step deconvolution to map low-resolution effusion features to high-resolution segmentation masks. This enables accurate recognition and segmentation of ultrasonic pulmonary effusion features.

[0074] Specifically, in this embodiment, a lung ultrasound image is first input as the original image, and the feature encoder is superimposed with multiple convolutional layers and activation layers to extract more abstract and higher-level semantic features from the input image step by step. These learned feature maps are input into the decoder, and the parameters of the upper convolutional layer are adjusted according to the feedback result of the decoder. The iteration is performed until the loss function converges, and then the size of the feature map is scaled to the same size as the original image, and the model parameters are output to obtain a lung effusion localization model. Finally, a segmentation mask corresponding to the lung effusion area is generated by the lung effusion localization model.

[0075] Please refer to Figure 3 In this embodiment, step S400: obtaining the physical size of the pulmonary effusion by the pixel size of the pulmonary effusion region in the multi-frame segmentation mask includes:

[0076] Step S420: stacking the boundary points of the lung effusion region of multiple consecutive frames of two-dimensional segmentation masks to obtain a three-dimensional point cloud data set.

[0077] Step S440: fitting the three-dimensional point cloud data set by surface fitting to obtain a three-dimensional surface equation corresponding to the surface of the pulmonary effusion pixels.

[0078] Step S460: Calculate the three-dimensional surface equation by multiple integration to obtain the pixel volume corresponding to the lung effusion area.

[0079] Step S480: Obtaining a first physical volume of the pulmonary effusion according to the pixel volume of the pulmonary effusion region and the original resolution information of the ultrasound image.

[0080] In this embodiment, after obtaining the two-dimensional lung effusion area of ​​the multi-frame segmentation mask, the two-dimensional pixel size in the multi-frame segmentation mask is mathematically fitted into a three-dimensional pixel size. Since the ultrasound image is generated in real time, after the lung effusion is fitted into three dimensions, the real physical volume of the lung effusion can be more accurately represented, and the error between the calculated value and the actual value can be reduced.

[0081] Specifically, firstly, the boundary points of the effusion area in multiple consecutive frames of two-dimensional ultrasound images are stacked to obtain a three-dimensional point cloud data set.

[0082] Among them, the point cloud data is the geometric shape information of the effusion surface represented by discrete three-dimensional points, and the surface fitting is to approximate these discrete point cloud data through a three-dimensional fitting model to obtain the effusion surface with the smallest error and the most accurate shape.

[0083] The volume of the effusion area was calculated using the numerical integration method using the obtained three-dimensional fitting model.

[0084] Specifically, according to the obtained three-dimensional surface model, the range of integration is determined, and then the integral equation is set. For a three-dimensional surface z=f(x, y, z), its volume V can be calculated by the triple integral formula. The specific form of the integral equation depends on the surface equation and shape.

[0085] In this embodiment, the total physical volume of the pulmonary effusion can be obtained by summing up the physical volume of each pulmonary effusion obtained based on the pixel volume of each pulmonary effusion area obtained and the original resolution information of the ultrasound image. Alternatively, the pixel volumes corresponding to the obtained multiple three-dimensional surface data can be summed to obtain the total pixel volume of the pulmonary effusion area, and then the total physical volume of the pulmonary effusion can be obtained by the original resolution information of the ultrasound detection image.

[0086] In this embodiment, since there may be multiple effusion areas, that is, the ultrasound detection image may include effusion areas in multiple locations, according to the requirements of different lung effusion assessment standards, some effusion areas with smaller volumes are ignored, and only the effusion area with the largest volume is selected to quickly obtain the corresponding lung effusion volume result.

[0087] In other embodiments, a numerical integration method may also be used in processing each frame of two-dimensional ultrasonic detection images to first discretize each effusion area, calculate the volume of each effusion area, and then sum up to obtain the total volume of pixels under each surface.

[0088] Finally, the actual physical volume of the effusion is calculated based on the relationship between the physical size of the image and the pixel size, that is, the original resolution information of the ultrasound detection image.

[0089] In this embodiment, step S430: fitting the three-dimensional point cloud data set by surface fitting to obtain the three-dimensional surface equation of the corresponding pulmonary effusion pixel surface includes:

[0090] The surface fitting parameters are obtained by solving the binary quadratic surface formula and the least squares method, thereby obtaining the three-dimensional surface equation of the effusion surface. After the format of the three-dimensional point cloud data set is converted into an array format, the corresponding coefficient matrix and target vector are generated, and the fitting error is minimized to obtain the surface fitting parameters.

[0091] In this embodiment, for a given set of point cloud data sets containing N three-dimensional points, the goal is to obtain a surface model f(x, y) to optimally fit these point cloud data, where x and y are coordinates on the plane and f(x, y) is the corresponding height.

[0092] In this embodiment, according to the characteristics of the ultrasound data and the measurement requirements, a polynomial surface is used to fit the point cloud data to obtain a more accurate surface shape of the effusion.

[0093] The 3D point cloud data format is (x, y, z). First, convert the point set into an array format, and then select the polynomial fitting method, such as applying the fitting coefficient to a two-dimensional quadratic surface fitting model: ax 2 +bxy+cy 2 +dx+ey+f=0 to obtain the fitting surface Z.

[0094] Specifically, the surface fitting problem is established as a least squares problem. The equation is: f(x,y) = a0 + a1x + a2y + a3x 2 +a4xy+a5y 2 +…, we need to find a0, a1, a2,… so that the surface minimizes the distance from the point to the surface.

[0095] Among them, by constructing the coefficient matrix and target vector, the three-dimensional point cloud data is brought into the fitting problem to obtain the coefficient matrix and target vector. For the point (xi, yi, zi), the coefficient matrix A and the target vector b are constructed, see Figure 4 .

[0096] Then, the parameter vector x is determined by solving the linear equation system Ax=b, and the least square method is used to solve the equation system to obtain the parameters a0, a1, a2, ... of the surface, and finally the fitting model is obtained.

[0097] Among them, each row in the coefficient matrix A corresponds to the coordinate information of a point, each column corresponds to a parameter of the surface fitting model, and each element of the target vector b corresponds to a height value of the point cloud dataset.

[0098] In this embodiment, step S400: obtaining the physical size of the pulmonary effusion by the pixel size of the pulmonary effusion region in the multi-frame segmentation mask further includes:

[0099] When the pixel value of the preset position outside the lung effusion region in the segmentation mask is a preset value, the preset position is determined to be a boundary point corresponding to the lung effusion region.

[0100] Specifically, in ultrasound images of pulmonary effusion, relatively bright echoes are displayed behind the pulmonary effusion, so in image segmentation and recognition, rear echo enhancement can be used as an important feature to assist in more accurately identifying and segmenting structures in images of pulmonary effusion.

[0101] In this embodiment, in the automatic image segmentation task, the posterior echo enhancement of the region of lung effusion can use the posterior echo enhancement as a boundary marker to help determine the edge of the effusion region, thereby improving the accuracy of segmentation.

[0102] Among them, the relatively bright echo displayed behind the pulmonary effusion corresponds to a pixel value of a preset position outside the pulmonary effusion area in the segmentation mask as a preset value. When the pixel value of the preset position outside the pulmonary effusion area detected in the segmentation mask is a preset value, or the number of pixel values ​​of the preset position outside the pulmonary effusion area detected in the segmentation mask as the preset value exceeds a preset number, the preset position is determined to be a boundary point corresponding to the pulmonary effusion area, thereby facilitating the recognition and segmentation of the pulmonary effusion area in the ultrasound image, and calculating the volume by three-dimensional fitting.

[0103] In another embodiment, during data collection and annotation, the region with enhanced echoes behind the pulmonary effusion is used as a label and annotated to facilitate distinguishing different anatomical structures and pathological conditions in automatic image analysis to determine whether it is pulmonary effusion.

[0104] In another embodiment, the effusion area in the image may be made more prominent by identifying the posterior echo enhancement areas and adjusting the contrast of these areas.

[0105] In another embodiment, the knowledge of rear echo enhancement can be used to enhance the data set when training the two-dimensional pulmonary effusion recognition model, for example, by simulating this feature to create more training samples, thereby improving the generalization ability of the pulmonary effusion model.

[0106] Please refer to Figure 5 and Figure 6 In this embodiment, the medical device includes a three-axis robotic arm, and the ultrasound device moves and scans through the three-axis robotic arm to generate an ultrasound detection image. The three-axis robotic arm is also used to generate three-dimensional posture coordinates in real time.

[0107] Step S400: Obtaining the physical size of the pulmonary effusion by using the pixel size of the pulmonary effusion region in the multi-frame segmentation mask includes:

[0108] Step S410: Calculate the number of pixels whose pixel values ​​are preset values ​​in each frame segmentation mask, wherein when the change in the three-dimensional pose coordinates generated by the three-axis robotic arm is within a preset range, stop calculating the number of pixels whose pixel values ​​are preset values ​​in the corresponding frame segmentation mask.

[0109] Step S430: summing the number of pixels whose pixel values ​​are preset values ​​in the segmentation masks of the consecutive multiple frames to obtain the total number of pixels of the lung effusion target.

[0110] Step S450: obtaining a second physical volume of the pulmonary effusion according to the total number of pixels of the pulmonary effusion target and the original resolution information of the ultrasound image.

[0111] Correspondingly, step S500: when the physical size exceeds the threshold, outputting the corresponding classification result includes:

[0112] An interval range of the first physical volume and / or the second physical volume is defined to output a corresponding classification result.

[0113] Specifically, compared with mathematically fitting the two-dimensional pixel size in the multi-frame segmentation mask to a three-dimensional pixel size, directly calculating the number of corresponding pulmonary effusion pixels in the multi-frame segmentation mask to characterize the volume of pulmonary effusion can simplify the calculation process and speed up the output of the output classification structure. However, by calculating the pixel value, due to the lack of three-dimensional fitting optimization of repeated frames, it is easy to cause inaccurate results. For example, when operating an ultrasound device, a pause during scanning will result in multiple frames of repeated two-dimensional ultrasound images at the same detection position, and when the overall pixel value is counted, a result larger than the actual pulmonary effusion volume will be obtained.

[0114] In this embodiment, an ultrasonic image is indirectly scanned and generated by operating an ultrasonic device through a three-axis robotic arm instead of directly operating the ultrasonic device. At the same time, the three-axis robotic arm returns the current posture coordinates in real time to determine whether the scanning is paused. When the posture coordinates returned by the three-axis robotic arm do not change or change very little, it is determined that the scanning is paused, and the calculation of the pixel values ​​is stopped, or the ultrasonic image generated in the previous frame is directly deleted, and only the currently generated ultrasonic image is saved, and the subsequent recognition and segmentation counting steps are continued after the posture coordinates change.

[0115] Therefore, in this embodiment, two physical volumes of lung effusion can be obtained successively, namely, a more accurate first physical volume and a faster second physical volume, and the accuracy of the calculation results can be verified by each other through the two physical volume results. In other embodiments, only one calculation process can be performed to obtain the corresponding physical volume according to the needs of the actual scenario, so as to obtain a more accurate or faster lung effusion volume result to facilitate subsequent classification.

[0116] In another embodiment, a one-to-one correspondence can be established between the three-axis robotic arm posture coordinate information and the current frame ultrasound image. When the same posture coordinate corresponds to multiple frames of ultrasound images, only the most recently generated ultrasound image is retained to further improve the counting accuracy.

[0117] At the same time, the three-axis robotic arm process ultrasound scanning can also realize remote operation by ultrasound doctors at a remote end to solve the problem of no professional personnel operating the ultrasound device outdoors.

[0118] Please refer to Figure 7 In this embodiment, in step S200, before identifying whether each frame of ultrasound image includes a lung effusion feature, the following steps are also included:

[0119] Step S120: Generate trauma grade scores for multiple frames of ultrasound images through a third neural network.

[0120] Step S140: When the score value is greater than a preset value, identifying whether each frame of the ultrasound image includes a lung effusion feature.

[0121] In this embodiment, before identifying and segmenting whether the ultrasound image includes lung effusion, a step of evaluating the lung injury level is also included.

[0122] Specifically, the third network model performs low-dimensional feature evaluation on the preprocessed ultrasound image data to obtain a trauma level score. When the score is greater than a preset value, that is, the lung trauma is severe and the possibility of pulmonary effusion is greater, it is then identified whether each frame of the lung trauma image includes pulmonary effusion features to improve the classification efficiency of the system.

[0123] Specifically, according to different trauma situations, the lung ultrasound images have different manifestations of the pleural line, A line and B line. An extractor for the above ultrasound features is established, and the third network model is used to evaluate the lung trauma level for these three attributes. Deep high-dimensional features are extracted from the image by superimposing convolutional layers, pooling layers, etc. After the features are extracted, a linear classifier is added to classify the image to complete the classification task. The output of the model is the predicted value of the trauma level, which is 0, 1, 2, and 3 points respectively.

[0124] Among them, the evaluation standard is LUS (lung ultrasound scoring standard), see Figure 8 .

[0125] Specifically, the third network model is a lightweight neural network structure, such as a MobileNet neural network, which focuses on identifying low-dimensional features, such as pleural lines, A lines, and B lines. It reduces the amount of calculation by setting a deep separable convolution structure, and sets hyperparameters to adjust the size of the model to increase the calculation speed for rapid pre-evaluation. When the score value is greater than a preset value, such as 2 or 3, it identifies whether each frame of the lung trauma image includes lung effusion features. When the score value is not greater than the preset value, the trauma score value is directly output as a separate category to end the classification.

[0126] Please refer to Figure 5 Another embodiment of the present application provides a medical device 10, comprising:

[0127] The ultrasound device 110 is used to generate ultrasound images by scanning.

[0128] The first processing unit 130 is used to execute the above steps S200-S500.

[0129] Specifically, in this embodiment, the medical device 10 further includes:

[0130] The three-axis robotic arm 120 , through which the ultrasonic device 110 moves and scans to generate ultrasonic detection images, and the three-axis robotic arm 120 is also used to generate three-dimensional posture coordinates in real time.

[0131] The second processing unit (not shown) is used to execute steps S120-S140 via a central processing unit.

[0132] The first processing unit 130 is further configured to execute steps S200-S500 through a graphics processor.

[0133] Specifically, CPU (central processing unit) and GPU (Graphics Processing Unit) are both processors for computing in computers, and both have three parts in terms of architecture: computing unit ALU, control unit Control and cache unit Cache. Among them, the cache unit accounts for about 50% in CPU, the control unit 25%, and the computing unit 25%; the cache unit accounts for about 5% in GPU, the control unit 5%, and the computing unit 90%.

[0134] The CPU's computing power is more balanced, but it is not suitable for large-scale computing. The GPU is more suitable for large-scale computing.

[0135] In this embodiment, since the amount of data for extracting low-dimensional trauma features is small, in order to facilitate quick execution and further determine whether there is a lung effusion feature, the step of extracting low-dimensional trauma features is first performed by the CPU, and high-dimensional features need to be extracted through multiple neural network models to identify whether there is lung effusion. The amount of data is large and the accuracy of the results is required to be higher. Therefore, it is executed by the GPU. On the other hand, the processors of existing standard computers generally include a CPU and a GPU. By allowing the CPU and GPU to work together, the system resource efficiency is improved. At the same time, there is no need to customize a dedicated processor, which reduces the hardware cost.

[0136] Please refer to Figure 5 , Fig. 9 and Fig.10 Another embodiment of the present application further provides a classification system for ultrasound images of pulmonary effusion, comprising:

[0137] The medical device 10 in the above embodiment.

[0138] Communication device 30.

[0139] The remote device 20 is communicatively connected with the medical device 10 via the communication device 30. The remote device 20 includes an operating device 210 and a display device 220. The operating device 210 is used to control the movement and scanning of the three-axis robotic arm 120 through the communication device 30 to generate an ultrasonic detection image. The display device 220 is used to receive and display the classification result through the communication device 30.

[0140] In this embodiment, the classification system of the ultrasonic image of lung effusion consists of two parts: the patient side, corresponding to the medical device 10 and the doctor side, corresponding to the remote device 20 .

[0141] Among them, the doctor's side includes an ultrasound display system, that is, the display device 220 of the embodiment of the present application, and a remote control system, that is, the operating device 210 of the embodiment of the present application. The patient's side includes an ultrasound imaging system, that is, the medical device 10 of the embodiment of the present application, and a robotic arm control system, wherein the robotic arm control system includes the three-axis robotic arm 120 of the embodiment of the present application.

[0142] The patient side and the doctor side are connected via a communication device 30, such as a 5G network connection. The doctor uses the doctor side remote control system to control the robotic arm and the ultrasound system to remotely and quickly evaluate the patient's pulmonary effusion. At the same time, the doctor can communicate with the patient in real-time video and voice, and inquire about the condition to facilitate evaluation.

[0143] Specifically, doctors can autonomously control the patient end through the six-degree-of-freedom contour-shaping manipulator system with sensor fusion technology, full-process simulated human operation, and high-definition audio and video system.

[0144] Among them, a dedicated communication device 30 is used to reduce the network delay to a very low millisecond level, so as to achieve synchronization of operations on the doctor side and the patient side.

[0145] The robotic arm is also equipped with precise force sensors and a reasonable force threshold system design, which can achieve flexible contact with the patient's lungs. When the collision force is too large, the robotic arm will automatically protect itself to ensure the safety of the patient and avoid secondary injuries.

[0146] Finally, based on the classification method of ultrasound images of lung effusion in the above embodiment, artificial intelligence methods are used to preprocess the data, evaluate the lung trauma level, segment the effusion, and automatically calculate the effusion area and volume, so as to achieve fast and accurate output of evaluation results.

[0147] Specifically, the rapid evaluation process of the complete classification system of lung effusion ultrasound images is as follows:

[0148] The patient lies flat on the bed;

[0149] The patient side creates new patient information (the patient information is pushed to the doctor side);

[0150] The doctor receives the push message and starts the examination;

[0151] Start the robotic arm and let it reach the zero point;

[0152] Doctors use the contour-based operating system to remotely control the robotic arm to fit the lungs;

[0153] Collect the required ultrasound images;

[0154] Implement classification methods;

[0155] Push the results to the doctor;

[0156] The classification assessment is completed.

[0157] In other embodiments, the ultrasound image may also be sent to the doctor side via the network, and the classification and evaluation process may be performed by the processing device of the doctor side to obtain the evaluation result.

[0158] Compared with sending the ultrasound image to the doctor side through the communication device 30 to perform the evaluation and classification process, the classification and evaluation process is performed by the medical device 10 on the patient side in this embodiment. Sending the evaluation results only through the communication device 30 has faster speed and efficiency.

[0159] The classification of ultrasound images of pulmonary effusion, medical equipment 10 and system provided in the above multiple embodiments of the present application perform two-dimensional recognition on ultrasound images of pulmonary effusion to determine whether there is pulmonary effusion, and generate a segmentation mask corresponding to the area of ​​pulmonary effusion, then obtain the pixel size of the area of ​​pulmonary effusion in the segmentation mask, calculate the pixel size of the segmentation mask generated in the image recognition and segmentation process to obtain the physical size of the pulmonary effusion, and output the corresponding classification result when the threshold is exceeded, thereby realizing automatic classification of ultrasound images of pulmonary effusion without human intervention, and at the same time, compared with three-dimensional recognition, two-dimensional recognition has higher accuracy for ultrasound images. High, the pixel size is calculated through the segmentation mask generated in the image recognition and segmentation process to obtain the physical size of the pulmonary effusion, and there is no need to extract the mask or crop the ultrasound image through the segmentation mask, and then evaluate the size of the pulmonary effusion by obtaining the final generated segmented image, which speeds up the classification of the pulmonary effusion in the ultrasound image. Therefore, the present application realizes the rapid and automatic classification of ultrasound images containing pulmonary effusion, and at the same time obtains the corresponding classification results according to the different severity of pulmonary effusion, and also realizes the classification of patients with different severity of pulmonary effusion, so as to facilitate the evaluation basis for taking further rescue measures.

[0160] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., may be located in one place, or may be distributed over multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0161] Through the description of the above embodiments, it will be appreciated by those skilled in the art that all or some steps and systems in the disclosed methods above can be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a readable medium, and the readable medium can include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as a computer-readable signal, a data structure, an instruction module or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, disk storage or other magnetic storage device, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media generally include computer readable signals, data structures, instruction modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0162] The embodiments of the present application are described in detail above in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in the relevant technical field without departing from the purpose of the present application.

Claims

1. A classification method for ultrasound images of pulmonary effusion, applied to medical equipment, characterized in that: The medical device comprises an ultrasound device, and the ultrasound device is used to generate an ultrasound image by scanning. The method comprises: Acquire multiple continuous ultrasound image frames, and identify whether each ultrasound image frame includes features of pulmonary effusion; When the lung effusion feature is identified, a segmentation mask of the lung effusion region of the lung trauma image of the corresponding frame is generated; The physical size of the pulmonary effusion is obtained by the pixel size of the pulmonary effusion area in the multi-frame segmentation mask; When the physical size exceeds a threshold, a corresponding classification result is output.

2. The classification method of lung effusion ultrasound images according to claim 1, characterized in that: The acquiring of multiple continuous ultrasound image frames and identifying whether each ultrasound image frame includes a lung effusion feature comprises: Pre-collecting a plurality of lung injury data sets, and pre-processing the pre-collected lung injury data sets, wherein the data sets include a plurality of frames of images, each frame of the image includes a category and an instance to generate a data fingerprint; Correspondingly, when the lung effusion feature is identified, generating a segmentation mask of the lung effusion region of the corresponding frame of the lung trauma image includes: The connected components with the largest size are retained to generate the segmentation mask.

3. The classification method of lung effusion ultrasound images according to claim 2, characterized in that: When the lung effusion feature is identified, a segmentation mask of the lung effusion region of the corresponding frame of lung injury image is generated, including: The acquired multiple frames of ultrasound images are used as original image inputs, and after preprocessing, the first neural network is used to perform convolution and pooling downsampling to locate the effusion feature area, and the second neural network is used to extract the global information of the effusion feature to obtain the low-resolution effusion feature; The first neural network is used to perform step-by-step deconvolution upsampling to map low-resolution effusion features into high-resolution segmentation masks.

4. The classification method of lung effusion ultrasound images according to claim 3, characterized in that: The method of obtaining the physical size of the pulmonary effusion by the pixel size of the pulmonary effusion region in the multi-frame segmentation mask comprises: The boundary points of the lung effusion area of ​​multiple consecutive frames of two-dimensional segmentation masks are stacked to obtain a three-dimensional point cloud dataset; Fitting the three-dimensional point cloud data set by surface fitting to obtain a three-dimensional surface equation of the corresponding pulmonary effusion pixel surface; Calculating the three-dimensional surface equation by multiple integration to obtain the pixel volume corresponding to the lung effusion area; A first physical volume of the pulmonary effusion is obtained according to the pixel volume of the pulmonary effusion region and the original resolution information of the ultrasound image.

5. The classification method of lung effusion ultrasound images according to claim 4, characterized in that: The method of fitting the three-dimensional point cloud data set by a surface to obtain a three-dimensional surface equation of a corresponding pulmonary effusion pixel surface includes: The surface fitting parameters are obtained by using a two-dimensional quadratic surface formula and a least square method to obtain a three-dimensional surface equation of the pulmonary effusion pixel surface; After the format of the three-dimensional point cloud data set is converted into an array format, a corresponding coefficient matrix and a target vector are generated, and the fitting error is minimized to obtain the surface fitting parameters.

6. The classification method of lung effusion ultrasound images according to claim 5, characterized in that: The step of obtaining the physical size of the pulmonary effusion by the pixel size of the pulmonary effusion region in the multi-frame segmentation mask further comprises: When the pixel value of the preset position outside the lung effusion region in the segmentation mask is a preset value, the preset position is determined to be a boundary point corresponding to the lung effusion region.

7. The classification method of ultrasonic images of pulmonary effusion according to claim 3 or 6, characterized in that: The medical device further includes a three-axis mechanical arm, and the ultrasound device is moved and scanned by the three-axis mechanical arm to generate an ultrasound image. The three-axis mechanical arm is also used to generate three-dimensional pose coordinates in real time. The physical size of the lung effusion obtained by the pixel size of the lung effusion area image in the multi-frame segmentation mask includes: Calculate the number of pixels whose pixel values ​​in each frame segmentation mask are preset values, wherein when the change in the three-dimensional pose coordinates generated by the three-axis robotic arm is within a preset range, stop calculating the number of pixels whose pixel values ​​in the corresponding frame segmentation mask are preset values; The number of pixels whose pixel values ​​in the segmentation masks of consecutive multiple frames are preset values ​​is summed to obtain the total number of pixels of the lung effusion target; Obtaining a second physical volume of the pulmonary effusion according to the total number of pixels of the pulmonary effusion target and the original resolution information of the ultrasound image; Correspondingly, the outputting of the corresponding classification result according to the physical size includes: An interval range of the first physical volume and / or the second physical volume is defined to output a corresponding classification result.

8. The method for classifying ultrasonic images of pulmonary effusion according to claim 7, characterized in that: Before identifying whether each frame of ultrasound image includes a lung effusion feature, the method further includes: Generate trauma grade scores of multiple frames of ultrasound images through a third neural network; When the score value is greater than a preset value, it is then identified whether each frame of the ultrasound image includes a lung effusion feature.

9. A medical device, characterized in that: include: An ultrasound device, the ultrasound device being used to generate an ultrasound image by scanning; The first processing unit is used to execute the classification method of the lung effusion ultrasound image according to any one of claims 1-6.

10. The medical device according to claim 9, characterized in that Also includes A three-axis mechanical arm, through which the ultrasonic device moves and scans to generate an ultrasonic detection image, and the three-axis mechanical arm is also used to generate three-dimensional posture coordinates in real time; A second processing unit, configured to execute the lung effusion ultrasound image classification method according to claim 8 through a central processing unit; Wherein, the first processing unit is further used to execute the classification method of the lung effusion ultrasound image according to any one of claims 1-7 through a graphics processor.

11. A classification system for ultrasound images of pulmonary effusion, characterized in that: include The medical device according to claim 10; communication devices; A remote device is communicatively connected with the medical device via the communication device, and the remote device includes an operating device and a display device. The operating device is used to control the movement and scanning of the three-axis robotic arm through the communication device to generate an ultrasonic detection image, and the display device is used to receive and display the classification result through the communication device.