Ultrasonic image quality assessment method, model training method and electronic equipment

Abstract

The present application discloses a quality assessment method for ultrasound images, a model training method and an electronic device, and belongs to the field of image processing technology. The method includes: obtaining a target ultrasound image of a target part, the target ultrasound image is obtained by detecting the target part with an ultrasound probe of an ultrasound scanning device; performing quality assessment on the target ultrasound image through a target image assessment model, the target image assessment model can assess the quality of any ultrasound image; wherein the target image assessment model is obtained by iteratively training a first image assessment model based on a training sample set, the training sample set includes at least one image group, each image group includes multiple sample ultrasound images of different qualities, and each sample ultrasound image is annotated with assessment information after quality assessment by medical personnel. In an embodiment of the present application, the target image assessment model can learn the quality assessment criteria of ultrasound images by medical personnel, thereby improving the accuracy of quality assessment.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular to an ultrasound image quality assessment method, a model training method and an electronic device. Background Art

[0002] With the progress of society, medical diagnostic methods are also constantly improving. For example, diseases can be diagnosed through medical ultrasound examination. Among them, medical ultrasound examination is a medical imaging diagnostic technology that uses ultrasound to detect the internal structure of the patient's body and generate ultrasound images that reflect the internal structure of the body. In order to reduce the burden on medical staff and avoid direct contact between doctors and patients, more and more institutions have begun to use ultrasound scanning equipment to autonomously collect ultrasound images. In order to improve the accuracy of diagnosis, ultrasound scanning equipment can evaluate the quality of ultrasound images during the scanning process, and then adjust the position of the ultrasound probe of the ultrasound scanning equipment according to the evaluation results.

[0003] At present, the quality of ultrasound images can be evaluated by ultrasound confidence maps. That is, after the ultrasound scanning device obtains the ultrasound image by scanning, it can process the ultrasound image by a confidence estimation method to obtain the ultrasound confidence map corresponding to the ultrasound image, and then evaluate the quality of the ultrasound image based on the ultrasound confidence map.

[0004] However, the ultrasound confidence map measures the reliability of the imaging at the corresponding pixel position of the ultrasound image, and does not consider the diagnostic significance of the overall content characteristics of the ultrasound image, which leads to inaccurate quality assessment of the ultrasound image by the ultrasound scanning equipment. Summary of the invention

[0005] The embodiment of the present application provides a method for evaluating the quality of an ultrasound image, a model training method, and an electronic device, which can solve the problem of inaccurate quality evaluation of ultrasound images by ultrasound scanning equipment in the related art. The technical solution is as follows:

[0006] In a first aspect, a method for evaluating the quality of an ultrasound image is provided, the method comprising:

[0007] Acquire a target ultrasonic image of the target part, wherein the target ultrasonic image is obtained by detecting the target part with an ultrasonic probe of an ultrasonic scanning device;

[0008] Performing quality assessment on the target ultrasound image by using a target image assessment model, wherein the target image assessment model can assess the quality of any ultrasound image;

[0009] Among them, the target image evaluation model is obtained by iteratively training the first image evaluation model based on a training sample set, and the training sample set includes at least one image group, each image group includes multiple sample ultrasound images of different qualities, and each sample ultrasound image is annotated with evaluation information after quality evaluation by medical staff.

[0010] In a second aspect, a model training method is provided, the method comprising:

[0011] Acquire a training sample set, wherein the training sample set includes at least one image group, each image group includes a plurality of sample ultrasound images of different qualities, and each sample ultrasound image is annotated with evaluation information after quality evaluation by medical personnel;

[0012] Iteratively training the first image evaluation model based on the training sample set to obtain a second image evaluation model;

[0013] If the iterative training end condition is currently met, a target image evaluation model is determined based on the second image evaluation model, and the target image evaluation model can evaluate the quality of any ultrasound image.

[0014] As an example of the present application, the step of obtaining a training sample set includes:

[0015] Acquire a plurality of sample ultrasound images, wherein the plurality of sample ultrasound images include multiple types of ultrasound images with different qualities;

[0016] According to the number of each type of sample ultrasonic images among the multiple sample ultrasonic images, assigning corresponding weights to each type of sample ultrasonic images, wherein the number of each type of sample ultrasonic images is negatively correlated with the weights corresponding to each type of sample ultrasonic images;

[0017] Based on the weights corresponding to the various types of sample ultrasound images, image sampling is performed from the multiple sample ultrasound images;

[0018] When the number of samples reaches a number threshold, combining sample ultrasound images obtained by sampling the number threshold into an image group;

[0019] The image sampling process is repeated until the plurality of sample ultrasound images are sampled, thereby obtaining the at least one image group.

[0020] As an example of the present application, the image sampling from the multiple sample ultrasound images based on the weights corresponding to the various types of sample ultrasound images includes:

[0021] Determining the number of samples to be sampled from the various types of sample ultrasonic images according to the weights corresponding to the various types of sample ultrasonic images, wherein the weights corresponding to the various types of sample ultrasonic images are positively correlated with the number of samples corresponding to the various types of sample ultrasonic images;

[0022] Image sampling is performed from the various types of sample ultrasonic images according to the sampling quantities corresponding to the various types of sample ultrasonic images.

[0023] As an example of the present application, if the iterative training end condition is currently met, determining the target image evaluation model based on the second image evaluation model includes:

[0024] If the iterative training end condition is currently met, the quality of the test ultrasound image is evaluated by the second image evaluation model to obtain a test evaluation result;

[0025] If the similarity between the test evaluation result and the reference evaluation result is greater than or equal to a similarity threshold, the second image evaluation model is determined as the target image evaluation model, and the reference evaluation result is a quality evaluation result of the test ultrasound image by medical personnel.

[0026] As an example of the present application, the first image evaluation model includes a feature extraction model and a classifier; the iterative training of the first image evaluation model based on the training sample set to obtain the second image evaluation model includes:

[0027] Based on the training sample set, forward computing the first image evaluation model to determine a first loss value of the first image evaluation model;

[0028] Based on the first loss value, updating the model parameters of the classifier by backpropagation;

[0029] Based on the training sample set, forward computing the first image evaluation model again to determine a second loss value of the first image evaluation model;

[0030] Based on the second loss value, updating the model parameters of the feature extraction model and the model parameters of the classifier by means of back propagation;

[0031] If the iterative training end condition is not currently met, return to the operation of performing forward calculation on the first image evaluation model based on the training sample set to determine a first loss value of the first image evaluation model until the iterative training end condition is currently met, terminate the iterative training, and obtain the second image evaluation model.

[0032] As an example of the present application, performing forward calculation on the first image evaluation model based on the training sample set to determine a first loss value of the first image evaluation model includes:

[0033] Inputting any image group in the training sample set into the first image evaluation model;

[0034] Processing any one of the image groups by using the first image evaluation model to obtain a quality prediction value corresponding to each sample ultrasound image in the any one of the image groups, wherein the quality prediction value is used to indicate an evaluation result of the first image evaluation model performing quality evaluation on the corresponding sample ultrasound image;

[0035] The first loss value is determined based on the quality prediction value corresponding to each sample ultrasound image in any one of the image groups through a target loss function.

[0036] As an example of the present application, if the similarity between the test evaluation result and the reference evaluation result is greater than or equal to a similarity threshold, before determining the second image evaluation model as the target image evaluation model, the method further includes:

[0037] determining a correlation coefficient between the test evaluation result and a reference evaluation result;

[0038] In the case that the absolute value of the correlation coefficient is greater than or equal to the coefficient threshold, it is determined that the similarity between the test evaluation result and the reference evaluation result is greater than or equal to the similarity threshold.

[0039] In a third aspect, a device for evaluating the quality of an ultrasound image is provided, the device comprising:

[0040] An acquisition module, used for acquiring a target ultrasonic image of a target part, wherein the target ultrasonic image is obtained by detecting the target part with an ultrasonic probe of an ultrasonic scanning device;

[0041] An evaluation module, used for performing quality evaluation on the target ultrasound image by using a target image evaluation model, wherein the target image evaluation model can evaluate the quality of any ultrasound image;

[0042] Among them, the target image evaluation model is obtained by iteratively training the first image evaluation model based on a training sample set, and the training sample set includes at least one image group, each image group includes multiple sample ultrasound images of different qualities, and each sample ultrasound image is annotated with evaluation information after quality evaluation by medical staff.

[0043] In a fourth aspect, a model training device is provided, the device comprising:

[0044] An acquisition module, used to acquire a training sample set, wherein the training sample set includes at least one image group, each image group includes a plurality of sample ultrasound images of different qualities, and each sample ultrasound image is annotated with evaluation information after quality evaluation by medical personnel;

[0045] A training module, configured to iteratively train the first image evaluation model based on the training sample set to obtain a second image evaluation model;

[0046] A determination module is used to determine a target image evaluation model based on the second image evaluation model if an iterative training end condition is currently met, and the target image evaluation model can evaluate the quality of any ultrasound image.

[0047] As an example of the present application, the acquisition module includes:

[0048] An acquisition submodule, configured to acquire a plurality of sample ultrasonic images, wherein the plurality of sample ultrasonic images include multiple types of ultrasonic images with different qualities;

[0049] an assignment submodule, configured to assign corresponding weights to various types of sample ultrasonic images according to the number of each type of sample ultrasonic images among the multiple sample ultrasonic images, wherein the number of each type of sample ultrasonic images is negatively correlated with the weights corresponding to the various types of sample ultrasonic images;

[0050] A sampling submodule, configured to sample images from the plurality of sample ultrasound images based on weights corresponding to the various types of sample ultrasound images;

[0051] A combining submodule, configured to combine sample ultrasound images obtained by sampling the number threshold into an image group when the number of samples reaches the number threshold;

[0052] The first triggering submodule is used to trigger the sampling submodule to repeat the image sampling process until the sampling of the multiple sample ultrasound images is completed to obtain the at least one image group.

[0053] As an example of the present application, the sampling submodule is used for:

[0054] Determining the number of samples to be sampled from the various types of sample ultrasonic images according to the weights corresponding to the various types of sample ultrasonic images, wherein the weights corresponding to the various types of sample ultrasonic images are positively correlated with the number of samples corresponding to the various types of sample ultrasonic images;

[0055] Image sampling is performed from the various types of sample ultrasonic images according to the sampling quantities corresponding to the various types of sample ultrasonic images.

[0056] As an example of the present application, the determining module includes:

[0057] An evaluation submodule, configured to, if the iterative training end condition is currently met, perform a quality evaluation on the test ultrasound image using the second image evaluation model to obtain a test evaluation result;

[0058] The first determination submodule is used to determine the second image evaluation model as the target image evaluation model if the similarity between the test evaluation result and the reference evaluation result is greater than or equal to a similarity threshold, and the reference evaluation result is the quality evaluation result of the test ultrasound image by medical personnel.

[0059] As an example of the present application, the first image evaluation model includes a feature extraction model and a classifier; the training module includes:

[0060] A first calculation submodule, configured to perform forward calculation on the first image evaluation model based on the training sample set to determine a first loss value of the first image evaluation model;

[0061] A first updating submodule, configured to update the model parameters of the classifier by backpropagation based on the first loss value;

[0062] A second calculation submodule is used to perform forward calculation on the first image evaluation model again based on the training sample set to determine a second loss value of the first image evaluation model;

[0063] A second updating submodule, used for updating the model parameters of the feature extraction model and the model parameters of the classifier by back-propagation based on the second loss value;

[0064] The second trigger submodule is used to trigger the first calculation submodule to perform forward calculation on the first image evaluation model based on the training sample set to determine a first loss value of the first image evaluation model if the iterative training end condition is not currently met, until the iterative training end condition is currently met, thereby ending the iterative training and obtaining the second image evaluation model.

[0065] As an example of the present application, the first computing submodule is used for:

[0066] Inputting any image group in the training sample set into the first image evaluation model;

[0067] Processing any one of the image groups by using the first image evaluation model to obtain a quality prediction value corresponding to each sample ultrasound image in the any one of the image groups, wherein the quality prediction value is used to indicate an evaluation result of the first image evaluation model performing quality evaluation on the corresponding sample ultrasound image;

[0068] The first loss value is determined based on the quality prediction value corresponding to each sample ultrasound image in any one of the image groups through a target loss function.

[0069] As an example of the present application, the determining module further includes:

[0070] A second determination submodule is used to determine the correlation coefficient between the test evaluation result and the reference evaluation result;

[0071] The third determination submodule is used to determine that the similarity between the test evaluation result and the reference evaluation result is greater than or equal to the similarity threshold when the absolute value of the correlation coefficient is greater than or equal to the coefficient threshold.

[0072] In a fifth aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for quality assessment of ultrasound images as described in the first aspect is implemented.

[0073] In a sixth aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for model training described in the second aspect is implemented.

[0074] In a seventh aspect, a computer-readable storage medium is provided, on which instructions are stored. When the instructions are executed by a processor, the method for quality assessment of ultrasound images described in the first aspect is implemented.

[0075] In an eighth aspect, a computer-readable storage medium is provided, on which instructions are stored, and when the instructions are executed by a processor, the method for model training described in the second aspect is implemented.

[0076] In a ninth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the method for ultrasound image quality assessment described in the first aspect.

[0077] In the tenth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the model training method described in the second aspect above.

[0078] It can be understood that the beneficial effects of the second to tenth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0079] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0080] In an embodiment of the present application, since the target image evaluation model is obtained by iteratively training a first image evaluation model based on at least one image group included in a training sample set, and each sample ultrasound image in each image group is annotated with evaluation information after quality evaluation by medical personnel, that is, the target image evaluation model can learn the quality evaluation standards of medical personnel for ultrasound images. Therefore, the quality evaluation of the target ultrasound image is performed through the target image evaluation model, thereby improving the accuracy of the quality evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0082] Figure 1 is a flow chart of a method for evaluating the quality of an ultrasound image according to an exemplary embodiment;

[0083] Figure 2 is a structural schematic diagram of a feature extraction module according to an exemplary embodiment;

[0084] Figure 3 is a structural diagram of a residual learning module according to an exemplary embodiment;

[0085] Figure 4 is a schematic diagram of the structure of a classifier according to an exemplary embodiment;

[0086] Figure 5 is a structural schematic diagram of a target image evaluation model according to an exemplary embodiment;

[0087] Figure 6 is a flow chart of a model training method according to an exemplary embodiment;

[0088] Figure 7 is a schematic structural diagram of a device for evaluating the quality of an ultrasound image according to an exemplary embodiment;

[0089] Figure 8 is a structural schematic diagram of a model training device according to an exemplary embodiment;

[0090] Fig. 9 The diagram is a schematic structural diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0091] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0092] It should be understood that the "multiple" mentioned in this application refers to two or more. In the description of this application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solution of this application, the words "first" and "second" are used to distinguish between the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not limit them to be different.

[0093] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "include", "including", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0094] Before explaining in detail a method for evaluating the quality of an ultrasound image provided in an embodiment of the application, an application scenario provided in an embodiment of the application is first introduced.

[0095] At present, in order to reduce the burden on doctors and ensure the consistency of ultrasound image quality, the use of autonomous scanning ultrasound scanning equipment (also known as ultrasound autonomous acquisition equipment, ultrasound autonomous scanning robot, etc.) for automatic acquisition of ultrasound images has become a trend in ultrasound diagnosis. Since the position, direction, and contact between the ultrasound probe of the ultrasound scanning equipment and the subject significantly affect the quality of the ultrasound image, and the ultrasound image has a lot of noise and artifacts, low resolution, blurred boundaries, and low contrast in imaging characteristics. Therefore, in the process of automatic acquisition by the ultrasound scanning equipment, the key step is to evaluate the quality of the acquired ultrasound image, and then the quality evaluation result can be fed back to the control system of the ultrasound scanning equipment, so that the control system of the ultrasound scanning equipment adjusts the position of the ultrasound probe according to the quality evaluation result of the ultrasound image.

[0096] At present, ultrasound scanning equipment can collect ultrasound images of the part to be examined by the ultrasound probe, process the ultrasound image by the confidence estimation method, obtain the ultrasound confidence map corresponding to the ultrasound image, and then evaluate the quality of the ultrasound image based on the ultrasound confidence map. However, the ultrasound confidence map measures the reliability of the imaging of the corresponding pixel position of the ultrasound image, and does not consider the significance of the overall content characteristics of the ultrasound image for diagnosis, which leads to inaccurate quality evaluation of the ultrasound image by the ultrasound scanning equipment.

[0097] Based on such an application scenario, an embodiment of the present application provides a quality assessment method for ultrasound images that can improve the accuracy of quality assessment.

[0098] Figure 1 The flowchart of a method for evaluating the quality of an ultrasonic image is shown according to an exemplary embodiment. This embodiment is illustrated by applying the method for evaluating the quality of an ultrasonic image to an electronic device. The method for evaluating the quality of an ultrasonic image may include the following steps:

[0099] Step 101: Acquire a target ultrasound image of a target part. The target ultrasound image is obtained by detecting the target part with an ultrasound probe of an ultrasound scanning device.

[0100] When performing ultrasonic diagnosis on a subject, an ultrasonic scanning device is required. The ultrasonic probe of the ultrasonic scanning device can detect the target part of the subject, thereby obtaining a target ultrasonic image of the target part.

[0101] It should be noted that the target part is any body part of the subject, for example, the abdomen, chest, etc.

[0102] In some embodiments, the electronic device may be an ultrasonic scanning device, or may be another device that controls the ultrasonic scanning device. When the electronic device is an ultrasonic scanning device, the ultrasonic scanning device may directly detect the target part through an ultrasonic probe to obtain a target ultrasonic image; when the electronic device is another device that controls the ultrasonic scanning device, the ultrasonic scanning device detects the target part through an ultrasonic probe to obtain a target ultrasonic image, and then sends the target ultrasonic image to the electronic device, so that the electronic device acquires the target ultrasonic image.

[0103] As an example, the ultrasonic scanning device may be a certain brand S2000 ultrasonic diagnostic instrument, the ultrasonic probe model used may be 9L4, and the probe frequency may be 5 to 12 MHz (megahertz).

[0104] Step 102: Perform quality assessment on the target ultrasound image using a target image assessment model. The target image assessment model can assess the quality of any ultrasound image.

[0105] As an example, the electronic device may input the target ultrasound image into a target image evaluation model, and the target image evaluation model may process the target ultrasound image, and after the processing is completed, output a quality evaluation result of the target ultrasound image.

[0106] It should be noted that the target image evaluation model is obtained by iteratively training the first image evaluation model based on the training sample set, the training sample set includes at least one image group, each image group includes a plurality of sample ultrasound images of different quality, and each sample ultrasound image is annotated with evaluation information after quality evaluation by medical personnel. The process of iteratively training the first image evaluation model to obtain the target image evaluation model can refer to the operations of steps 601 to 603 described below.

[0107] As an example, the target image evaluation model may include a feature extraction model and a classifier. The feature extraction model can perform convolution, pooling and other operations on the target ultrasound image to obtain feature information of the target ultrasound image, and then input the extracted feature information into the classifier. The classifier can classify the feature information and output the classification processing result, that is, the output quality evaluation result.

[0108] In order to facilitate the understanding of the embodiments of the present application, the feature extraction model and the classifier are now explained.

[0109] In one embodiment, the feature extraction model may be a ResNet (residual neural network) 18, and in the embodiment of the present application, see Figure 2 The first module of ResNet18 is the preprocessing module, which consists of a convolutional layer of size 7*7 and stride 2 and a maximum pooling layer of size 3*3 and stride 2. Four residual learning modules are connected after the preprocessing module, each of which consists of two residual learning modules. The structure of each residual learning module can be shown as follows: Figure 3 As shown in the figure, each residual learning module can be composed of two 3*3 convolutions and a jump layer connection. The last layer of ResNet18 is a fully connected layer, and the number of nodes in the output layer is the number of categories.

[0110] It should be noted that in the embodiment of the present application, a jump-layer connection is added to the residual learning module. By jumping before the activation function, the output of the previous layer or layers is added to the output calculated by this layer, and the sum is input into the activation function as the output of this layer. This process can be expressed by the following first formula.

[0111] y=F(x,{W i})+x (1)

[0112] It should be noted that in the first formula (1), x is the input information of the residual learning module, y is the output information, F is the residual between the input information and the output information, and W i is the weight of the corresponding layer network. Wherein, the residual F can be expressed by the following second formula.

[0113] F=W 2 σ(W 1 x) (2)

[0114] It should be noted that in the above second formula (2), W 1 and W 2 are the weights of the two-layer network respectively, and σ() is the ReLU (Rectified Linear Units, linear rectification function) activation function.

[0115] It should also be noted that the feature extraction module can not only be ResNet18, but also a feature extraction module of other structures. The embodiment of the present application only takes the feature extraction module of ResNet18 as an example for illustration, and does not make specific restrictions on the structure of the feature extraction model.

[0116] In one embodiment, in order to improve the accuracy of image quality assessment, the embodiment of the present application can use a high-order feature classifier in fine-grained image classification to replace the fully connected layer. Since the task scenario of the fine-grained network is to identify subcategories under large categories, the characteristics of fine-grained classification include subtle differences between subcategories, huge differences within subcategories, and being affected by viewing angles, backgrounds, occlusions, etc. These characteristics are similar to the quality assessment of ultrasound images. The differences between ultrasound images of different qualities are not significant, while ultrasound images of the same quality may have huge differences due to the location and angle of ultrasound acquisition. Therefore, in the embodiment of the present application, the high-order feature encoder method BCNN (Bilinear Convolutional Neural Networks) commonly used in fine-grained classification can be selected as a classifier.

[0117] As an example, the BCNN method uses quadratic linear pooling to combine the features extracted by two CNNs (Convolutional Neural Networks) to improve classification accuracy. See the schematic diagram of BCNN for details. Figure 4 , Figure 4 In this paper, CNN A and CNN B are used as examples. Based on this, the structure of the target image evaluation model can be seen in Figure 5. Among them, the specific operation of quadratic linear pooling includes: performing outer product operation on the feature information extracted by the two networks. For example, A and B are the feature information extracted by the two CNN networks respectively. The process of performing outer product operation on feature information A and feature information B can be expressed by the third formula; then, summing and pooling the results obtained after the outer product operation to obtain bilinear features. The process of summing and pooling the results after the outer product operation can be expressed by the following fourth formula; finally, the bilinear features are transformed by the square root of the sign, which is expressed by the following fifth formula, and standardized (for example, l2 standardization), which is expressed by the following sixth formula. Then the result is input into the classifier to complete the classification task.

[0118] As an example, the third formula (3), the fourth formula (4), the fifth formula (5) and the sixth formula (6) are as follows:

[0119] X(I)=A(I) T B(I) (3)

[0120]

[0121]

[0122]

[0123] It should be noted that in the above third formula (3), fourth formula (4), fifth formula (5) and sixth formula (6), A(I) is the uncharacterized information A, B(I) is the characteristic information B, X(I) is the result after the outer product operation, X is the bilinear feature, Y is the result of the signed square root transformation of the bilinear feature, and Z is the result obtained after standardization.

[0124] It should also be noted that the classifier can not only be BCNN, but also can be a classifier of other structures. The embodiment of the present application only takes the classifier as BCNN as an example for illustration, and does not impose any specific restrictions on the structure of the classifier.

[0125] Step 103: Based on the quality assessment result of the target ultrasound image, the position and posture of the ultrasound probe are adjusted.

[0126] As an example, the electronic device may control the contact posture between the ultrasound probe and the target part to change based on the quality assessment result of the target ultrasound image, and at the same time, control the position of the ultrasound probe to move so that the contact position between the ultrasound probe and the target part changes.

[0127] As an example, the electronic device can control the position and posture of the ultrasound probe to change randomly based on the quality assessment result of the target ultrasound image, or can control the position and posture of the ultrasound probe to change according to a specified rule. The embodiments of the present application do not impose specific restrictions on this.

[0128] It should be noted that the operation of adjusting the position and posture of the ultrasound probe by the electronic device based on the quality assessment result of the target ultrasound image can refer to the relevant technology, and the embodiments of the present application will not describe them one by one.

[0129] From the above, it can be seen that the electronic device can be an ultrasonic scanning device or other device that controls the ultrasonic scanning device; when the electronic device is an ultrasonic scanning device, the electronic device can directly adjust the posture of the ultrasonic probe; when the electronic device is other device that controls the ultrasonic scanning device, the electronic device can send control information to the control system of the ultrasonic scanning device based on the quality evaluation result of the target ultrasonic image, and the ultrasonic scanning device adjusts the posture of the ultrasonic probe after receiving the control information.

[0130] In an embodiment of the present application, since the target image evaluation model is obtained by iteratively training a first image evaluation model based on at least one image group included in a training sample set, and each sample ultrasound image in each image group is annotated with evaluation information after quality evaluation by medical personnel, that is, the target image evaluation model can learn the quality evaluation standards of medical personnel for ultrasound images. Therefore, the quality evaluation of the target ultrasound image is performed through the target image evaluation model, thereby improving the accuracy of the quality evaluation.

[0131] Next, the model training method is explained, see Figure 6 , the embodiment of the present application provides a flow chart of a model training method. This embodiment uses the model training method applied to an electronic device as an example for illustration. The model training method may include the following steps:

[0132] Step 601: Obtain a training sample set.

[0133] It should be noted that the training sample set includes at least one image group, each image group includes a plurality of sample ultrasound images of different qualities, and each sample ultrasound image is annotated with evaluation information after quality evaluation by medical personnel.

[0134] As an example, the evaluation information may include the score of the corresponding sample ultrasound image scored by the medical staff, that is, each sample ultrasound image may be judged by the medical staff in advance based on the blur level, noise amount, clarity, contrast, resolution and field of view of the sample ultrasound image, and scored according to the judgment result, and the scoring range is 1 point to 5 points. Among them, 5 points means that the quality of the corresponding sample ultrasound image is excellent and can be used for diagnosis, and the satisfaction level of the corresponding sample ultrasound image is very satisfactory; 4 points means that the quality of the corresponding sample ultrasound image is very good and can be used for diagnosis, and the satisfaction level of the corresponding sample ultrasound image is satisfactory; 3 points means that the quality of the corresponding sample ultrasound image is defective, which does not affect the diagnosis, and the satisfaction level of the corresponding sample ultrasound image is generally satisfactory; 2 points means that the quality of the corresponding sample ultrasound image is poor, which affects the diagnosis, and the satisfaction level of the corresponding sample ultrasound image is unsatisfactory; 1 point means that the quality of the corresponding sample ultrasound image is poor and cannot be diagnosed, and the satisfaction level of the corresponding sample ultrasound image is unsatisfactory.

[0135] As an example, in order to improve the evaluation accuracy of the target image evaluation model for ultrasound images, sample ultrasound images can be screened. For example, for sample ultrasound images with different scores, when the difference in scores given by different medical staff is greater than 2 points, the sample ultrasound image is eliminated. The final score of the screened sample ultrasound image is the average score of the scores given by N medical staff, where N is a positive integer greater than 2 set in advance. For example, N can be 3, 4, and so on.

[0136] In some embodiments, the operation of an electronic device acquiring a training sample set includes: acquiring multiple sample ultrasonic images, the multiple sample ultrasonic images including multiple categories of ultrasonic images with different qualities; assigning corresponding weights to each category of sample ultrasonic images according to the number of each category of sample ultrasonic images in the multiple sample ultrasonic images, and the number of each category of sample ultrasonic images is negatively correlated with the weight corresponding to each category of sample ultrasonic images; based on the weights corresponding to each category of sample ultrasonic images, sampling images from the multiple sample ultrasonic images; when the number of samples reaches a number threshold, combining the sample ultrasonic images obtained by the number threshold samples into an image group; repeating the image sampling process until the multiple sample ultrasonic images are sampled and at least one image group is obtained.

[0137] Since the multiple sample ultrasound images have serious class imbalance problems in quality, the class imbalance problem refers to the large difference in the number of sample ultrasound images of different categories. For example, the number of high-quality and low-quality sample ultrasound images is small, while the number of sample ultrasound images of general quality is the largest, and the overall distribution is close to normal. The class imbalance problem will result in a limited number of training samples of extreme categories of ultrasound images, resulting in inaccurate target image evaluation models obtained through training, which may lead to inaccurate evaluation of ultrasound images by the target image evaluation model. Therefore, in order to improve the accuracy of the training of the target image evaluation model, corresponding weights can be assigned to each type of sample ultrasound image according to the number of each type of sample ultrasound images in the multiple sample ultrasound images.

[0138] As an example, the reciprocal of the ratio of the number of each type of sample ultrasonic images to the number of all sample ultrasonic images can be determined as the weight corresponding to each type of sample ultrasonic image; or, the reciprocal of the number of each type of sample ultrasonic images can be determined as the weight corresponding to each type of sample ultrasonic image.

[0139] In a possible implementation, the operation of sampling images from multiple sample ultrasonic images by an electronic device based on weights corresponding to each type of sample ultrasonic image includes: determining the number of samples to be sampled from each type of sample ultrasonic image according to the weights corresponding to each type of sample ultrasonic image, wherein the weights corresponding to each type of sample ultrasonic image are positively correlated with the number of samples corresponding to each type of sample ultrasonic image; and sampling images from each type of sample ultrasonic image according to the number of samples corresponding to each type of sample ultrasonic image.

[0140] In another possible implementation, the operation of sampling images from multiple sample ultrasonic images by the electronic device based on the weights corresponding to each type of sample ultrasonic image includes: determining a sampling frequency for sampling from each type of sample ultrasonic image according to the weights corresponding to each type of sample ultrasonic image, wherein the weights corresponding to each type of sample ultrasonic image are positively correlated with the sampling frequency corresponding to each type of sample ultrasonic image; and sampling images from each type of sample ultrasonic image according to the sampling frequency corresponding to each type of sample ultrasonic image.

[0141] As an example, the electronic device may determine the weights corresponding to each type of sample ultrasound image as the sampling frequency for sampling from each type of sample ultrasound image, or may determine it in other ways, and the embodiments of the present application do not impose any specific restrictions on this.

[0142] In another possible implementation, the operation of sampling images from multiple sample ultrasonic images by the electronic device based on the weights corresponding to each type of sample ultrasonic image includes: determining the weights corresponding to each type of sample ultrasonic image as the sampling probability of sampling from each type of sample ultrasonic image; and sampling images from each type of sample ultrasonic image according to the sampling probability corresponding to each type of sample ultrasonic image.

[0143] It is worth noting that in the related art, during the sampling process, each batch of sample ultrasonic images is obtained by random and equal sampling, while in the embodiment of the present application, different weights are assigned to sample ultrasonic images of different qualities, and the weight depends on the proportion of the number of each type of sample ultrasonic images to the number of all sample ultrasonic images. Therefore, the lower the number of each type of sample ultrasonic images, the higher the corresponding weight, and the higher the weight, the higher the probability of being sampled, thereby achieving focused training on a certain type of sample ultrasonic images with a smaller number, alleviating the class imbalance problem existing in multiple sample ultrasonic images, and avoiding the overfitting problem caused by oversampling and the sample information loss problem caused by undersampling in the related art.

[0144] Usually, during the iterative training process, each training step inputs the sample ultrasound images into the model in batches for iterative training. Therefore, in order to facilitate iterative training, when the number of samples reaches a threshold number, the sample ultrasound images obtained by sampling the threshold number of samples are combined into an image group.

[0145] It should be noted that the quantity threshold can be set according to the demand, for example, the quantity threshold can be 24.

[0146] Step 602: Iteratively train the first image evaluation model based on the training sample set to obtain a second image evaluation model.

[0147] It should be noted that the first image evaluation model is an image evaluation model to be trained, and the first image evaluation model can be an image evaluation model that has not undergone any training, or can be an image evaluation model that has undergone some training. The embodiment of the present application does not impose any specific restrictions on this.

[0148] Due to the scarcity of sample ultrasound images and sample ultrasound images carrying evaluation information after quality evaluation by medical staff, the small number of sample ultrasound images may be unfavorable for iterative training of the first image evaluation model. Therefore, the strategy of transfer learning can be used to initialize the parameters of the first image evaluation model. The ResNet18 pre-trained in the ImageNet dataset is used as the initial state of the feature extraction part in the first image evaluation model, and subsequent iterative training is carried out based on this.

[0149] As can be seen from the above, the first image evaluation model includes a feature extraction model and a classifier. As an example, the electronic device iteratively trains the first image evaluation model based on the training sample set to obtain the second image evaluation model. The operation includes: based on the training sample set, forward calculating the first image evaluation model to determine the first loss value of the first image evaluation model; based on the first loss value, updating the model parameters of the classifier by back propagation; based on the training sample set, forward calculating the first image evaluation model again to determine the second loss value of the first image evaluation model; based on the second loss value, updating the model parameters of the feature extraction model and the model parameters of the classifier by back propagation; if the iterative training end condition is not met at present, returning to the operation of forward calculating the first image evaluation model based on the training sample set to determine the first loss value of the first image evaluation model, until the iterative training end condition is met at present, ending the iterative training, and obtaining the second image evaluation model.

[0150] As an example, an electronic device performs forward calculation on a first image evaluation model based on a training sample set to determine a first loss value of the first image evaluation model, and the operation includes: inputting any image group in the training sample set into the first image evaluation model; processing any image group through the first image evaluation model to obtain a quality prediction value corresponding to each sample ultrasound image in any image group, and the quality prediction value is used to indicate an evaluation result of the first image evaluation model performing quality evaluation on the corresponding sample ultrasound image; and determining the first loss value based on the quality prediction value corresponding to each sample ultrasound image in any image group through a target loss function.

[0151] It should be noted that the electronic device processes any image group through the first image evaluation model to obtain the quality prediction value corresponding to each sample ultrasonic image in any image group, including: extracting the features of each sample ultrasonic image in any image group through the feature extraction module in the first image evaluation model to obtain the feature information corresponding to each sample ultrasonic image, and classifying the feature information corresponding to each sample ultrasonic image through a classifier to obtain the quality prediction value corresponding to each sample ultrasonic image in any image group.

[0152] As an example, the electronic device determines the first loss value based on the quality prediction value corresponding to each sample ultrasonic image in any image group through a target loss function, and the operations include: determining the mean square error between the quality prediction value corresponding to each sample ultrasonic image and the evaluation information carried in the corresponding sample ultrasonic image through the target loss function; determining the mean square error corresponding to multiple sample ultrasonic images in any image group through the target loss function to obtain the first loss value.

[0153] It should be noted that the target loss function is a pre-set function, for example, the target loss function may be an MSE (Mean Squared Error) function, of course, the target loss function may also be other loss functions, and the embodiment of the present application does not impose specific restrictions on this. Among them, the MES function can be expressed by the following seventh formula.

[0154]

[0155] It should be noted that in the seventh formula (7), x n is the quality prediction value corresponding to the nth sample ultrasound image in any image group, y n is the evaluation information carried by the nth sample ultrasound image, l n is the mean square error corresponding to the nth sample ultrasound image, and l(x, y) is the first loss value.

[0156] It should also be noted that the operation of the electronic device performing forward calculation on the first image evaluation model again based on the training sample set to determine the second loss value of the first image evaluation model can refer to the operation of the above-mentioned electronic device performing forward calculation on the first image evaluation model based on the training sample set to determine the first loss value of the first image evaluation model, and the embodiments of the present application will not elaborate on this one by one.

[0157] In some embodiments, in the process of updating the model parameters of the classifier by back-propagation based on the first loss value, the electronic device can freeze the model parameters of the feature extraction model and update the parameters of the fully connected layer in the classifier based on the first loss value. That is, in the process of updating the model parameters of the classifier by back-propagation based on the first loss value, the parameters of the fully connected layer in the classifier are updated, and the parameters of the feature extraction module remain in their original state.

[0158] It should be noted that the operation of the electronic device updating the model parameters of the classifier by back-transfer based on the first loss value, and updating the model parameters of the feature extraction model and the model parameters of the classifier by back-transfer based on the second loss value can refer to the relevant technology, and the embodiments of the present application will not describe them one by one.

[0159] It is worth noting that, since the classifier can be trained once separately during the iterative training process, the classifier without pre-training can be fully trained.

[0160] In some embodiments, the iterative training end condition can be set in advance according to needs. For example, the iterative training end condition may include: the loss value corresponding to the target loss function converges to the target range, or the number of training times reaches a preset number.

[0161] It should be noted that both the target range and the preset number can be set in advance according to needs. For example, the target range can be 1, 2, etc., and the preset number can be 50 times, 100 times, 200 times, etc.

[0162] Step 603: If the iterative training end condition is currently met, a target image evaluation model is determined based on the second image evaluation model, and the target image evaluation model can evaluate the quality of any ultrasound image.

[0163] In one possible implementation, if the iterative training end condition is currently met, the operation of the electronic device determining the target image evaluation model based on the second image evaluation model includes: if the iterative training end condition is currently met, the test ultrasound image is quality evaluated by the second image evaluation model to obtain a test evaluation result; if the similarity between the test evaluation result and the reference evaluation result is greater than or equal to a similarity threshold, the second image evaluation model is determined as the target image evaluation model, and the reference evaluation result is the quality evaluation result of the test ultrasound image by the medical staff.

[0164] In order to improve the accuracy of quality assessment of ultrasound images, when the conditions for ending iterative training are currently met, the accuracy of the second image evaluation model in assessing the quality of ultrasound images can also be verified, and when it is verified that the similarity between the test evaluation result and the reference evaluation result is greater than or equal to the similarity threshold, the second image evaluation model is determined as the target image evaluation model.

[0165] As an example, the electronic device may directly calculate the similarity between the test evaluation result and the reference evaluation result, and when the similarity is greater than or equal to a similarity threshold, determine the second image evaluation model as the target image evaluation model.

[0166] It should be noted that the operation of the electronic device calculating the similarity between the test evaluation result and the reference evaluation result can refer to the relevant technology, and the embodiments of the present application will not elaborate on this one by one.

[0167] As an example, the electronic device can also express the similarity between the test evaluation result and the reference evaluation result through other parameters. For example, the electronic device can express it through the correlation coefficient between the test evaluation result and the reference evaluation result. That is, the electronic device can determine the correlation coefficient between the test evaluation result and the reference evaluation result; when the absolute value of the correlation coefficient is greater than or equal to the coefficient threshold, it is determined that the similarity between the test evaluation result and the reference evaluation result is greater than the similarity threshold.

[0168] In some embodiments, the electronic device can use PLCC (Pearson linear correlation coefficient, Spearman rank correlation coefficient) to express the similarity between the test evaluation results and the reference evaluation results. The PLCC can be determined by the following eighth formula. That is, the electronic device can determine the correlation coefficient between the test evaluation results and the reference evaluation results by the following eighth formula.

[0169]

[0170] It should be noted that in the eighth formula (8), ρ p is the correlation coefficient, X i is the reference evaluation result of the i-th test ultrasound image, Y i is the test evaluation result of the i-th test ultrasound image, is the mean of the reference evaluation results of N test ultrasound images, is the mean of the test evaluation results of N test ultrasound images, where N is the number of test ultrasound images.

[0171] It should also be noted that PLCC can describe the linear correlation between the test evaluation results and the reference evaluation results. The larger the correlation coefficient, the better the correlation and the higher the similarity between the two. The closer the correlation coefficient is to 1 or -1, the stronger the correlation and the higher the similarity. The closer the correlation coefficient is to 0, the weaker the correlation and the lower the similarity. Since the correlation coefficient may be positive or negative, when the similarity is represented by the correlation coefficient, it is necessary to determine the absolute value of the correlation coefficient, and when the absolute value of the correlation coefficient is greater than or equal to the coefficient threshold, it is determined that the similarity between the test evaluation results and the reference evaluation results is greater than the similarity threshold.

[0172] As another example, the electronic device may also determine a test error between the test evaluation result and a reference evaluation, and when the test error is within an error range, determine that the similarity between the test evaluation result and the reference evaluation result is greater than a similarity threshold.

[0173] It should be noted that the coefficient threshold, similarity threshold and error range can be set in advance according to needs. For example, the coefficient threshold can be 0.5, 0.6, etc., the similarity threshold can be 90%, 95%, etc., and the error range can be 0-1, 0-2, etc.

[0174] In one embodiment, if the similarity between the test evaluation result and the reference evaluation result is less than the similarity threshold, the second image evaluation model is iteratively trained in the same manner as the iterative training of the first image evaluation model until the similarity between the test evaluation result and the reference evaluation result is greater than or equal to the similarity threshold.

[0175] In another possible implementation, if the iterative training end condition is currently met, the second image evaluation model is determined as the target image evaluation model.

[0176] In the embodiment of the present application, since the target image evaluation model is obtained by iteratively training the first image evaluation model based on at least one image group included in the training sample set, and each sample ultrasound image in each image group is annotated with evaluation information after quality evaluation by medical personnel, that is, the target image evaluation model can learn the quality evaluation standards of medical personnel for ultrasound images, therefore, the target ultrasound image is quality evaluated by the target image evaluation model, thereby improving the accuracy of quality evaluation. In addition, since when sampling images during model training, the weights corresponding to each type of sample ultrasound image can be used, and the weight depends on the proportion of the number of each type of sample ultrasound image to the number of all sample ultrasound images, the lower the number of each type of sample ultrasound image, the higher the corresponding weight, and the higher the weight, the higher the probability of being sampled, thereby achieving focused training on a certain type of sample ultrasound image with a smaller number, alleviating the class imbalance problem existing in multiple sample ultrasound images.

[0177] It should be understood that the serial numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0178] Figure 7 701 is a schematic diagram of a structure of an ultrasound image quality assessment device according to an exemplary embodiment, and the ultrasound image quality assessment device can be implemented by software, hardware or a combination of both. The ultrasound image quality assessment device can include: an acquisition module 701 and an assessment module 702.

[0179] An acquisition module 701 is used to acquire a target ultrasonic image of a target part, where the target ultrasonic image is obtained by detecting the target part with an ultrasonic probe of an ultrasonic scanning device;

[0180] An evaluation module 702, configured to perform quality evaluation on the target ultrasound image by using a target image evaluation model, wherein the target image evaluation model can evaluate the quality of any ultrasound image;

[0181] Among them, the target image evaluation model is obtained by iteratively training the first image evaluation model based on a training sample set, and the training sample set includes at least one image group, each image group includes multiple sample ultrasound images of different qualities, and each sample ultrasound image is annotated with evaluation information after quality evaluation by medical staff.

[0182] In an embodiment of the present application, since the target image evaluation model is obtained by iteratively training a first image evaluation model based on at least one image group included in a training sample set, and each sample ultrasound image in each image group is annotated with evaluation information after quality evaluation by medical personnel, that is, the target image evaluation model can learn the quality evaluation standards of medical personnel for ultrasound images. Therefore, the quality evaluation of the target ultrasound image is performed through the target image evaluation model, thereby improving the accuracy of the quality evaluation.

[0183] Figure 8 801 is a schematic diagram of a model training device according to an exemplary embodiment, and the model training device can be implemented by software, hardware or a combination of the two. The model training device can include: an acquisition module 801, a training module 802 and a determination module 803.

[0184] An acquisition module 801 is used to acquire a training sample set, wherein the training sample set includes at least one image group, each image group includes a plurality of sample ultrasound images of different qualities, and each sample ultrasound image is annotated with evaluation information after quality evaluation by medical personnel;

[0185] A training module 802 is used to iteratively train the first image evaluation model based on the training sample set to obtain a second image evaluation model;

[0186] The determination module 803 is used to determine a target image evaluation model based on the second image evaluation model if the iterative training end condition is currently met, and the target image evaluation model can evaluate the quality of any ultrasound image.

[0187] As an example of the present application, the acquisition module 801 includes:

[0188] An acquisition submodule, configured to acquire a plurality of sample ultrasonic images, wherein the plurality of sample ultrasonic images include multiple types of ultrasonic images with different qualities;

[0189] an assignment submodule, configured to assign corresponding weights to various types of sample ultrasonic images according to the number of each type of sample ultrasonic images among the multiple sample ultrasonic images, wherein the number of each type of sample ultrasonic images is negatively correlated with the weights corresponding to the various types of sample ultrasonic images;

[0190] A sampling submodule, configured to sample images from the plurality of sample ultrasound images based on weights corresponding to the various types of sample ultrasound images;

[0191] A combining submodule, configured to combine sample ultrasound images obtained by sampling the number threshold into an image group when the number of samples reaches the number threshold;

[0192] The first triggering submodule is used to trigger the sampling submodule to repeat the image sampling process until the sampling of the multiple sample ultrasound images is completed to obtain the at least one image group.

[0193] As an example of the present application, the sampling submodule is used for:

[0194] Determining the number of samples to be sampled from the various types of sample ultrasonic images according to the weights corresponding to the various types of sample ultrasonic images, wherein the weights corresponding to the various types of sample ultrasonic images are positively correlated with the number of samples corresponding to the various types of sample ultrasonic images;

[0195] Image sampling is performed from the various types of sample ultrasonic images according to the sampling quantities corresponding to the various types of sample ultrasonic images.

[0196] As an example of the present application, the determining module 803 includes:

[0197] An evaluation submodule, configured to, if the iterative training end condition is currently met, perform a quality evaluation on the test ultrasound image using the second image evaluation model to obtain a test evaluation result;

[0198] The first determination submodule is used to determine the second image evaluation model as the target image evaluation model if the similarity between the test evaluation result and the reference evaluation result is greater than or equal to a similarity threshold, and the reference evaluation result is the quality evaluation result of the test ultrasound image by medical personnel.

[0199] As an example of the present application, the first image evaluation model includes a feature extraction model and a classifier; the training module 802 includes:

[0200] A first calculation submodule, configured to perform forward calculation on the first image evaluation model based on the training sample set to determine a first loss value of the first image evaluation model;

[0201] A first updating submodule, configured to update the model parameters of the classifier by backpropagation based on the first loss value;

[0202] A second calculation submodule is used to perform forward calculation on the first image evaluation model again based on the training sample set to determine a second loss value of the first image evaluation model;

[0203] A second updating submodule, used for updating the model parameters of the feature extraction model and the model parameters of the classifier by back-propagation based on the second loss value;

[0204] The second trigger submodule is used to trigger the first calculation submodule to perform forward calculation on the first image evaluation model based on the training sample set to determine a first loss value of the first image evaluation model if the iterative training end condition is not currently met, until the iterative training end condition is currently met, thereby ending the iterative training and obtaining the second image evaluation model.

[0205] As an example of the present application, the first computing submodule is used for:

[0206] Inputting any image group in the training sample set into the first image evaluation model;

[0207] Processing any one of the image groups by using the first image evaluation model to obtain a quality prediction value corresponding to each sample ultrasound image in the any one of the image groups, wherein the quality prediction value is used to indicate an evaluation result of the first image evaluation model performing quality evaluation on the corresponding sample ultrasound image;

[0208] The first loss value is determined based on the quality prediction value corresponding to each sample ultrasound image in any one of the image groups through a target loss function.

[0209] As an example of the present application, the determining module 803 further includes:

[0210] A second determination submodule is used to determine the correlation coefficient between the test evaluation result and the reference evaluation result;

[0211] The third determination submodule is used to determine that the similarity between the test evaluation result and the reference evaluation result is greater than or equal to the similarity threshold when the absolute value of the correlation coefficient is greater than or equal to the coefficient threshold.

[0212] In the embodiment of the present application, since the target image evaluation model is obtained by iteratively training the first image evaluation model based on at least one image group included in the training sample set, and each sample ultrasound image in each image group is annotated with evaluation information after quality evaluation by medical personnel, that is, the target image evaluation model can learn the quality evaluation standards of medical personnel for ultrasound images, therefore, the target ultrasound image is quality evaluated by the target image evaluation model, thereby improving the accuracy of quality evaluation. In addition, since when sampling images during model training, the weights corresponding to each type of sample ultrasound image can be used, and the weight depends on the proportion of the number of each type of sample ultrasound image to the number of all sample ultrasound images, the lower the number of each type of sample ultrasound image, the higher the corresponding weight, and the higher the weight, the higher the probability of being sampled, thereby achieving focused training on a certain type of sample ultrasound image with a smaller number, alleviating the class imbalance problem existing in multiple sample ultrasound images.

[0213] Fig. 9 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Fig. 9 As shown, the electronic device 9 of this embodiment includes: at least one processor 90 ( Fig. 9 Only one is shown in the figure), a memory 91, and a computer program 92 stored in the memory 91 and executable on the at least one processor 90, wherein the processor 90 implements the steps of any of the above-mentioned method embodiments when executing the computer program 92.

[0214] The electronic device 9 may be a computing device such as an ultrasonic barcode scanning device, a desktop computer, a notebook, a PDA, or a cloud server. The electronic device may include, but is not limited to, a processor 90 and a memory 91. Those skilled in the art will appreciate that Fig. 9 It is only an example of the electronic device 9 and does not constitute a limitation on the electronic device 9. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0215] The processor 90 may be a CPU (Central Processing Unit), or other general-purpose processors, DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0216] In some embodiments, the memory 91 may be an internal storage unit of the electronic device 9, such as a hard disk or memory of the electronic device 9. In other embodiments, the memory 91 may also be an external storage device of the electronic device 9, such as a plug-in hard disk, SMC (Smart Media Card), SD (Secure Digital) card, Flash Card, etc. equipped on the electronic device 9. Further, the memory 91 may also include both an internal storage unit of the electronic device 9 and an external storage device. The memory 91 is used to store an operating system, an application program, a boot loader, data, and other programs, such as the program code of the computer program. The memory 91 may also be used to temporarily store data that has been output or is to be output.

[0217] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0218] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0219] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for evaluating the quality of an ultrasound image, characterized in that: The method comprises: Acquire a target ultrasonic image of the target part, wherein the target ultrasonic image is obtained by detecting the target part with an ultrasonic probe of an ultrasonic scanning device; Performing quality assessment on the target ultrasound image by using a target image assessment model, wherein the target image assessment model can assess the quality of any ultrasound image; The target image evaluation model is obtained by iteratively training the first image evaluation model based on the training sample set, the training sample set includes at least one image group, each image group includes multiple sample ultrasound images of different qualities, each sample ultrasound image is annotated with evaluation information after quality evaluation by medical personnel, the evaluation information includes the score of the corresponding sample ultrasound image scored by the medical personnel, and for sample ultrasound images with differences in scores, sample ultrasound images with differences greater than 2 points are eliminated, and the final score of the screened sample ultrasound images is the average score of the scores of N medical personnel, where N is a positive integer greater than 2; The process of iteratively training the first image evaluation model based on the training sample set includes: Based on the training sample set, forward computing is performed on the first image evaluation model to determine a first loss value of the first image evaluation model, wherein the first image evaluation model includes a feature extraction model and a classifier; Based on the first loss value, updating the model parameters of the classifier by backpropagation; In the process of updating the model parameters of the classifier, freezing the model parameters of the feature extraction model; Based on the training sample set, forward computing the first image evaluation model again to determine a second loss value of the first image evaluation model; Based on the second loss value, updating the model parameters of the feature extraction model and the model parameters of the classifier by means of back propagation; If the iterative training end condition is not met currently, returning to the operation of performing forward calculation on the first image evaluation model based on the training sample set to determine a first loss value of the first image evaluation model, until the iterative training end condition is met currently, the iterative training is ended, and a second image evaluation model is obtained; If the iterative training end condition is currently met, the target image evaluation model is determined based on the second image evaluation model.

2. A model training method, characterized in that: The method comprises: Acquire a training sample set, wherein the training sample set includes at least one image group, each image group includes a plurality of sample ultrasound images of different qualities, and each sample ultrasound image is annotated with evaluation information after quality evaluation by medical personnel; The evaluation information includes the scores of the corresponding sample ultrasound images scored by the medical staff, and for the sample ultrasound images with different scores, the sample ultrasound images with a difference greater than 2 points are eliminated, and the final score of the screened sample ultrasound images is the average score of the scores of N medical staff, where N is a positive integer greater than 2; Based on the training sample set, forward calculation is performed on a first image evaluation model to determine a first loss value of the first image evaluation model, wherein the first image evaluation model includes a feature extraction model and a classifier; Based on the first loss value, updating the model parameters of the classifier by backpropagation; In the process of updating the model parameters of the classifier, freezing the model parameters of the feature extraction model; Based on the training sample set, forward computing the first image evaluation model again to determine a second loss value of the first image evaluation model; Based on the second loss value, updating the model parameters of the feature extraction model and the model parameters of the classifier by means of back propagation; If the iterative training end condition is not met currently, returning to the operation of performing forward calculation on the first image evaluation model based on the training sample set to determine a first loss value of the first image evaluation model, until the iterative training end condition is met currently, the iterative training is ended, and a second image evaluation model is obtained; If the iterative training end condition is currently met, a target image evaluation model is determined based on the second image evaluation model, and the target image evaluation model can evaluate the quality of any ultrasound image.

3. The method according to claim 2, characterized in that The step of obtaining a training sample set includes: Acquire a plurality of sample ultrasound images, wherein the plurality of sample ultrasound images include multiple types of ultrasound images with different qualities; According to the number of each type of sample ultrasonic images among the multiple sample ultrasonic images, assigning corresponding weights to each type of sample ultrasonic images, wherein the number of each type of sample ultrasonic images is negatively correlated with the weights corresponding to each type of sample ultrasonic images; Based on the weights corresponding to the various types of sample ultrasound images, image sampling is performed from the multiple sample ultrasound images; When the number of samples reaches a number threshold, combining sample ultrasound images obtained by sampling the number threshold into an image group; The image sampling process is repeated until the plurality of sample ultrasound images are sampled, thereby obtaining the at least one image group.

4. The method according to claim 3, characterized in that The performing image sampling from the plurality of sample ultrasound images based on the weights corresponding to the various types of sample ultrasound images comprises: Determining the number of samples to be sampled from the various types of sample ultrasonic images according to the weights corresponding to the various types of sample ultrasonic images, wherein the weights corresponding to the various types of sample ultrasonic images are positively correlated with the number of samples corresponding to the various types of sample ultrasonic images; Image sampling is performed from the various types of sample ultrasonic images according to the sampling quantities corresponding to the various types of sample ultrasonic images.

5. The method according to any one of claims 2 to 4, characterized in that: If the iterative training end condition is currently met, determining the target image evaluation model based on the second image evaluation model includes: If the iterative training end condition is currently met, the quality of the test ultrasound image is evaluated by the second image evaluation model to obtain a test evaluation result; If the similarity between the test evaluation result and the reference evaluation result is greater than or equal to a similarity threshold, the second image evaluation model is determined as the target image evaluation model, and the reference evaluation result is a quality evaluation result of the test ultrasound image by medical personnel.

6. The method according to claim 2, characterized in that The performing forward calculation on the first image evaluation model based on the training sample set to determine a first loss value of the first image evaluation model includes: Inputting any image group in the training sample set into the first image evaluation model; Processing any one of the image groups by using the first image evaluation model to obtain a quality prediction value corresponding to each sample ultrasound image in the any one of the image groups, wherein the quality prediction value is used to indicate an evaluation result of the first image evaluation model performing quality evaluation on the corresponding sample ultrasound image; The first loss value is determined based on the quality prediction value corresponding to each sample ultrasound image in any one of the image groups through a target loss function.

7. The method according to claim 6, characterized in that The step of processing the arbitrary image group by using the first image evaluation model to obtain a quality prediction value corresponding to each sample ultrasound image in the arbitrary image group includes: Performing a feature extraction operation on each sample ultrasound image in any one of the image groups through a feature extraction module in the first image evaluation model to obtain feature information corresponding to each sample ultrasound image; The feature information corresponding to each sample ultrasound image is classified by the classifier to obtain a quality prediction value corresponding to each sample ultrasound image in any one of the image groups.

8. The method according to claim 5, characterized in that If the similarity between the test evaluation result and the reference evaluation result is greater than or equal to a similarity threshold, before determining the second image evaluation model as the target image evaluation model, the method further includes: determining a correlation coefficient between the test evaluation result and a reference evaluation result; In the case that the absolute value of the correlation coefficient is greater than or equal to the coefficient threshold, it is determined that the similarity between the test evaluation result and the reference evaluation result is greater than or equal to the similarity threshold.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to claim 1 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 2 to 8 is implemented.

Images