Online learning method and equipment for lesion prediction model

Abstract

The present invention provides an online learning method and device for a lesion prediction model. The method includes: obtaining an ultrasound image containing a breast lesion and the BI-RADS grade of the breast lesion; extracting multiple feature subvectors corresponding to BI-RADS features from the ultrasound image; determining a target feature subvector with annotation information from the multiple feature subvectors; determining a first loss function corresponding to the multiple feature subvectors; determining a second loss function corresponding to the target feature subvector; fusing the multiple feature subvectors to obtain a fused feature vector, and determining a third loss function corresponding to the fused feature vector based on the obtained BI-RADS grade; and online training a breast lesion prediction model based on the first loss function, the second loss function, and the third loss function. The method implements online training of a breast lesion prediction model based on ultrasound images with missing annotations, shortens the model update cycle, and simplifies the update operation steps.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of medical ultrasound technology, and specifically to an online learning method and device for a lesion prediction model. Background Art

[0002] Breast cancer is a malignant tumor that occurs in the epithelial tissue of the breast. According to cancer statistics, breast cancer ranks first in the incidence of malignant tumors in women. Therefore, early screening for breast cancer is particularly important. Breast ultrasound images can clearly show the location, morphology, internal structure and changes in adjacent tissues of the soft tissues of the breast and the lesions therein. It has many advantages such as being economical, convenient, non-invasive, painless, non-radioactive and highly repeatable. It has become one of the important methods of breast examination. The most widely used and relatively authoritative diagnostic standard in clinical diagnosis is the Breast Imaging Reporting and Data System (BI-RADS) proposed by the American College of Radiology (ACR). BI-RADS uses unified and professional terminology to classify the characteristics and grades of breast lesions.

[0003] In clinical practice, when different doctors analyze the BI-RADS features and BI-RADS grades of breast lesions, there is often a large degree of subjectivity involved, resulting in inconsistent diagnostic results from different doctors, and even inconsistent diagnostic results from the same doctor at different time periods. With the continuous development of computer science and technology, computer-aided diagnosis (CAD) systems are gradually being used to perform intelligent diagnosis on breast ultrasound images. This not only reduces the workload of doctors and improves their work efficiency, but also effectively reduces the differences in diagnoses between different doctors or the same doctor at different times. After the CAD system is installed on the ultrasound equipment, the breast lesion prediction model in the CAD system is usually updated by replacing the software in the ultrasound equipment. This update method not only has a long cycle but is also more cumbersome to operate. Summary of the Invention

[0004] The embodiments of the present invention provide an online learning method and device for a lesion prediction model, which are used to solve the problems of long update cycle and complicated operation of breast lesion prediction models in existing methods.

[0005] In a first aspect, an embodiment of the present invention provides an online learning method for a lesion prediction model, comprising:

[0006] Obtain ultrasound images containing breast lesions and BI-RADS grades of breast lesions;

[0007] Extracting multiple feature subvectors corresponding to BI-RADS features from the ultrasound image, where the BI-RADS features include at least two of a shape feature, a direction feature, an edge feature, an internal echo feature, a rear echo feature, a calcification feature, and a blood flow feature;

[0008] Determining a target feature subvector with labeling information from the plurality of feature subvectors, wherein the labeling information indicates descriptive information of a type of BI-RADS feature indicating the presence of a breast lesion;

[0009] Determine a first loss function corresponding to the plurality of feature sub-vectors;

[0010] Determine a second loss function corresponding to the target feature subvector;

[0011] Fusing multiple feature sub-vectors to obtain a fused feature vector, and determining a third loss function corresponding to the fused feature vector according to the obtained BI-RADS grade;

[0012] The breast lesion prediction model is trained online according to the first loss function, the second loss function and the third loss function. The breast lesion prediction model is used to process and analyze the ultrasound images containing breast lesions to be analyzed to obtain the type and BI-RADS grade of the breast lesions.

[0013] In a second aspect, an embodiment of the present invention provides an online learning method for a lesion prediction model, comprising:

[0014] Obtain ultrasound images containing breast lesions and BI-RADS grades of breast lesions;

[0015] Extracting multiple feature subvectors corresponding to BI-RADS features from the ultrasound image, where the BI-RADS features include at least two of a shape feature, a direction feature, an edge feature, an internal echo feature, a rear echo feature, a calcification feature, and a blood flow feature;

[0016] Determine a first loss function corresponding to the plurality of feature sub-vectors;

[0017] Fusing multiple feature sub-vectors to obtain a fused feature vector, and determining a third loss function corresponding to the fused feature vector according to the obtained BI-RADS grade;

[0018] The breast lesion prediction model is trained online according to the first loss function and the third loss function. The breast lesion prediction model is used to process and analyze the ultrasound image containing the breast lesion to be analyzed to obtain the type and BI-RADS grade of the breast lesion.

[0019] In a third aspect, an embodiment of the present invention provides an online learning method for a lesion prediction model, comprising:

[0020] Acquiring an ultrasound image containing a thyroid lesion and the TI-RADS grade of the thyroid lesion;

[0021] Extracting a plurality of feature subvectors corresponding to TI-RADS features from the ultrasound image, wherein the TI-RADS features include at least two of a component feature, an echo feature, a shape feature, an edge feature, and a focal hyperechoic feature;

[0022] Determine a target feature subvector having annotation information from the multiple feature subvectors, wherein the annotation information indicates descriptive information of a type of the TI-RADS feature of the thyroid lesion;

[0023] Determining a first loss function corresponding to the plurality of feature sub-vectors;

[0024] Determining a second loss function corresponding to the target feature subvector;

[0025] fusing the multiple feature sub-vectors to obtain a fused feature vector, and determining a third loss function corresponding to the fused feature vector according to the obtained TI-RADS grade;

[0026] The thyroid lesion prediction model is trained online according to the first loss function, the second loss function and the third loss function. The thyroid lesion prediction model is used to process and analyze the ultrasound image containing the thyroid lesion to be analyzed to obtain the type and TI-RADS grade of the thyroid lesion's TI-RADS features.

[0027] In a fourth aspect, an embodiment of the present invention provides an online learning method for a lesion prediction model, comprising:

[0028] Acquiring an ultrasound image containing a thyroid lesion and the TI-RADS grade of the thyroid lesion;

[0029] Extracting a plurality of feature subvectors corresponding to TI-RADS features from the ultrasound image, wherein the TI-RADS features include at least two of a component feature, an echo feature, a shape feature, an edge feature, and a focal hyperechoic feature;

[0030] Determining a first loss function corresponding to the plurality of feature sub-vectors;

[0031] fusing the multiple feature sub-vectors to obtain a fused feature vector, and determining a third loss function corresponding to the fused feature vector according to the obtained TI-RADS grade;

[0032] The thyroid lesion prediction model is trained online according to the first loss function and the third loss function. The thyroid lesion prediction model is used to process and analyze the ultrasound image containing the thyroid lesion to be analyzed to obtain the type and TI-RADS grade of the thyroid lesion's TI-RADS features.

[0033] In a fifth aspect, an embodiment of the present invention provides an ultrasonic imaging device, comprising:

[0034] Ultrasound probe;

[0035] The transmitting circuit is used to output the corresponding transmitting sequence to the ultrasonic probe according to the set mode, so as to control the ultrasonic probe to transmit the corresponding ultrasonic wave;

[0036] A receiving circuit, used for receiving the ultrasonic echo signal output by the ultrasonic probe and outputting ultrasonic echo data;

[0037] A display for outputting visual information;

[0038] A processor is used to execute the online learning method of the lesion prediction model as described in any one of the first to fourth aspects above.

[0039] In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement an online learning method for a lesion prediction model as described in any one of the first to fourth aspects above.

[0040] The online learning method and device for a lesion prediction model provided by an embodiment of the present invention obtains an ultrasound image containing a breast lesion and the BI-RADS grade of the breast lesion; extracts multiple feature subvectors corresponding to BI-RADS features from the ultrasound image; determines a target feature subvector with labeled information from the multiple feature subvectors; determines a first loss function corresponding to the multiple feature subvectors; determines a second loss function corresponding to the target feature subvector; fuses the multiple feature subvectors to obtain a fused feature vector, and determines a third loss function corresponding to the fused feature vector based on the acquired BI-RADS grade; and performs online training of the breast lesion prediction model based on the first, second, and third loss functions. This method implements online training of a breast lesion prediction model based on ultrasound images with missing annotations, shortens the model update cycle, and simplifies the update operation steps. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A structural block diagram of an ultrasonic imaging device provided in one embodiment of the present invention;

[0042] Figure 2A flowchart of an online learning method for a lesion prediction model provided by one embodiment of the present invention;

[0043] Figure 3 A schematic diagram of the process of online training of a lesion prediction model provided by one embodiment of the present invention;

[0044] Figure 4 A flowchart of an online learning method for a lesion prediction model provided in yet another embodiment of the present invention. DETAILED DESCRIPTION

[0045] The present invention will be further described in detail below by means of specific embodiments in conjunction with the accompanying drawings. Similar elements in different embodiments are numbered with associated similar elements. In the following embodiments, many detailed descriptions are provided to enable the present application to be better understood. However, those skilled in the art will readily appreciate that some of the features may be omitted in different circumstances, or may be replaced by other elements, materials, or methods. In some cases, some operations related to the present application are not shown or described in the specification. This is to avoid the core portion of the present application being overwhelmed by excessive descriptions, and for those skilled in the art, it is not necessary to describe these related operations in detail. They will fully understand the related operations based on the description in the specification and the general technical knowledge in the art.

[0046] In addition, the features, operations, or characteristics described in the specification may be combined in any appropriate manner to form various embodiments. Furthermore, the steps or actions in the method description may be reordered or adjusted in a manner readily apparent to those skilled in the art. Therefore, the various sequences in the specification and drawings are provided solely for the purpose of clearly describing a particular embodiment and are not intended to be mandatory, unless otherwise specified.

[0047] The serial numbers assigned to components herein, such as "first," "second," etc., are used solely to distinguish the objects being described and do not convey any sequential or technical meaning. References to "connection" and "coupling" herein, unless otherwise specified, include both direct and indirect connections (couplings).

[0048] like Figure 1 As shown, the ultrasound imaging device provided by the present invention may include: an ultrasound probe 20, a transmitting / receiving circuit 30 (i.e., a transmitting circuit 310 and a receiving circuit 320), a beamforming module 40, an IQ demodulation module 50, a memory 60, a processor 70, and a human-computer interaction device. The processor 70 may include a control module 710 and an image processing module 720.

[0049] The ultrasound probe 20 includes a transducer (not shown) composed of multiple array elements arranged in an array. The array elements can be arranged in a row to form a linear array, or in a two-dimensional matrix to form a planar array. The array elements can also form a convex array. The array elements are used to transmit ultrasonic beams based on excitation electrical signals, or to convert received ultrasonic beams into electrical signals. Therefore, each array element can be used to convert electrical pulse signals into and out of ultrasonic beams, thereby transmitting ultrasonic waves to a target area of human tissue (e.g., a breast area containing a breast lesion or a thyroid area containing a thyroid lesion in this embodiment) and receiving echoes of ultrasonic waves reflected from the tissue. During ultrasonic testing, the transmitting circuit 310 and the receiving circuit 320 can control which array elements are used to transmit and which are used to receive ultrasonic beams, or control the time slots used to transmit and receive ultrasonic beams. The array elements involved in ultrasonic transmission can be excited by electrical signals simultaneously, thereby transmitting ultrasonic waves simultaneously; or they can be excited by multiple electrical signals with a certain time interval, thereby continuously transmitting ultrasonic waves with a certain time interval.

[0050] In this embodiment, the user moves the ultrasound probe 20 to select a suitable position and angle to transmit ultrasound to the breast or thyroid region 10 and receives the echo of the ultrasound returned by the breast or thyroid region 10, obtains and outputs the electrical signal of the echo, and the electrical signal of the echo is a channel analog electrical signal formed by the receiving array element as a channel, which carries amplitude information, frequency information and time information.

[0051] The transmitting circuit 310 is configured to generate a transmit sequence under the control of the control module 710 of the processor 70. The transmit sequence is used to control some or all of the multiple array elements to transmit ultrasound waves toward biological tissue. Transmit sequence parameters include the array element positions, the number of array elements, and ultrasound beam transmission parameters (e.g., amplitude, frequency, number of transmissions, transmission interval, transmission angle, waveform, focal position, etc.). In some cases, the transmitting circuit 310 is further configured to phase-delay the transmitted beam so that different transmitting array elements transmit ultrasound waves at different times, allowing each transmitted ultrasound beam to be focused on a predetermined region of interest. Transmit sequence parameters may vary for different operating modes, such as B-image mode, C-image mode, and D-image mode (Doppler mode). After the echo signals are received by the receiving circuit 320 and processed by subsequent modules and corresponding algorithms, a B image reflecting tissue anatomical structure, a C image reflecting tissue anatomical structure and blood flow information, and a D image reflecting Doppler spectrum images can be generated.

[0052] The receiving circuit 320 is used to receive and process the electrical signals of ultrasonic echoes from the ultrasonic probe 20. The receiving circuit 320 may include one or more amplifiers, analog-to-digital converters (ADCs), and other components. The amplifiers are used to amplify the received electrical signals of ultrasonic echoes after appropriate gain compensation, and the ADCs are used to sample the analog echo signals at predetermined intervals, converting them into digitized signals. The digitized echo signals still retain amplitude, frequency, and phase information. The data output by the receiving circuit 320 can be sent to the beamforming module 40 for processing or to the memory 60 for storage.

[0053] The beamforming module 40 is signal-connected to the receiving circuit 320 and is used to perform beamforming processing, such as delay and weighted summation, on the signal output by the receiving circuit 320. Because the distances between the ultrasound receiving points in the measured tissue and the receiving array elements vary, the channel data of the same receiving point output by different receiving array elements have different delays. This requires delay processing, phase alignment, and weighted summation of the different channel data from the same receiving point to obtain beamformed ultrasound image data. The ultrasound image data output by the beamforming module 40 is also called radio frequency data (RF data). The beamforming module 40 outputs the RF data to the IQ demodulation module 50. In some embodiments, the beamforming module 40 may also output the RF data to the memory 60 for caching or storage, or directly output the RF data to the image processing module 720 of the processor 70 for image processing.

[0054] The beamforming module 40 may perform the aforementioned functions in hardware, firmware, or software. For example, the beamforming module 40 may include a central controller circuit (CPU), one or more microprocessor chips, or any other electronic components capable of processing input data according to specific logic instructions. When the beamforming module 40 is implemented in software, it may execute instructions stored on a tangible and non-transitory computer-readable medium (e.g., the memory 60) to perform beamforming calculations using any appropriate beamforming method.

[0055] The IQ demodulation module 50 removes the signal carrier through IQ demodulation, extracts the tissue structure information contained in the signal, and performs filtering to remove noise. The resulting signal is called a baseband signal (IQ data pair). The IQ demodulation module 50 outputs the IQ data pair to the image processing module 720 of the processor 70 for image processing. In some embodiments, the IQ demodulation module 50 also outputs the IQ data pair to the memory 60 for caching or storage, so that the image processing module 720 can read the data from the memory 60 for subsequent image processing.

[0056] The processor 70 is configured to be a central control circuit (CPU), one or more microprocessors, a graphics controller circuit (GPU) or any other electronic component that can process input data according to specific logical instructions. It can control peripheral electronic components according to input instructions or predetermined instructions, or read and / or save data from the memory 60. It can also process the input data by executing the program in the memory 60, for example, performing one or more processing operations on the collected ultrasound data according to one or more working modes. The processing operations include but are not limited to adjusting or limiting the form of ultrasound waves emitted by the ultrasound probe 20, generating various image frames for subsequent display on the display 80 of the human-computer interaction device, or adjusting or limiting the content and form displayed on the display 80, or adjusting one or more image display settings displayed on the display 80 (such as ultrasound images, interface components, and positioning areas of interest).

[0057] Image processing module 720 processes the data output by beamforming module 40 or IQ demodulation module 50 to generate a grayscale image showing signal strength variations within the scanning range. This grayscale image reflects the internal anatomical structure of the tissue, referred to as a B-image. Image processing module 720 can output the B-image to display on display 80 of the human-computer interaction device.

[0058] The human-computer interaction device is used for human-computer interaction, that is, receiving user input and outputting visual information; it can receive user input using a keyboard, operation buttons, mouse, trackball, etc., or a touch screen integrated with a display; it outputs visual information using a display 80.

[0059] The memory 60 can be a tangible and non-transitory computer-readable medium, such as a flash memory card, a solid-state memory, a hard disk, etc., for storing data or programs. For example, the memory 60 can be used to store the acquired ultrasound data or image frames generated by the processor 70 that are not immediately displayed, or the memory 60 can store a graphical user interface, one or more default image display settings, and programming instructions for the processor, the beamforming module, or the IQ decoding module.

[0060] The ultrasound imaging device provided in this embodiment can be equipped with a CAD system, which uses a breast lesion prediction model to process and analyze ultrasound images containing breast lesions, and outputs the type of BI-RADS features and BI-RADS grade of the breast lesions. Before the CAD system is installed on the ultrasound imaging device, it is necessary to complete the initial training of the breast lesion prediction model based on the training set. Before performing the initial training, it is necessary for a senior doctor to complete the annotation of the type of BI-RADS features and BI-RADS grade of the sample ultrasound images in the training set. It is understandable that the CAD system can use a thyroid lesion prediction model to process and analyze ultrasound images containing thyroid lesions, and output the type of TI-RADS features and TI-RADS grade of the thyroid lesions. Before the CAD system is installed on the ultrasound imaging device, it is necessary to complete the initial training of the thyroid lesion prediction model based on the training set. Before performing the initial training, it is necessary for a senior doctor to complete the annotation of the type of TI-RADS features and TI-RADS grade of the sample ultrasound images in the training set.

[0061] It should be noted that Figure 1 The structure shown is for illustration only and may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure may be implemented using hardware and / or software. Figure 1 The ultrasonic imaging device shown can be used to execute the online learning method of the breast lesion prediction model provided by any embodiment of the present invention.

[0062] At present, the updating of breast lesion prediction model and / or thyroid lesion prediction model in CAD system is often carried out by replacing software, which is not only long in cycle but also cumbersome in operation. In order to shorten the cycle and simplify the operation, the breast lesion prediction model and / or thyroid lesion prediction model are updated by online learning in this application. For example, a switch for turning on the online learning function can be set in the CAD system. When the online learning function of the system is activated, the breast lesion prediction model and / or thyroid lesion prediction model are trained online based on the training samples. A large amount of high-end medical resources will be consumed for the annotation of the training samples, and the manpower cost is high. A large number of ultrasonic images containing breast lesions or thyroid lesions will be produced clinically. If the online training of the breast lesion prediction model and / or thyroid lesion prediction model can be completed based on the ultrasonic images produced clinically, not only can the cost be greatly reduced, but also the breast lesion prediction model and / or thyroid lesion prediction model obtained after training can be made more adaptable to the data collection style and data distribution state of the current hospital. However, in actual clinical procedures, when doctors view ultrasound images containing breast lesions or thyroid lesions, they often only provide information on the BI-RADS classification or TI-RADS classification, and do not have much time and energy to provide information on the types of all BI-RADS or TI-RADS features. That is to say, the ultrasound images obtained in actual clinical practice may only have BI-RADS classification or TI-RADS classification information, or have BI-RADS classification or TI-RADS classification information and information on the types of some BI-RADS features or TI-RADS features. How to complete the online optimization of the breast lesion prediction model based on the incompletely labeled ultrasound images obtained in actual clinical practice has important research value. This application will elaborate on how to achieve online learning for the case of having BI-RADS classification or TI-RADS classification information and information on the types of some BI-RADS features or TI-RADS features and the case of only having BI-RADS classification or TI-RADS classification information, respectively.

[0063] The following is a detailed explanation using breast BI-RADS as an example. The relevant explanation of thyroid TI-RADS can be understood by referring to breast BI-RADS, which will not be repeated in this application.

[0064] Please refer to Figure 2 The online learning method of the lesion prediction model provided by one embodiment of the present invention may include:

[0065] S201. Acquire an ultrasound image containing a breast lesion and the BI-RADS grade of the breast lesion.

[0066] The ultrasound images acquired in this embodiment can be from actual clinical processes, and each ultrasound image has corresponding BI-RADS grading information. The ultrasound probe of the ultrasound imaging device can transmit ultrasound waves to the breast region containing the breast lesion and receive ultrasound echoes returned from the breast region to obtain ultrasound echo data. Ultrasound images containing the breast lesions can be generated in real time based on the ultrasound echo data. Specifically, the doctor can apply a coupling agent to the fully exposed skin surface of the subject's breast and then hold the ultrasound probe close to the patient's breast skin for scanning. Alternatively, a pre-stored ultrasound image containing the breast lesion can be obtained from a storage device.

[0067] In one optional embodiment, when the CAD system's online learning function is enabled, ultrasound images containing breast lesions and the BI-RADS classification of breast lesions can be acquired in real time. In another optional embodiment, to reduce the impact of online learning on the physician's normal work, ultrasound images containing breast lesions and the BI-RADS classification of breast lesions can be acquired within a preset time period. The preset time period can be the physician's non-working time period. The preset time period can be set by the physician.

[0068] S202. Extract multiple feature subvectors corresponding to BI-RADS features from the ultrasound image, where the BI-RADS features include at least two of shape features, direction features, edge features, internal echo features, rear echo features, calcification features, and blood flow features.

[0069] In this embodiment, multiple feature subvectors corresponding to BI-RADS features can be extracted from ultrasound images based on traditional image processing methods or convolutional neural network models. For example, the grayscale value of the ultrasound image can be calculated first, and then BI-RADS features can be extracted from the ultrasound image using operators such as Harris, SIFT, SURF, LBF, HOG, DPM, and ORB. The extracted BI-RADS features can be grouped into BI-RADS feature groups to obtain multiple corresponding feature subvectors. Alternatively, BI-RADS features can be extracted from ultrasound images based on pre-trained models such as VGG, ResNet, DenseNet, ShuffleNet, SENet, and EfficientNet. The extracted BI-RADS features can be grouped into BI-RADS feature groups to obtain multiple corresponding feature subvectors. The BI-RADS features in this embodiment can include at least two of the following: shape features, direction features, edge features, internal echo features, posterior echo features, calcification features, and blood flow features. The TI-RADS features in this embodiment can include at least two of the following: component features, echo features, shape features, edge features, and focal hyperechoic features. For example, feature subvectors corresponding to shape features and feature subvectors corresponding to directional features can be extracted from ultrasound images; feature subvectors corresponding to shape features, feature subvectors corresponding to directional features, and feature subvectors corresponding to rear echo features can also be extracted from ultrasound images; all feature subvectors corresponding to all BI-RADS features can also be extracted from ultrasound images.

[0070] S203: Determine a target feature sub-vector with labeling information from the multiple feature sub-vectors, wherein the labeling information indicates descriptive information of the type of BI-RADS feature indicating the presence of a breast lesion.

[0071] Considering that the ultrasound images obtained in actual clinical practice may only have information about some BI-RADS feature types, in this embodiment, after extracting the feature sub-vectors, it is also necessary to determine which feature sub-vectors have descriptive information about the corresponding BI-RADS feature types and which feature sub-vectors lack descriptive information about the BI-RADS feature types.

[0072] Ultrasound images collected in actual clinical practice usually have corresponding diagnostic information, and the diagnostic information is determined based on the doctor's input or obtained by the doctor modifying the output of the breast lesion prediction model. For example, the doctor can select the type corresponding to the current breast lesion from all types of BI-RADS features through a drop-down menu, or can enter descriptive information of the type of BI-RADS feature of the current breast lesion through a text box. In an optional embodiment, determining a target feature subvector with annotation information from multiple feature subvectors can specifically include: obtaining diagnostic information of the breast lesion, the diagnostic information is determined based on the doctor's input or obtained by the doctor modifying the output of the breast lesion prediction model; obtaining the type of BI-RADS feature of the breast lesion contained in the diagnostic information by performing natural language processing on the diagnostic information; matching the type of BI-RADS feature contained in the diagnostic information with each feature subvector, and determining the feature subvector that matches the type of BI-RADS feature as the target feature subvector.

[0073] S204: Determine a first loss function corresponding to the plurality of feature sub-vectors.

[0074] In this embodiment, after obtaining multiple feature sub-vectors, the first loss functions corresponding to the multiple feature sub-vectors can be determined based on an unsupervised clustering method.

[0075] In an optional embodiment, the first loss function may be determined based on the correlation within each feature sub-vector and the correlation between multiple feature sub-vectors. group It can be determined according to the following expression:

[0076] L group =(1-mean(D intra ))+mean(D inter )

[0077] Among them, D intra Represents the correlation matrix within the same set of feature sub-vectors, D inter Represents the correlation matrix between different groups of feature sub-vectors, and mean() represents the average value of the elements in the matrix. The correlation matrix D is defined as follows:

[0078] D=bmm(norm(F),norm(F) T )

[0079] Among them, F represents the extracted feature sub-vector, the size is (batch, channels, w*h), norm(F) represents the normalization of the feature sub-vector, norm(F) TIt is the transposed matrix of norm(F), of size (batch,w*h,channels), and bmm() represents the batch-based matrix product.

[0080] S205: Determine a second loss function corresponding to the target feature sub-vector.

[0081] In this embodiment, after determining the target feature subvector labeled with the type of BI-RADS feature, the second loss function corresponding to the target feature subvector can also be determined based on methods such as mean square loss, cross entropy loss, and focal loss according to the type of the labeled BI-RADS feature and the type of BI-RADS feature output by the breast lesion prediction model.

[0082] In an optional embodiment, the second loss function L corresponding to the target feature subvector is features ′ can be determined according to the following expression:

[0083]

[0084] Where y′ represents the prediction vector of the BI-RADS feature output by the breast lesion prediction model, y represents the annotation information of the target feature subvector after conversion to the one-hot encoding format, and the value range of f′ corresponds to the number of target feature subvectors with annotation information.

[0085] S206 , fusing multiple feature sub-vectors to obtain a fused feature vector, and determining a third loss function corresponding to the fused feature vector according to the obtained BI-RADS grade.

[0086] In this embodiment, after extracting multiple feature sub-vectors corresponding to BI-RADS features from the ultrasound image, a fused feature vector corresponding to the BI-RADS grade can be obtained by fusing the multiple feature sub-vectors. In an optional implementation, the fused feature vector can be obtained by fusing multiple feature sub-vectors by feature splicing or feature addition. Then, the third loss function corresponding to the fused feature vector is determined based on the BI-RADS grade obtained in step S201 and the BI-RADS grade output by the breast lesion prediction model. Optionally, the third loss function L corresponding to the fused feature vector birads It can be determined according to the following expression:

[0087]

[0088] The value range of i is (0, 5), which corresponds to 2, 3, 4A, 4B, 4C and 5 in the BI-RADS classification. i′ represents the prediction vector of BI-RADS grade output by the breast lesion prediction model, and each value represents the probability of belonging to a certain BI-RADS grade, y i The vector corresponding to the BI-RADS grade obtained in step S201 using the one-hot encoding format is represented as follows: BI-RADS grade 2 corresponds to [1, 0, 0, 0, 0], BI-RADS grade 3 corresponds to [0, 1, 0, 0, 0], BI-RADS grade 4a corresponds to [0, 0, 1, 0, 0, 0], BI-RADS grade 4b corresponds to [0, 0, 0, 1, 0, 0], BI-RADS grade 4c corresponds to [0, 0, 0, 0, 1, 0], and BI-RADS grade 5 corresponds to [0, 0, 0, 0, 0, 1].

[0089] S207. Online training of a breast lesion prediction model is performed according to the first loss function, the second loss function, and the third loss function. The breast lesion prediction model is used to process and analyze the ultrasound image containing the breast lesion to be analyzed to obtain the type of BI-RADS features and the BI-RADS grade of the breast lesion.

[0090] In this embodiment, after obtaining the first loss function, the second loss function, and the third loss function, the breast lesion prediction model can be trained online based on the first loss function, the second loss function, and the third loss function. The target loss function can be determined based on the first loss function, the second loss function, and the third loss function, and the breast lesion prediction model can be trained online with the goal of minimizing the target loss function. The target loss function can, for example, be the weighted sum of the first loss function, the second loss function, and the third loss function. Optionally, the target loss function L′ can be determined according to the following expression:

[0091] L′=α*L group +β*L features ′+γ*L birads

[0092] Among them, α, β, and γ are balance factors that can be used to balance the loss functions of each part. They can be determined based on experimental results, for example, they can be constants between 0 and 1. group Represents the first loss function, L features ′ represents the second loss function, L birads represents the third loss function.

[0093] The breast lesion prediction model is a model for processing and analyzing ultrasound images containing breast lesions to be analyzed to obtain the type of BI-RADS features and BI-RADS grading of the breast lesions. It is understandable that before online training is performed on it, or in other words, before use, it is necessary to complete the initial training of the breast lesion prediction model based on the training samples in the training set. The training samples have annotation information of the BI-RADS grading and the types of all BI-RADS features. Optionally, the breast lesion prediction model can be initially trained based on the training samples in the training set according to the following method:

[0094] First, feature extraction is performed on the training samples, and feature subvectors corresponding to each BI-RADS feature are extracted from the training samples, and the first loss function L provided in step S204 is used. group The expression of determines the value of the first loss function. Then, according to the annotation information of the training sample and the prediction information output by the breast lesion prediction model, the second loss function L is determined. features And the third loss function L birads The value of .

[0095]

[0096] The value range of f is (0,6), corresponding to shape features, direction features, edge features, internal echo features, rear echo features, calcification features, and blood flow features. y′ represents the prediction vector for the type of BI-RADS feature output by the breast lesion prediction model, and y represents the annotation information of the training sample in one-hot encoding format. L birads The expression provided in step S206 can be used for determination. Finally, the target loss function L is determined according to the following expression and iterative training is performed with the goal of minimizing the target loss function:

[0097] L=α*L group +β*L features +γ*L birads

[0098] Among them, α, β, and γ are balancing factors used to balance the losses of each part. During the training process, the classification accuracy of the breast lesion prediction model is tested on the validation set. When the classification accuracy on the validation set remains stable or the number of training times reaches a preset value, the initial training of the breast lesion prediction model is completed. In one embodiment, it is possible to determine whether the classification accuracy obtained in this training is approaching stability based on the difference between the classification accuracy obtained in this training and the classification accuracy obtained in at least one previous iterative training. For example, if the difference is less than a preset value, it is determined that the classification accuracy obtained in this training is approaching stability.

[0099] The online learning method of the lesion prediction model provided in this embodiment obtains an ultrasound image containing a breast lesion and the BI-RADS grade of the breast lesion; extracts multiple feature subvectors corresponding to the BI-RADS features from the ultrasound image; determines a target feature subvector with labeled information from the multiple feature subvectors; determines a first loss function corresponding to the multiple feature subvectors; determines a second loss function corresponding to the target feature subvector; fuses the multiple feature subvectors to obtain a fused feature vector, and determines a third loss function corresponding to the fused feature vector based on the obtained BI-RADS grade; and performs online training on the breast lesion prediction model based on the first loss function, the second loss function, and the third loss function. This method realizes online training of the breast lesion prediction model based on ultrasound images with missing annotations, shortens the model update cycle, and simplifies the update operation steps. Moreover, optimizing and updating the breast lesion prediction model using ultrasound images collected in actual clinical practice can not only make the model more adaptable to the current hospital's data collection style and data distribution status, but also make the model's output more consistent with the current hospital's diagnostic style. The relevant description and effects of thyroid TI-RADS can be understood with reference to breast BI-RADS, which will not be elaborated here.

[0100] To further ensure that the breast lesion prediction model is updated in a manner that improves classification accuracy, the model's classification accuracy needs to be tested on a test set. Only when the classification accuracy of the updated model is greater than that of the original model is the updated model used to replace the original model. During online training, if the classification accuracy is calculated once for each iteration, the training efficiency of the model will be greatly affected. Therefore, based on the above embodiment, in order to balance classification accuracy and training efficiency, the method provided in this embodiment may further include: performing online training on the breast lesion prediction model for a preset number of times according to the first loss function, the second loss function, and the third loss function to obtain a new breast lesion prediction model; determining the classification accuracy of the breast lesion prediction model and the new breast lesion prediction model based on the same test set; and updating the breast lesion prediction model to the new breast lesion prediction model when the classification accuracy of the new breast lesion prediction model is greater than that of the original breast lesion prediction model. The specific value of the preset number of times can be set according to actual needs. When the CAD system is sensitive to classification accuracy, a smaller number of times can be set; when the CAD system is sensitive to computational complexity, a larger number of times can be set. Assuming that 500 ultrasound images containing breast lesions are collected for online training of a breast lesion prediction model, the model classification accuracy can be determined on the test set after every 10 iterative training, or the model classification accuracy can be determined on the training set after 100 iterative training, so as to determine whether to replace the original model based on the classification accuracy.

[0101] Based on the above embodiments, the following further describes how to extract multiple feature subvectors corresponding to BI-RADS features from ultrasound images. In an optional embodiment, extracting multiple feature subvectors corresponding to BI-RADS features from an ultrasound image may specifically include: extracting a feature vector from an ultrasound image, dividing the extracted feature vector into multiple feature subvectors, each feature subvector corresponding to a BI-RADS feature. For example, feature vectors can be extracted from an ultrasound image based on a convolutional neural network ResNet50, and then the extracted features are feature grouped to obtain feature subvectors corresponding to each BI-RADS feature. Optionally, the extracted feature vector can be divided into five feature subvectors, corresponding to shape features, direction features, edge features, internal echo features, and rear echo features, respectively.

[0102] Please refer to Figure 3 , Figure 3 The process of online training of breast lesion prediction model is shown. Figure 3 As shown in the figure, a convolutional neural network model (CNN) is first used to extract feature vectors from ultrasound images containing breast lesions. The extracted feature vectors are then divided into five feature sub-vectors, corresponding to shape features, direction features, edge features, internal echo features, and rear echo features, respectively. Based on the diagnostic information, the type of shape feature is determined to be elliptical, the type of direction feature is parallel, the type of edge feature is smooth, the type of internal echo feature is anechoic, and the rear echo feature does not have annotation information. That is, the target feature sub-vector in this example includes the feature sub-vectors corresponding to the shape feature, direction feature, edge feature, and internal echo feature. Finally, the five feature sub-vectors are fused to obtain a fused feature vector corresponding to the BI-RADS grade. In this example, the BI-RADS grade is 3. Based on the five feature sub-vectors, the four target feature sub-vectors, and the fused feature vector, online training of the breast lesion prediction model can be achieved.

[0103] In another optional embodiment, extracting multiple feature subvectors corresponding to BI-RADS features from an ultrasound image can specifically include: using multiple pre-trained feature extraction convolutional neural network models to extract multiple feature subvectors corresponding to BI-RADS features from the ultrasound image, each feature extraction convolutional neural network model is used to extract a feature subvector corresponding to a BI-RADS feature from the ultrasound image. For example, a shape feature extraction convolutional neural network model can be pre-trained to extract feature subvectors corresponding to shape features, a directional feature extraction convolutional neural network model is used to extract feature subvectors corresponding to directional features, an edge feature extraction convolutional neural network model is used to extract feature subvectors corresponding to edge features, an internal echo feature extraction convolutional neural network model is used to extract feature subvectors corresponding to internal echo features, a rear echo feature extraction convolutional neural network model is used to extract feature subvectors corresponding to rear echo features, a calcification feature extraction convolutional neural network model is used to extract feature subvectors corresponding to calcification features, and a blood flow feature extraction convolutional neural network model is used to extract feature subvectors corresponding to blood flow features. In this embodiment, after the feature extraction convolutional neural network model is trained, the feature subvectors corresponding to each BI-RADS feature can be directly obtained from the ultrasound image without further vector segmentation.

[0104] The above embodiment explains how to realize online learning of breast lesion prediction model in the case of information with BI-RADS grading information and partial BI-RADS feature types. In actual clinical practice, there are some ultrasound images containing breast lesions that only have BI-RADS grading information. The following embodiment will explain how to realize online learning of breast lesion prediction model in the case of only BI-RADS grading information. It should be noted that the following embodiment is not only applicable to the case of only having BI-RADS grading information, but also to the case of having BI-RADS grading information and partial BI-RADS feature types. In this case, it can be understood that only the BI-RADS grading information is used to realize online learning of breast lesion prediction model. Please refer to Figure 4 The online learning method of the lesion prediction model provided in this embodiment may include:

[0105] S401. Acquire an ultrasound image containing a breast lesion and the BI-RADS grade of the breast lesion.

[0106] The specific implementation method can be referred to S201 and will not be repeated here.

[0107] S402. Extract multiple feature subvectors corresponding to BI-RADS features from the ultrasound image, where the BI-RADS features include at least two of shape features, direction features, edge features, internal echo features, rear echo features, calcification features, and blood flow features.

[0108] The specific implementation method can be referred to S202 and will not be repeated here.

[0109] S403: Determine a first loss function corresponding to the plurality of feature sub-vectors.

[0110] The specific implementation method can be referred to S204 and will not be described in detail here.

[0111] S404 , fusing multiple feature sub-vectors to obtain a fused feature vector, and determining a third loss function corresponding to the fused feature vector according to the obtained BI-RADS grade.

[0112] The specific implementation method can be referred to S206 and will not be repeated here.

[0113] S405. Online training is performed on a breast lesion prediction model according to the first loss function and the third loss function. The breast lesion prediction model is used to process and analyze the ultrasound image containing the breast lesion to be analyzed to obtain the type and BI-RADS grade of the BI-RADS feature of the breast lesion.

[0114] In this embodiment, after obtaining the first loss function and the third loss function, the breast lesion prediction model can be trained online based on the first loss function and the third loss function. A target loss function can be determined based on the first loss function and the third loss function, and the breast lesion prediction model can be trained online with the goal of minimizing the target loss function. The target loss function can, for example, be a weighted sum of the first loss function and the third loss function. The initial training process of the breast lesion prediction model based on the training samples in the training set can be referred to step S207, which will not be repeated here.

[0115] The online learning method of the lesion prediction model provided in this embodiment obtains an ultrasound image containing a breast lesion and the BI-RADS grade of the breast lesion; extracts multiple feature subvectors corresponding to the BI-RADS features from the ultrasound image; determines a first loss function corresponding to the multiple feature subvectors; fuses the multiple feature subvectors to obtain a fused feature vector, and determines a third loss function corresponding to the fused feature vector based on the obtained BI-RADS grade; and performs online training on the breast lesion prediction model based on the first loss function and the third loss function. This method realizes online training of the breast lesion prediction model based solely on BI-RADS grade information, so that all ultrasound images containing breast lesions collected in the clinic can be used for online training of the breast lesion prediction model, reducing the cost of constructing training samples, shortening the model update cycle, and simplifying the update operation steps. Moreover, by optimizing and updating the breast lesion prediction model using ultrasound images collected in actual clinical practice, not only can the model be more adapted to the current hospital's data collection style and data distribution status, but also can make the model's output more consistent with the current hospital's diagnostic style. For the relevant descriptions and effects of thyroid TI-RADS, please refer to breast BI-RADS for understanding, which will not be repeated here.

[0116] On the basis of the above embodiment, in order to take into account both classification accuracy and training efficiency, the method provided in this embodiment may further include: performing online training of the breast lesion prediction model for a preset number of times according to the first loss function and the third loss function to obtain a new breast lesion prediction model; determining the classification accuracy of the breast lesion prediction model and the new breast lesion prediction model based on the same test set; when the classification accuracy of the new breast lesion prediction model is higher than the classification accuracy of the breast lesion prediction model, updating the breast lesion prediction model to the new breast lesion prediction model. Wherein, the specific value of the preset number of times can be set according to actual needs. The method provided in this embodiment can improve the efficiency of online learning of the breast lesion prediction model under the premise of ensuring classification accuracy.

[0117] In an optional embodiment, determining the first loss function corresponding to the plurality of feature subvectors may specifically include: determining the first loss function based on the correlation within each feature subvector and the correlation between the plurality of feature subvectors. The correlation may be represented by a correlation matrix.

[0118] In an optional embodiment, extracting multiple feature subvectors corresponding to BI-RADS features from an ultrasound image may include:

[0119] Extracting feature vectors from ultrasound images, dividing the extracted feature vectors into multiple feature sub-vectors, each feature sub-vector corresponding to a BI-RADS feature;

[0120] or,

[0121] Multiple pre-trained feature extraction convolutional neural network models are used to extract multiple feature sub-vectors corresponding to BI-RADS features from ultrasound images. Each feature extraction convolutional neural network model is used to extract a feature sub-vector corresponding to a BI-RADS feature from the ultrasound image.

[0122] This document is described with reference to various exemplary embodiments. However, those skilled in the art will recognize that changes and modifications may be made to the exemplary embodiments without departing from the scope of this document. For example, the various operational steps and components used to perform the operational steps may be implemented in different ways (e.g., one or more steps may be deleted, modified, or incorporated into other steps) depending on the specific application or considering any number of cost functions associated with the operation of the system.

[0123] Additionally, as will be appreciated by those skilled in the art, the principles of this disclosure may be embodied in a computer program product on a computer-readable storage medium pre-loaded with computer-readable program code. Any tangible, non-transitory computer-readable storage medium may be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD-ROMs, DVDs, Blu-ray discs, etc.), flash memory, and / or the like. These computer program instructions may be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing device to form a machine, such that the instructions executed on the computer or other programmable data processing device can generate a device that implements a specified function. These computer program instructions may also be stored in a computer-readable memory, which can instruct the computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the computer-readable memory can form an article of manufacture that includes an implementation device that implements the specified function. The computer program instructions may also be loaded onto a computer or other programmable data processing device, causing the computer or other programmable device to execute a series of operational steps to produce a computer-implemented process, such that the instructions executed on the computer or other programmable device provide the steps for implementing the specified function.

[0124] Although the principles of this invention have been shown in various embodiments, many modifications of structure, arrangement, proportion, elements, materials and components that are particularly suitable for specific environments and operational requirements can be used without departing from the principles and scope of this invention. The above modifications and other changes or amendments are intended to be included within the scope of this invention.

[0125] The foregoing detailed description has been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of this disclosure. Therefore, the present disclosure will be considered in an illustrative rather than a restrictive sense, and all such modifications will be included within its scope. Similarly, the advantages, other advantages and solutions to the problems of the various embodiments have been described above. However, the benefits, advantages, solutions to the problems and any elements that can produce these, or make them more specific, should not be interpreted as critical, required or necessary. The term "comprising" and any other variants used in this article are all non-exclusive inclusions, so that a process, method, article or device that includes a list of elements includes not only these elements, but also other elements that are not explicitly listed or do not belong to the process, method, system, article or device. In addition, the term "coupled" and any other variants used in this article refer to physical connections, electrical connections, magnetic connections, optical connections, communication connections, functional connections and / or any other connections.

[0126] The above examples are used to illustrate the present invention, which are only used to help understand the present invention and are not intended to limit the present invention. Those skilled in the art can make several simple deductions, modifications or substitutions based on the concept of the present invention.

Claims

1. An online learning method for a lesion prediction model, characterized in that: include: Acquiring an ultrasound image containing a breast lesion and a BI-RADS grade of the breast lesion; Extracting a plurality of feature subvectors corresponding to BI-RADS features from the ultrasound image, wherein the BI-RADS features include at least two of a shape feature, a direction feature, an edge feature, an internal echo feature, a rear echo feature, a calcification feature, and a blood flow feature; Determining a target feature subvector having annotation information from the multiple feature subvectors, wherein the annotation information indicates descriptive information of a type of BI-RADS feature of the breast lesion; Determining first loss functions corresponding to the plurality of feature subvectors according to the correlation within each feature subvector and the correlation between the plurality of feature subvectors; Determining a second loss function corresponding to the target feature subvector based on mean square loss, cross entropy loss, or focal loss according to the type of the annotated BI-RADS feature and the type of the BI-RADS feature output by the breast lesion prediction model; fusing the multiple feature sub-vectors to obtain a fused feature vector, and determining a third loss function corresponding to the fused feature vector according to the acquired BI-RADS grade and the BI-RADS grade output by the breast lesion prediction model; The breast lesion prediction model is trained online according to the first loss function, the second loss function and the third loss function. The breast lesion prediction model is used to process and analyze the ultrasound image containing the breast lesion to be analyzed to obtain the type of BI-RADS features and BI-RADS grade of the breast lesion.

2. The method according to claim 1, wherein The method further comprises: Performing online training on the breast lesion prediction model for a preset number of times according to the first loss function, the second loss function, and the third loss function to obtain a new breast lesion prediction model; Determining the classification accuracy of the breast lesion prediction model and the new breast lesion prediction model based on the same test set; When the classification accuracy of the new breast lesion prediction model is higher than the classification accuracy of the breast lesion prediction model, the breast lesion prediction model is updated to the new breast lesion prediction model.

3. The method according to claim 1, wherein The step of extracting a plurality of feature subvectors corresponding to BI-RADS features from the ultrasound image includes: A feature vector is extracted from the ultrasound image, and the extracted feature vector is divided into a plurality of feature sub-vectors according to maximizing the correlation of intra-group features and minimizing the correlation of inter-group features, each feature sub-vector corresponding to a BI-RADS feature.

4. The method according to claim 3, wherein The extracted feature vector is divided into multiple feature sub-vectors, each feature sub-vector corresponding to a BI-RADS feature, including: The extracted feature vector is divided into five feature sub-vectors, corresponding to shape features, direction features, edge features, internal echo features and rear echo features respectively.

5. The method according to claim 1, wherein The step of extracting a plurality of feature subvectors corresponding to BI-RADS features from the ultrasound image includes: Multiple pre-trained feature extraction convolutional neural network models are used to extract multiple feature sub-vectors corresponding to BI-RADS features from the ultrasound image, and each feature extraction convolutional neural network model is used to extract a feature sub-vector corresponding to a BI-RADS feature from the ultrasound image.

6. The method according to claim 1, wherein The fusing the multiple feature sub-vectors to obtain a fused feature vector includes: A fused feature vector is obtained by fusing multiple feature sub-vectors through feature concatenation or feature addition.

7. The method according to claim 1, wherein Determining a target feature subvector having labeled information from the multiple feature subvectors includes: Obtaining diagnostic information of the breast lesion, where the diagnostic information is determined based on a doctor's input or is obtained by the doctor modifying an output of a breast lesion prediction model; Obtaining the type of BI-RADS features of the breast lesion contained in the diagnostic information by performing natural language processing on the diagnostic information; The type of the BI-RADS feature included in the diagnosis information is matched with each of the feature sub-vectors, and the feature sub-vector that matches the type of the BI-RADS feature is determined as the target feature sub-vector.

8. The method according to claim 1, wherein The obtaining of an ultrasound image containing a breast lesion comprises: Transmitting ultrasound to a breast region containing a breast lesion, and receiving ultrasound echoes returned by the breast region to obtain ultrasound echo data, and generating an ultrasound image containing the breast lesion in real time based on the ultrasound echo data; or, A pre-stored ultrasound image containing a breast lesion is obtained from a storage device.

9. An online learning method for a lesion prediction model, characterized in that: include: Acquiring an ultrasound image containing a breast lesion and a BI-RADS grade of the breast lesion; Extracting a plurality of feature subvectors corresponding to BI-RADS features from the ultrasound image, wherein the BI-RADS features include at least two of a shape feature, a direction feature, an edge feature, an internal echo feature, a rear echo feature, a calcification feature, and a blood flow feature; Determining first loss functions corresponding to the plurality of feature subvectors according to the correlation within each feature subvector and the correlation between the plurality of feature subvectors; fusing the multiple feature sub-vectors to obtain a fused feature vector, and determining a third loss function corresponding to the fused feature vector according to the acquired BI-RADS grade and the BI-RADS grade output by the breast lesion prediction model; The breast lesion prediction model is trained online according to the first loss function and the third loss function. The breast lesion prediction model is used to process and analyze the ultrasound image containing the breast lesion to be analyzed to obtain the type and BI-RADS grade of the BI-RADS feature of the breast lesion.

10. The method according to claim 9, wherein The method further comprises: Performing online training on the breast lesion prediction model for a preset number of times according to the first loss function and the third loss function to obtain a new breast lesion prediction model; Determining the classification accuracy of the breast lesion prediction model and the new breast lesion prediction model based on the same test set; When the classification accuracy of the new breast lesion prediction model is higher than the classification accuracy of the breast lesion prediction model, the breast lesion prediction model is updated to the new breast lesion prediction model.

11. The method according to claim 9, wherein The step of extracting a plurality of feature subvectors corresponding to BI-RADS features from the ultrasound image includes: Extracting a feature vector from the ultrasound image, dividing the extracted feature vector into a plurality of feature sub-vectors according to maximizing the correlation of intra-group features and minimizing the correlation of inter-group features, each feature sub-vector corresponding to a BI-RADS feature; or, Multiple pre-trained feature extraction convolutional neural network models are used to extract multiple feature sub-vectors corresponding to BI-RADS features from the ultrasound image, and each feature extraction convolutional neural network model is used to extract a feature sub-vector corresponding to a BI-RADS feature from the ultrasound image.

12. An online learning method for a lesion prediction model, characterized in that: include: Acquiring an ultrasound image containing a thyroid lesion and the TI-RADS grade of the thyroid lesion; Extracting a plurality of feature subvectors corresponding to TI-RADS features from the ultrasound image, wherein the TI-RADS features include at least two of a component feature, an echo feature, a shape feature, an edge feature, and a focal hyperechoic feature; Determine a target feature subvector having annotation information from the multiple feature subvectors, wherein the annotation information indicates descriptive information of a type of the TI-RADS feature of the thyroid lesion; Determining first loss functions corresponding to the plurality of feature subvectors according to the correlation within each feature subvector and the correlation between the plurality of feature subvectors; According to the type of the annotated TI-RADS feature and the type of the TI-RADS feature output by the thyroid lesion prediction model, a second loss function corresponding to the target feature subvector is determined based on mean square loss, cross entropy loss, or focal loss; fusing the multiple feature subvectors to obtain a fused feature vector, and determining a third loss function corresponding to the fused feature vector according to the acquired TI-RADS grade and the TI-RADS grade output by the thyroid lesion prediction model; The thyroid lesion prediction model is trained online according to the first loss function, the second loss function and the third loss function. The thyroid lesion prediction model is used to process and analyze the ultrasound image containing the thyroid lesion to be analyzed to obtain the type and TI-RADS grade of the thyroid lesion's TI-RADS features.

13. The method according to claim 12, wherein: The method further comprises: Performing online training on the thyroid lesion prediction model for a preset number of times according to the first loss function, the second loss function, and the third loss function to obtain a new thyroid lesion prediction model; Determining the classification accuracy of the thyroid lesion prediction model and the new thyroid lesion prediction model based on the same test set; When the classification accuracy of the new thyroid lesion prediction model is higher than the classification accuracy of the thyroid lesion prediction model, the thyroid lesion prediction model is updated to the new thyroid lesion prediction model.

14. The method according to claim 12, wherein: The step of extracting a plurality of feature subvectors corresponding to the TI-RADS features from the ultrasound image includes: A feature vector is extracted from the ultrasound image, and the extracted feature vector is divided into a plurality of feature sub-vectors according to maximizing the correlation of intra-group features and minimizing the correlation of inter-group features, each feature sub-vector corresponding to a TI-RADS feature.

15. The method according to claim 14, wherein The extracted feature vector is divided into multiple feature sub-vectors, each feature sub-vector corresponding to a TI-RADS feature, including: The extracted feature vector is divided into five feature sub-vectors, corresponding to component features, echo features, shape features, edge features, and focal strong echo features, respectively.

16. The method according to claim 12, wherein The step of extracting a plurality of feature subvectors corresponding to the TI-RADS features from the ultrasound image includes: Multiple pre-trained feature extraction convolutional neural network models are used to extract multiple feature sub-vectors corresponding to TI-RADS features from the ultrasound image, and each feature extraction convolutional neural network model is used to extract a feature sub-vector corresponding to a TI-RADS feature from the ultrasound image.

17. The method according to claim 12, wherein The fusing the multiple feature sub-vectors to obtain a fused feature vector includes: A fused feature vector is obtained by fusing multiple feature sub-vectors through feature concatenation or feature addition.

18. The method according to claim 12, wherein Determining a target feature subvector having labeled information from the multiple feature subvectors includes: Obtaining diagnostic information of the thyroid lesion, where the diagnostic information is determined based on a doctor's input or is obtained by the doctor modifying an output of a thyroid lesion prediction model; Obtaining the type of TI-RADS feature of the thyroid lesion contained in the diagnostic information by performing natural language processing on the diagnostic information; The type of the TI-RADS feature included in the diagnosis information is matched with each of the feature sub-vectors, and the feature sub-vector that matches the type of the TI-RADS feature is determined as the target feature sub-vector.

19. The method according to claim 12, wherein The obtaining of an ultrasound image containing a thyroid lesion comprises: Transmitting ultrasound to a thyroid region containing a thyroid lesion, and receiving ultrasound echoes returned by the thyroid region to obtain ultrasound echo data, and generating an ultrasound image containing the thyroid lesion in real time based on the ultrasound echo data; or, A pre-stored ultrasound image containing a thyroid lesion is acquired from a storage device.

20. An online learning method for a lesion prediction model, characterized in that: include: Acquiring an ultrasound image containing a thyroid lesion and the TI-RADS grade of the thyroid lesion; Extracting a plurality of feature subvectors corresponding to TI-RADS features from the ultrasound image, wherein the TI-RADS features include at least two of a component feature, an echo feature, a shape feature, an edge feature, and a focal hyperechoic feature; Determining first loss functions corresponding to the plurality of feature subvectors according to the correlation within each feature subvector and the correlation between the plurality of feature subvectors; fusing the multiple feature subvectors to obtain a fused feature vector, and determining a third loss function corresponding to the fused feature vector according to the acquired TI-RADS grade and the TI-RADS grade output by the thyroid lesion prediction model; The thyroid lesion prediction model is trained online according to the first loss function and the third loss function. The thyroid lesion prediction model is used to process and analyze the ultrasound image containing the thyroid lesion to be analyzed to obtain the type and TI-RADS grade of the thyroid lesion's TI-RADS features.

21. The method according to claim 20, wherein The method further comprises: Performing online training on the thyroid lesion prediction model for a preset number of times according to the first loss function and the third loss function to obtain a new thyroid lesion prediction model; Determining the classification accuracy of the thyroid lesion prediction model and the new thyroid lesion prediction model based on the same test set; When the classification accuracy of the new thyroid lesion prediction model is higher than the classification accuracy of the thyroid lesion prediction model, the thyroid lesion prediction model is updated to the new thyroid lesion prediction model.

22. The method according to claim 20, wherein The step of extracting a plurality of feature subvectors corresponding to the TI-RADS features from the ultrasound image includes: Extracting a feature vector from the ultrasound image, dividing the extracted feature vector into a plurality of feature subvectors according to maximizing the correlation of intra-group features and minimizing the correlation of inter-group features, each feature subvector corresponding to a TI-RADS feature; or, Multiple pre-trained feature extraction convolutional neural network models are used to extract multiple feature sub-vectors corresponding to TI-RADS features from the ultrasound image, and each feature extraction convolutional neural network model is used to extract a feature sub-vector corresponding to a TI-RADS feature from the ultrasound image.

23. An ultrasonic imaging device, characterized in that: include: Ultrasound probe; a transmitting circuit, configured to output a corresponding transmitting sequence to the ultrasonic probe according to a set mode, so as to control the ultrasonic probe to transmit corresponding ultrasonic waves; a receiving circuit, configured to receive the ultrasonic echo signal output by the ultrasonic probe and output ultrasonic echo data; A display for outputting visual information; A processor, configured to execute the online learning method for the lesion prediction model according to any one of claims 1 to 22.

Images